Power Engineering Safety Monitoring Method and System Based on Deep Learning

Abstract

The power engineering safety monitoring method and system based on deep learning provided by the present invention relate to the technical field of deep learning. In the present invention, first, the target personnel trajectory data is mined to form a target personnel trajectory vector, and the target personnel monitoring image is mined to form a target personnel image vector; secondly, the target personnel image vector is aggregated into the target personnel trajectory vector to form a target personnel global vector; then, the semantic vector parameters of a specified size in the target personnel global vector are hidden so that perturbations are imposed on the target personnel global vector to form a target personnel perturbation vector; further, the perturbations of the target personnel perturbation vector are removed to form a target personnel restored vector; finally, a target safety monitoring result is determined based on the target personnel restored vector. Based on the above, the problem of relatively low reliability of power engineering safety monitoring existing in the prior art can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning, and in particular, to a method and system for power engineering safety monitoring based on deep learning. Background Art

[0002] With the rapid development of power engineering, the safety management problem of power construction has gradually become more complex and urgent. The power engineering field often involves complex equipment and high-voltage power systems, which pose a great threat to the safety of personnel during operation. Therefore, the safety monitoring of power engineering has become an important part of ensuring project safety. Among them, traditional safety monitoring systems mostly rely on fixed cameras and manual intervention, making it difficult to achieve accurate safety warnings, prone to missing some abnormal behaviors or potential safety hazards, and unable to fully understand the behavior patterns of targets. For example, in high-voltage power areas, the behaviors of staff may be somewhat ambiguous, and these behaviors are difficult to accurately identify and judge by traditional methods, resulting in relatively low reliability of safety monitoring. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method and system for power engineering safety monitoring based on deep learning to improve the problem of relatively low reliability of power engineering safety monitoring existing in the prior art.

[0004] To achieve the above purpose, the present invention adopts the following technical solutions:

[0005] A method for power engineering safety monitoring based on deep learning, comprising:

[0006] Determine the target personnel trajectory data and target personnel monitoring images of the target power engineering area, and mine the target personnel trajectory data to form a target personnel trajectory vector, and mine the target personnel monitoring images to form a target personnel image vector. Among them, during the mining process of the target personnel trajectory data, at least the embedding processing of the word embedding model is included, and during the mining process of the target personnel monitoring images, at least the convolution processing of the convolutional network is included;

[0007] Aggregate the target personnel image vector into the target personnel trajectory vector to form a target personnel global vector, where the target personnel global vector includes multiple semantic vector parameters, and the target personnel global vector is used to represent the global semantic information in the image dimension and the trajectory dimension;

[0008] Hide the semantic vector parameters of a specified size in the global vector of the target person, so as to apply perturbations to the global vector of the target person to form a perturbed vector of the target person, where the specified size is used to reflect the number of the hidden semantic vector parameters, and the hidden semantic vector parameters are determined after evaluating the possibility of each semantic vector parameter in the global vector of the target person being hidden;

[0009] Remove the perturbations from the perturbed vector of the target person to form a restored vector of the target person, where the perturbation removal is achieved by using the context-related semantic information in the perturbed vector of the target person, so as to achieve the interactive fusion of the context-related semantic information during the process of removing the perturbations;

[0010] Determine the target safety monitoring result based on the restored vector of the target person, where the process of determining the target safety monitoring result at least includes a fully connected process for mapping the restored vector of the target person to the probability of whether there is abnormal construction, and the target safety monitoring result is used to reflect whether there is abnormal construction in the target power engineering area.

[0011] In a preferred choice of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, the step of aggregating the target person image vector into the target person trajectory vector to form a global vector of the target person includes:

[0012] Perform at least one segmentation on the target person trajectory vector to form at least one segmentation position;

[0013] Perform at least one segmentation on the target person image vector to form at least two local person image vectors, or use the target person image vector as the local person image vector, where the number of the formed local person image vectors is equal to the number of the segmentation positions;

[0014] Configure image semantic identifiers at both the head and tail ends of each of the local person image vectors to form new local person image vectors;

[0015] Stitch a corresponding new local person image vector at each of the segmentation positions to form a global vector of the target person.

[0016] In a preferred choice of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, the step of hiding the semantic vector parameters of a specified size in the global vector of the target person, so as to apply perturbations to the global vector of the target person to form a perturbed vector of the target person includes:

[0017] Determine the hidden coordinates of the global vector of the target person through the interference forward processing network, and output a coordinate set that matches the proportion of the quantity characterized by the specified size, where the interference forward processing network is jointly trained with the interference backward processing network, and the interference backward processing network is used to remove the perturbation of the target person perturbation vector;

[0018] Hide the semantic vector parameters corresponding to the coordinate set in the global vector of the target person, so that a perturbation is applied to the global vector of the target person to form a target person perturbation vector, where the hidden semantic vector parameters are replaced with random values or specific values, and the specific values include 0.

[0019] In a preferred choice of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, the step of determining the hidden coordinates of the global vector of the target person through the interference forward processing network and outputting a coordinate set that matches the proportion of the quantity characterized by the specified size includes:

[0020] Load the global vector of the target person to be loaded into the interference forward processing network;

[0021] Determine the coordinate distribution parameters corresponding to the global vector of the target person, where the coordinate distribution parameters are used to reflect the coordinates of multiple semantic vector parameters in the global vector of the target person;

[0022] Concatenate the global vector of the target person and the coordinate distribution parameters to form a concatenated vector;

[0023] Perform saliency mining on the concatenated vector to form a saliency mining vector. The interference forward processing network includes a query matrix, a key matrix, and a value matrix. The process of saliency mining includes: multiplying the query matrix, the key matrix, and the value matrix with the concatenated vector respectively to obtain corresponding query vectors, key vectors, and value vectors, calculating the dot product between the query vector and the transposed result of the key vector, and based on this dot product, performing weighted summation on the value vectors to obtain the corresponding saliency mining vector;

[0024] Evaluate the hidden coordinates based on the saliency mining vector to form a hidden coordinate evaluation parameter. The evaluation process includes: mapping the saliency mining vector through the fully connected network layer in the interference forward processing network to obtain a fully connected vector with the same size as the global vector of the target person, and outputting the fully connected vector through the softmax function to obtain a probability distribution as the hidden coordinate evaluation parameter;

[0025] Generate a set of coordinates that matches the proportion of the quantity characterized by the specified size according to the hidden coordinate evaluation parameter, where the hidden coordinate evaluation parameter is used to reflect the likelihood of each of the semantic vector parameters being hidden.

[0026] In a preferred option of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, the step of removing the perturbation of the target person perturbation vector to form a target person restoration vector includes:

[0027] Remove the perturbation of the target person perturbation vector through an interference backward processing network, and output the perturbation removal data of the current processing stage, where the perturbation is applied to the target person global vector through an interference forward processing network;

[0028] Remove the perturbation of the perturbation removal data of the current processing stage through the interference backward processing network, and when the current processing stage belongs to the target processing stage, output the target person restoration vector.

[0029] In a preferred option of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, the step of removing the perturbation of the target person perturbation vector through an interference backward processing network and outputting the perturbation removal data of the current processing stage includes:

[0030] Load the target person perturbation vector to be loaded into the interference backward processing network;

[0031] Calculate the first mapping parameter, the second mapping parameter, and the third mapping parameter of each of the semantic vector parameters;

[0032] Poll multiple semantic vector parameters, and perform a dot product operation on the first mapping parameter of the currently polled semantic vector parameter and the second mapping parameters of each of the other semantic vector parameters, and output the significance parameter between the currently polled semantic vector parameter and each of the other semantic vector parameters;

[0033] Poll multiple semantic vector parameters, and use the respective significance parameters of the currently polled semantic vector parameter as weight coefficients to fuse the third mapping parameters of each of the other semantic vector parameters, and output the significance fusion parameter of the currently polled semantic vector parameter, where the fusion process refers to using the significance parameter as a weight parameter to perform a weighted sum calculation on the third mapping parameters of each semantic vector parameter;

[0034] Merge the significance fusion parameters of multiple semantic vector parameters to obtain the perturbation removal data of the current processing stage after perturbation removal.

[0035] In a preferred selection of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, the step of calculating the first mapping parameter, the second mapping parameter, and the third mapping parameter of each of the semantic vector parameters includes:

[0036] Determine the first transformation parameter, the second transformation parameter, and the third transformation parameter of each of the semantic vector parameters from the interference backward processing network;

[0037] For each of the semantic vector parameters, perform a multiplication operation on the semantic vector parameter and the first transformation parameter to output the first mapping parameter of the semantic vector parameter, perform a multiplication operation on the semantic vector parameter and the second transformation parameter to output the second mapping parameter of the semantic vector parameter, and perform a multiplication operation on the semantic vector parameter and the third transformation parameter to output the third mapping parameter of the semantic vector parameter.

[0038] In a preferred selection of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, before the step of hiding the semantic vector parameters of a specified size in the target person global vector so as to apply a perturbation to the target person global vector to form a target person perturbation vector, the power engineering safety monitoring method based on deep learning further includes:

[0039] Determine a target parameter based on the target processing stage, where the target parameter is a randomly generated parameter less than the target processing stage;

[0040] Obtain the specified size based on the ratio between the target parameter and the target processing stage, where the specified size is used to reflect the proportion of the number of semantic vector parameters hidden in the target person global vector.

[0041] In a preferred selection of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, before the step of determining the target person trajectory data and the target person monitoring image of the target power engineering area, and mining the target person trajectory data to form a target person trajectory vector, and mining the target person monitoring image to form a target person image vector, the power engineering safety monitoring method based on deep learning further includes:

[0042] Determine the training person trajectory data and the corresponding training person monitoring image, and mine the training person trajectory data to form a training person trajectory vector, and mine the training person monitoring image to form a training person image vector;

[0043] After aggregating the training personnel image vectors into the training personnel trajectory vectors, a perturbation is applied using a forward interference processing network to form a training personnel perturbation vector, and the training personnel perturbation vector is subjected to perturbation removal using a backward interference processing network to form a training personnel restoration vector;

[0044] Based on the training personnel restoration vector, a training safety monitoring result is determined, and based on the error between the training safety monitoring result and the corresponding safety label data, the forward interference processing network and the backward interference processing network are trained to form a trained forward interference processing network and a trained backward interference processing network.

[0045] On this basis, the present invention also provides a power engineering safety monitoring system based on deep learning, including:

[0046] A memory for storing a computer program;

[0047] A processor connected to the memory for executing the computer program stored in the memory to implement the above-mentioned power engineering safety monitoring method based on deep learning.

[0048] The power engineering safety monitoring method and system based on deep learning provided by the present invention, first, mines the target personnel trajectory data to form a target personnel trajectory vector, and mines the target personnel monitoring images to form a target personnel image vector; secondly, aggregates the target personnel image vectors into the target personnel trajectory vector to form a target personnel global vector; then, hides the semantic vector parameters of a specified size in the target personnel global vector to apply a perturbation to the target personnel global vector to form a target personnel perturbation vector; further, performs perturbation removal on the target personnel perturbation vector to form a target personnel restoration vector; finally, determines a target safety monitoring result based on the target personnel restoration vector. Based on the above, on the one hand, by mining the trajectory data and monitoring images, potential semantic information can be mined, and compared with traditional data analysis techniques, potential behavioral semantic features can be captured, thus, the accuracy of judgment can be improved to a certain extent. In addition, after aggregating to form the target personnel global vector, through further perturbation application and removal, the fusion of potential semantic information in two dimensions of trajectory and image can be realized in this process (the semantics of trajectory information is simple but continuous, and the semantics of image information is rich but local, so through fusion, mutual reinforcement can be achieved, that is, both the richness and continuity of semantics are taken into account), so as to obtain a target personnel restoration vector with relatively higher representation accuracy, and the reliability of the determined target safety monitoring result can be further improved, and the problem of relatively low reliability of power engineering safety monitoring existing in the prior art can be improved. Description of the Drawings

[0049] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the following specifically gives preferred embodiments and, in conjunction with the accompanying drawings, provides a detailed description as follows.

[0050] Figure 1 It is a structural block diagram of a power engineering safety monitoring system based on deep learning provided by an embodiment of the present invention.

[0051] Figure 2 It is a schematic block diagram of a power engineering safety monitoring device based on deep learning provided by an embodiment of the present invention.

[0052] Figure 3 It is a schematic flow diagram of a power engineering safety monitoring method based on deep learning provided by an embodiment of the present invention.

[0053] Figure 4 It is a schematic diagram of convolution processing based on edge padding provided by an embodiment of the present invention.

[0054] Figure 5 It is a schematic diagram of the mining process of the target person image vector provided by an embodiment of the present invention.

[0055] Figure 6 It is a schematic diagram of the segmentation and extraction of vectors provided by an embodiment of the present invention.

[0056] Figure 7 It is a schematic diagram of vector aggregation provided by an embodiment of the present invention. Detailed Embodiments

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of them. Usually, the components of the embodiments of the present invention described and illustrated herein can be arranged and designed in various different configurations.

[0058] Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0059] As Figure 1 shown, an embodiment of the present invention provides a power engineering safety monitoring system based on deep learning. Among them, the power engineering safety monitoring system based on deep learning may include a memory, a processor, and a power engineering safety monitoring device based on deep learning.

[0060] Specifically, the memory and the processor are electrically connected directly or indirectly to achieve data transmission or interaction. For example, the memory and the processor can be electrically connected through one or more communication buses or signal lines. The power engineering safety monitoring device based on deep learning includes at least one software function module stored in the memory in the form of software or firmware. The processor is used to execute the executable computer programs stored in the memory, such as the software function modules and computer programs included in the power engineering safety monitoring device based on deep learning, to implement the power engineering safety monitoring method based on deep learning provided in the embodiments of the present invention.

[0061] Optionally, the memory can be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0062] Optionally, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0063] Optionally, in a specific implementation manner, the power engineering safety monitoring device based on deep learning may include: Figure 2 a semantic mining module, configured to determine target personnel trajectory data and target personnel monitoring images of a target power engineering area, and mine the target personnel trajectory data to form a target personnel trajectory vector, and mine the target personnel monitoring images to form a target personnel image vector;

[0064]

[0065] ​A vector aggregation module, configured to aggregate the target person image vectors into the target person trajectory vectors to form target person global vectors, where the target person global vectors include multiple semantic vector parameters;

[0066] A perturbation application module, configured to hide the semantic vector parameters of a specified size in the target person global vectors, so that perturbations are applied to the target person global vectors to form target person perturbation vectors;

[0067] A perturbation removal module, configured to remove the perturbations from the target person perturbation vectors to form target person restoration vectors;

[0068] An anomaly analysis module, configured to determine a target security monitoring result based on the target person restoration vectors, where the target security monitoring result is used to reflect whether there is abnormal construction in the target power engineering area.

[0069] It can be understood that Figure 1 The structure shown is only for illustration, and the power engineering security monitoring system based on deep learning may further include more or fewer components than those shown in Figure 1 or have a different configuration from that shown in Figure 1 For example, it may further include a communication unit for information interaction with other devices. In addition, the power engineering security monitoring system based on deep learning may be an electronic device, such as a server or a cluster composed of multiple servers.

[0070] Combined with Figure 3 , an embodiment of the present invention further provides a power engineering security monitoring method based on deep learning that can be applied to the above-mentioned power engineering security monitoring system based on deep learning. Among them, the method steps defined by the processes related to the power engineering security monitoring method based on deep learning can be implemented by the power engineering security monitoring system (hereinafter simply referred to as the monitoring system). The following will Figure 3 elaborate in detail on the specific processes shown.

[0071] Step S110, determine the target person trajectory data and target person monitoring images of the target power engineering area, and mine the target person trajectory data to form target person trajectory vectors, and mine the target person monitoring images to form target person image vectors.

[0072] In an embodiment of the present invention, the monitoring system can determine the target personnel trajectory data of the target power engineering area (exemplarily, by setting positioning devices, such as RFID tags, etc., on the work clothes of each staff member working in the target power engineering area to collect and form the trajectory data of each staff member) and the target personnel monitoring images (exemplarily, by devices such as cameras set in the target power engineering area to collect and form images), and further, mine the target personnel trajectory data to form a target personnel trajectory vector, and mine the target personnel monitoring images to form a target personnel image vector. Among them, the mining process of the target personnel trajectory data at least includes the embedding process of the word embedding model, and the mining process of the target personnel monitoring images at least includes the convolution process of the convolutional network. Exemplarily, the target personnel trajectory data can be embedded through a word embedding model to form a corresponding target personnel trajectory vector. For example, the word embedding vectors of each staff member can be spliced to form a target personnel trajectory vector. In addition, the target personnel monitoring images can be convolved through a convolutional network to form a corresponding target personnel image vector (such as Figure 4As shown, the size of the target person image vector formed by convolution can be the same as the size of the target person monitoring image. That is to say, for each pixel value in the target person monitoring image, a region can be determined according to the size of the convolution kernel of the convolution network (for example, the size can be 3*3). Then, the pixel values in this region are weighted and summed with the parameters in the convolution kernel. In this way, a convolution parameter corresponding to the center point can be obtained. Thus, for each pixel value in the target person monitoring image, a corresponding convolution parameter can be obtained. Then, the pixel value can be replaced with the corresponding convolution parameter to form the target person image vector. It should be noted that for the pixel points at the edges, edge padding can be performed (for example, all filled with 0). For example, when there are multiple target person monitoring images (formed by multiple cameras monitoring multiple local areas, or one camera monitoring one local area for multiple time periods, which can be specifically selected according to actual needs), the convolution vectors (matrices) corresponding to each target person monitoring image can be directly expanded or mapped to row vectors or column vectors through a fully connected network. Then, the row vectors or column vectors corresponding to each target person monitoring image are concatenated to form the corresponding target person image vector. In addition, for the embedding processing of the target person trajectory data, assuming that part of the content in the target person trajectory data is "the trajectory coordinates of the staff are in sequence", then, the "the trajectory coordinates of the staff are in sequence" can be tokenized first to obtain the corresponding words "work", "person", "of", "trajectory", "coordinates", "in sequence", "are", respectively. Then, word embedding processing is performed on each word to obtain word vectors, such as:

[0073] Word vector of "work":

[0074] [0.3, 0.3, -0.1, 0.2, -0.4, 0.7, -0.8, 0.2,..., 0.9];

[0075] Word vector of "person":

[0076] [0.1, 0.3, -0.2, 0.5, -0.4, 0.6, -0.1, 0.2,..., 0.3];

[0077] Word vector of "of":

[0078] [0.5, -0.1, 0.2, -0.3, 0.4, 0.1, 0.3, -0.2,..., 0.1];

[0079] Word vector of "trajectory":

[0080] [0.4, -0.2, 0.1, 0.3, -0.5, 0.2, -0.4, 0.6,..., 0.5];

[0081] The word vector of "coordinate":

[0082] [-0.3, 0.1, 0.4, 0.2, 0.3, -0.1, 0.4, 0.5,..., -0.2];

[0083] The word vector of "in sequence":

[0084] [0.2, -0.3, 0.4, -0.2, 0.1, 0.3, -0.1, 0.4,..., 0.6];

[0085] The word vector of "is":

[0086] [0.1, 0.2, -0.3, 0.4, -0.1, 0.5, -0.3, 0.2,..., 0.3].

[0087] Then, the word vectors of the above-mentioned various words and the word vectors of other words can be concatenated to form the word embedding vector of the corresponding staff member, as follows:

[0088] .

[0089] Step S120: Aggregate the target person image vector into the target person trajectory vector to form a target person global vector.

[0090] In the embodiment of the present invention, after forming the target person image vector and the target person trajectory vector, the monitoring system can aggregate the target person image vector into the target person trajectory vector to form a target person global vector. In this way, the target person global vector can represent the global semantic information in the image dimension and the trajectory dimension. Among them, the target person global vector includes multiple semantic vector parameters, such as M*N.

[0091] Step S130: Hide the semantic vector parameters of the specified size in the target person global vector, so that perturbations are applied to the target person global vector to form a target person perturbation vector.

[0092] In an embodiment of the present invention, after forming the global vector of the target person, the monitoring system may hide the semantic vector parameters of a specified size in the global vector of the target person, so as to apply a perturbation to the global vector of the target person to form a perturbed vector of the target person. The specified size is used to reflect the number of the hidden semantic vector parameters, and the hidden semantic vector parameters are determined after evaluating the possibility of each semantic vector parameter in the global vector of the target person being hidden, so as to improve the reliability of perturbation application. It should be noted that both the trajectory data and the monitoring images have certain distortions in representing the real situation, that is, there are perturbations or noises. Therefore, by applying perturbations, the robustness and the effective representation of the real situation can be increased. Exemplarily, the specified size may be a determined ratio (such as 35%, 40%, etc.), or an arbitrary ratio, or a ratio configured according to actual requirements. For example, it is configured based on the number of semantic vector parameters in the global vector of the target person, and this ratio determines how many semantic vector parameters will be hidden. According to this specified size, a hidden matrix having the same size as the global vector of the target person is randomly generated. The parameters in this hidden matrix may be 0 or 1, where 1 indicates that the semantic vector parameter at the corresponding position will be hidden (i.e., a perturbation is applied), and 0 indicates remaining unchanged (alternatively, it may be the opposite). And the ratio of the parameters equal to 1 in this hidden matrix is equal to the specified size. Then, the generated hidden matrix is applied to the global vector of the target person. For the positions where the value in the hidden matrix is 1, the corresponding semantic vector parameters are replaced with random values or specific values (such as 0, etc.), or perturbation identifiers may be added to these corresponding semantic vector parameters. At this time, these semantic vector parameters containing perturbation identifiers do not contain any valid semantic information, and the vector after the hiding process is the perturbed vector of the target person. For example, if the specified size is 20%, then 20% of the semantic vector parameters in the global vector of the target person will be hidden, that is, 20% of the elements in the corresponding hidden matrix are 1. Assume that the global vector of the target person contains 10 semantic vector parameters [X0, X1, X2, X3, X4, X5, X6, X7, X8, X9]. Based on the specified size of 20%, a hidden matrix having the same size as the global vector of the target person can be randomly generated, such as [0, 1, 0, 0, 0, 0, 1, 0, 0, 0]. Then, for the positions where the value in the hidden matrix is 1, the corresponding semantic vector parameters in the global vector of the target person are replaced with 0, that is, [X0, 0, X2, X3, X4, X5, 0, X7, X8, X9] is obtained.

[0093] Step S140, perform perturbation removal on the perturbed vector of the target person to form a restored vector of the target person.

[0094] In an embodiment of the present invention, after obtaining the target person perturbation vector, the monitoring system can remove the perturbation from the target person perturbation vector to form a target person restoration vector. Since the applied perturbation information does not have an effective semantic representation function, it still needs to be removed. Thus, the perturbation can be removed by utilizing the context-related semantic information in the target person perturbation vector, so that the context-related semantic information can be interactively fused during the perturbation removal process, thereby obtaining a target person restoration vector with better representation ability. In this way, interactive fusion is performed during the perturbation application and removal processes. In the case of realizing the interactive fusion of context-related semantic information, the overfitting problem caused by directly performing the interactive fusion of context-related semantic information on the target person global vector can also be avoided. Moreover, since the perturbation information is removed by mining the context-related semantic information, the irrelevant perturbation information can be effectively removed, and other useless information that is originally irrelevant in the target person global vector can also be removed, which can further improve the semantic representation accuracy of the target person restoration vector.

[0095] Step S150, determine a target safety monitoring result based on the target person restoration vector.

[0096] In an embodiment of the present invention, after forming the target person restoration vector, the monitoring system can determine a target safety monitoring result based on the target person restoration vector. Among them, the target safety monitoring result is used to reflect whether there is abnormal construction in the target power engineering area. Exemplarily, a fully connected process can be performed on the target person restoration vector, so that the target person restoration vector can be mapped to a vector of a target size, such as 1*2, that is, including two vector parameters. Then, through a classification function such as softmax, this vector can be mapped to obtain a probability distribution of size 1*2, that is, the probability of the existence of abnormal construction and the probability of the non-existence of abnormal construction, and the situation corresponding to the larger value is taken as the target safety monitoring result. For example, in some dangerous areas, there are a large number of staff trajectories, indicating that there may be abnormalities. Then, if the content in the monitoring image shows that the work content involves wearing safety equipment and construction tools, it means that normal construction is being carried out, so it does not belong to an abnormal situation.

[0097] Exemplarily, the target person restoration vector can be a vector of size 1*n, such as X = (x1, x2,..., x n), during the process of fully connected processing, the target person reduction vector can be multiplied by a weight parameter distribution of size n*2 in matrix multiplication to obtain a vector of size 1*2 (such as y in the following matrix multiplication formula), and then, this vector can be added to a bias vector of size 1*2 to complete the fully connected processing. Then, through classification functions such as softmax, the added vector is mapped to obtain a probability distribution of size 1*2.

[0098] Among them, the weight parameter distribution of n*2 can be expressed as:

[0099] W = .

[0100] Among them, each parameter in W can be equal to 0 in the initial stage, and then, during the training process, corresponding updates can be made to obtain the final specific values.

[0101] The bias vector of size 1*2 can be expressed as:

[0102] B = (b1, b2).

[0103] Among them, b1 and b2 can also be equal to 0 in the initial stage, and then, during the training process, corresponding updates can be made to obtain the final specific values.

[0104] In addition, the above matrix multiplication is as follows:

[0105] .

[0106] Based on the above, on the one hand, by mining trajectory data and monitoring images, potential semantic information can be mined. Compared with traditional data analysis techniques, potential behavioral semantic features can be captured. Therefore, the accuracy of judgment can be improved to a certain extent. In addition, after aggregating to form the global vector of the target person, through further perturbation application and removal, the fusion of potential semantic information in the two dimensions of trajectory and image can be realized during this process (the semantics of trajectory information are simple but continuous, and the semantics of image information are rich but local. Therefore, through fusion, mutual reinforcement can be achieved, that is, both richness and continuity of semantics are considered), so as to obtain a target person reduction vector with relatively higher representation accuracy, further improving the reliability of the determined target security monitoring result and improving the problem of relatively low reliability of power engineering security monitoring existing in the prior art.

[0107] It should be further noted that for step S110 in the above embodiments, in the above step S110, the specific process of mining the target personnel monitoring image to form the target personnel image vector is not limited and can be selected according to actual application requirements.

[0108] For example, in a specific implementation manner, in order to improve the semantic representation reliability of the mined target personnel image vector, the above step S110 may further include the following contents:

[0109] In the first step, the target personnel monitoring image can be subjected to convolution processing to obtain a monitoring image convolution vector corresponding to the target personnel monitoring image, as shown in Figure 5 shown;

[0110] In the second step, the target personnel monitoring image can be subjected to contour extraction (for example, the Canny algorithm can be used for contour extraction) to obtain a corresponding contour image. Among them, the value corresponding to the pixel points belonging to the contour in the contour image can be 1 (or, in other embodiments, it can also be 255), and the value corresponding to the pixel points not belonging to the contour can be 0;

[0111] In the third step, the contour image can be subjected to convolution processing to obtain a contour image convolution vector corresponding to the contour image, where the contour image convolution vector and the monitoring image convolution vector can have the same size;

[0112] In the fourth step, based on the contour image convolution vector, the monitoring image convolution vector can be enhanced to obtain the target personnel image vector.

[0113] Regarding the above enhancement processing, in a specific implementation manner, it may include the following contents:

[0114] In the first step, the contour image can be segmented to form at least one contour segmentation image. The segmentation principle can be to obtain the largest possible number of contour segmentation images, and on this basis, it is necessary to ensure that the pixel points on the same contour are all in one contour segmentation image;

[0115] In the second step, for each of the contour segmentation images, a first local vector with the same position coordinates as the contour segmentation image can be extracted from the monitoring image convolution vector, and a second local vector with the same position coordinates as the contour segmentation image can be extracted from the contour image convolution vector, as shown in Figure 6 , where the target personnel monitoring image, the monitoring image convolution vector, the contour image, and the contour image convolution vector all have the same size;

[0116] In the third step, for each of the second local vectors, based on the second local vector, cross-attention processing is performed on the corresponding first local vector to obtain a first local enhanced vector corresponding to the second local vector. Further, based on the distribution relationship between the corresponding contour segmentation images, the first local enhanced vectors corresponding to each of the second local vectors are concatenated to form a concatenated local enhanced vector. Then, the concatenated local enhanced vector and the monitoring image convolution vector are added or averaged to obtain the target person image vector.

[0117] It should be further noted that for step S120 in the above embodiment, in the above step S120, the specific process of aggregating the target person image vector into the target person trajectory vector is not limited and can be selected according to actual application requirements.

[0118] For example, in a specific implementation manner, to improve the efficiency of vector aggregation, step S120 above may include the following: concatenating the target person image vector and the target person trajectory vector head to tail to form a target person global vector.

[0119] For another example, in another specific implementation manner, to improve the reliability of vector aggregation, step S120 above may further include the following (in combination with Figure 7 ):

[0120] In the first step, the target person trajectory vector can be segmented at least once to form at least one segmentation position; exemplarily, the target person trajectory vector can be first unfolded into a row vector (or the target person trajectory vector obtained in the previous step is a row vector), such as [Y0, Y1, Y2, Y3, Y4, Y5, Y6, Y7, Y8, Y9]. For example, by performing one segmentation, a segmentation position can be formed at the position between Y4 and Y5;

[0121] In the second step, the target person image vector can be segmented at least once to form at least two local person image vectors, or the target person image vector can be used as the local person image vector, where the number of formed local person image vectors is equal to the number of segmentation positions. For example, one segmentation position and one local person image vector are formed;

[0122] In the third step, image semantic identifiers can be configured at both the head and tail ends of each of the local person image vectors to form new local person image vectors. For example, the image semantic identifier at the head end is start, and the image semantic identifier at the tail end is finish, such as [start, U0, U1, U2, U3, U4, finish]; exemplarily, in other implementation manners, image semantic identifiers may not be configured.

[0123] In the fourth step, a corresponding new local personnel image vector can be spliced at each of the segmentation positions to form a target personnel global vector, such as [Y0, Y1, Y2, Y3, Y4, start, U0, U1, U2, U3, U4, finish, Y5, Y6, Y7, Y8, Y9].

[0124] It should be further noted that for step S130 in the above embodiment, in the above step S130, the specific process of hiding the semantic vector parameters of the specified size in the target personnel global vector is not limited and can be selected according to actual application requirements.

[0125] For example, in a specific implementation manner, the semantic vector parameters of the target size can be randomly selected from the target personnel global vector for hiding.

[0126] Again, for example, in another specific implementation manner, in order to improve the reliability of hiding semantic vector parameters, the above step S130 can further include step S131 and step S132, and the specific implementation processes of each step are as follows.

[0127] Step S131, determine the hidden coordinates of the target personnel global vector through an interference forward processing network, and output a coordinate set that matches the quantity ratio represented by the specified size.

[0128] In the embodiment of the present invention, the hidden coordinates of the target personnel global vector can be determined through an interference forward processing network, and a coordinate set that matches the quantity ratio represented by the specified size can be output, that is, the coordinates to be hidden are determined through the interference forward processing network, so that the reliability of the determined hidden coordinates is relatively high. Among them, the interference forward processing network is jointly trained with an interference backward processing network, and the interference backward processing network is used to remove the perturbation of the target personnel perturbation vector. It can be understood that both the interference forward processing network and the interference backward processing network belong to neural networks and can be trained together with the word embedding model and convolutional network used to execute step S110.

[0129] Step S132, hide the semantic vector parameters corresponding to the coordinate set in the target personnel global vector, so that a perturbation is applied to the target personnel global vector to form a target personnel perturbation vector.

[0130] In an embodiment of the present invention, after obtaining the coordinate set, the semantic vector parameters corresponding to the coordinate set in the global vector of the target person can be hidden, so that perturbations are imposed on the global vector of the target person to form a perturbed vector of the target person. Specifically, the hiding method can refer to the relevant descriptions above and will not be elaborated here one by one.

[0131] It should be further noted that for the above step S131, in the above step S131, the specific process of determining the hidden coordinates of the global vector of the target person through the interference forward processing network is not limited and can be selected according to actual needs.

[0132] For example, in a specific implementation manner, since the global vector of the target person is a semantic vector aggregated from semantic vectors of different dimensions, the interference forward processing network can assign a weight to each semantic vector parameter in the global vector of the target person. This weight can reflect the importance of different semantic vector parameters for predicting hidden coordinates. By adjusting these weights, it helps the interference forward processing network balance the impact of hiding semantic vector parameters of different dimensions on subsequent processing. Based on this, the above step S131 can further include the following content:

[0133] The first step is to load the global vector of the target person into the interference forward processing network, that is, subsequent processing can be performed through the interference forward processing network.

[0134] The second step is to determine the coordinate distribution parameters corresponding to the global vector of the target person. The coordinate distribution parameters are used to reflect the coordinates of multiple semantic vector parameters in the global vector of the target person. The size of the coordinate distribution parameters can be the same as the size of the global vector of the target person. In this way, each parameter in the coordinate distribution parameters is used to reflect the coordinate of a semantic vector parameter in the global vector of the target person. Specifically, a coordinate value can be mapped to a word vector, and then the mean or maximum value of the parameters in the word vector can be calculated as a parameter corresponding to the coordinate value, thereby forming the corresponding coordinate distribution parameters.

[0135] The third step is to cascade the global vector of the target person and the coordinate distribution parameters to form a cascaded vector. For example, the coordinate distribution parameters can be connected to the backend of the global vector of the target person, that is, concatenated to obtain a large-sized cascaded vector.

[0136] In the fourth step, the cascade vector can be subjected to saliency mining to form a saliency mining vector. Exemplarily, the interference forward processing network can include a query matrix, a key matrix, and a value matrix. Then, the query matrix, the key matrix, and the value matrix can be multiplied by the cascade vector respectively to obtain corresponding query vectors, key vectors, and value vectors. After that, the dot product between the query vector and the transposed result of the key vector can be calculated. Finally, based on this dot product, a weighted sum of the value vectors can be performed to obtain the corresponding saliency mining vector.

[0137] In the fifth step, the hidden coordinates can be evaluated based on the saliency mining vector to form hidden coordinate evaluation parameters. Exemplarily, through the fully connected network layer in the interference forward processing network, the saliency mining vector can be mapped to obtain a fully connected vector with the same size as the global vector of the target person. Then, this fully connected vector can be output (e.g., through the softmax function) to obtain a probability distribution, that is, the hidden coordinate evaluation parameters. Among them, each probability value can represent the probability that the corresponding coordinate is used as a hidden coordinate.

[0138] In the sixth step, a coordinate set matching the proportion of the specified size representation can be generated based on the hidden coordinate evaluation parameters. The hidden coordinate evaluation parameters are used to reflect the likelihood of each semantic vector parameter being hidden. For example, the coordinates with the largest proportion of the specified size representation with the highest probability of being a hidden coordinate can be combined to form a coordinate set.

[0139] It should be further noted that for step S140 in the above embodiment, in the above step S140, the specific process of removing the perturbation from the target person perturbation vector is not limited and can be selected according to actual application requirements.

[0140] For example, in a specific implementation manner, in order to remove the perturbation from the target person perturbation vector according to the information learned from the training data to improve the reliability of perturbation removal, the above step S140 can further include step S141 and step S142, and the specific content is as follows.

[0141] Step S141, removing the perturbation from the target person perturbation vector through the interference backward processing network and outputting the perturbation removal data at the current processing stage.

[0142] In the embodiment of the present invention, the perturbation of the target person perturbation vector can be removed through the interference backward processing network, and the perturbation removal data at the current processing stage can be output. Among them, the perturbation applied to the global vector of the target person is realized through the interference forward processing network, as described above.

[0143] Step S142: Use the interference backward processing network to remove the perturbation from the perturbation removal data in the current processing stage, and when the current processing stage belongs to the target processing stage, output the restored vector of the target person.

[0144] In an embodiment of the present invention, after obtaining the perturbation removal data in the current processing stage, the interference backward processing network can be used to remove the perturbation from the perturbation removal data in the current processing stage (i.e., using the perturbation removal data in the current processing stage as a new perturbation vector of the target person), and when the current processing stage belongs to the target processing stage, output the restored vector of the target person. That is to say, the perturbation removal can be performed on the perturbation vector of the target person in at least two stages, so that the reliability of the removal of the perturbation information is higher, and thus a reliable restored vector of the target person can be obtained. Among them, the specific number of stages of the target processing stage is not limited and can be selected according to actual situations. For example, based on the requirement of efficiency, the smaller the number of stages, and based on the requirement of accuracy, the larger the number of stages.

[0145] It should be further noted that for the above-mentioned step S141, in the above-mentioned step S141, the specific process of removing the perturbation from the perturbation vector of the target person by the interference backward processing network is not limited and can be selected according to actual situations.

[0146] For example, in a specific implementation manner, in order to be able to fully remove the perturbation information in the perturbation vector of the target person by capturing context-related semantic information, the above-mentioned step S141 can further include step S141a, step S141b, step S141c, step S141d, and step S141e. The specific content of each step is described as follows.

[0147] Step S141a: Load the perturbation vector of the target person to be loaded into the interference backward processing network.

[0148] In an embodiment of the present invention, the perturbation vector of the target person can be loaded to be loaded into the interference backward processing network, that is, subsequent interference removal processing is performed in the interference backward processing network. The specific processing process is described as follows.

[0149] Step S141b: Calculate the first mapping parameter, the second mapping parameter, and the third mapping parameter of each semantic vector parameter.

[0150] In an embodiment of the present invention, in the interference backward processing network, the first mapping parameter, the second mapping parameter, and the third mapping parameter of each of the semantic vector parameters can be calculated. For example, the interference backward processing network includes a first mapping vector, a second mapping vector, and a third mapping vector formed by training, and the sizes of the three mapping vectors can be the same as the size of the target person perturbation vector. Thus, for each semantic vector parameter in the target person perturbation vector, the semantic vector parameter can be multiplied by the parameters at the corresponding positions in the first mapping vector, the second mapping vector, and the third mapping vector to obtain the corresponding first mapping parameter, second mapping parameter, and third mapping parameter.

[0151] Step S141c, polling multiple semantic vector parameters, and performing a dot product operation on the first mapping parameter of the currently polled semantic vector parameter and the second mapping parameters of each of the other semantic vector parameters, and outputting the significance parameter between the currently polled semantic vector parameter and each of the other semantic vector parameters.

[0152] In an embodiment of the present invention, after obtaining the first mapping parameter, the second mapping parameter, and the third mapping parameter of each semantic vector parameter, multiple semantic vector parameters can be polled, and a dot product operation is performed on the first mapping parameter of the currently polled semantic vector parameter and the second mapping parameters of each of the other semantic vector parameters, and the significance parameter between the currently polled semantic vector parameter and each of the other semantic vector parameters, that is, the result of the dot product, is output. Thus, each semantic vector parameter can be polled so that the significance parameter between each semantic vector parameter and each of the other semantic vector parameters can be obtained. For example, the target person perturbation vector can be [a1, a2, a3]. Thus, for a1, the product of the first mapping parameter corresponding to a1 and the second mapping parameter corresponding to a2 can be calculated respectively, and the product of the first mapping parameter corresponding to a1 and the second mapping parameter corresponding to a3 can be calculated; for a2, the product of the first mapping parameter corresponding to a2 and the second mapping parameter corresponding to a1 can be calculated respectively, and the product of the first mapping parameter corresponding to a2 and the second mapping parameter corresponding to a3 can be calculated; for a3, the product of the first mapping parameter corresponding to a3 and the second mapping parameter corresponding to a1 can be calculated respectively, and the product of the first mapping parameter corresponding to a3 and the second mapping parameter corresponding to a1 can be calculated. Thus, each significance parameter can be obtained.

[0153] Step S141d, poll the multiple semantic vector parameters, and use each of the salience parameters of the currently polled semantic vector parameter as a weight coefficient to fuse the third mapping parameters of each of the other semantic vector parameters, and output the salience fusion parameter of the currently polled semantic vector parameter.

[0154] In an embodiment of the present invention, after obtaining the corresponding salience parameters, the multiple semantic vector parameters can be polled, and each of the salience parameters of the currently polled semantic vector parameter is used as a weight coefficient to fuse the third mapping parameters of each of the other semantic vector parameters, and output the salience fusion parameter of the currently polled semantic vector parameter. For example, for a1, the corresponding salience fusion parameter = (the first mapping parameter corresponding to a1 * the second mapping parameter corresponding to a2) * the third mapping parameter corresponding to a2 + (the first mapping parameter corresponding to a1 * the second mapping parameter corresponding to a3) * the third mapping parameter corresponding to a2.

[0155] Step S141e, merge the salience fusion parameters of the multiple semantic vector parameters to obtain the perturbation removal data in the current processing stage after perturbation removal.

[0156] In an embodiment of the present invention, after obtaining each salience fusion parameter, the salience fusion parameters of the multiple semantic vector parameters can be merged to obtain the perturbation removal data in the current processing stage after perturbation removal, such as [the salience fusion parameter corresponding to a1, the salience fusion parameter corresponding to a2, the salience fusion parameter corresponding to a3].

[0157] For the above step S141b, in a specific implementation manner, it may include the following content:

[0158] In the first step, the first transformation parameter, the second transformation parameter, and the third transformation parameter of each of the semantic vector parameters can be determined from the interference backward processing network. For example, based on the first mapping vector, the second mapping vector, and the third mapping vector formed by training included in the interference backward processing network, the first transformation parameter, the second transformation parameter, and the third transformation parameter can be determined respectively, that is, the parameters at the positions corresponding to the semantic vector parameters in the first mapping vector, the second mapping vector, and the third mapping vector are respectively used as the first transformation parameter, the second transformation parameter, and the third transformation parameter;

[0159] Second, for each of the semantic vector parameters, perform a multiplication operation on the semantic vector parameter and the first transformation parameter to output the first mapping parameter of the semantic vector parameter, perform a multiplication operation on the semantic vector parameter and the second transformation parameter to output the second mapping parameter of the semantic vector parameter, and perform a multiplication operation on the semantic vector parameter and the third transformation parameter to output the third mapping parameter of the semantic vector parameter.

[0160] Combined with the content of step S140 above, it should also be noted that for step S130 above, in order to ensure the effective implementation of step S130 above, the specified size can also be determined first. That is to say, before the step of hiding the semantic vector parameters of the specified size in the target person's global vector so that perturbations are applied to the target person's global vector to form a target person perturbation vector, the power engineering safety monitoring method based on deep learning further includes:

[0161] First, based on the target processing stage, a target parameter can be determined, where the target parameter is a randomly generated parameter less than the target processing stage. For example, when the target processing stage is 10, the target parameter can be 2, 3, 4, etc.;

[0162] Second, based on the ratio between the target parameter and the target processing stage, the specified size can be obtained, where the specified size is used to reflect the proportion of the number of semantic vector parameters hidden in the target person's global vector, such as 20%, 30%, 40%, etc.

[0163] It should be noted that, assuming the target processing stage is 10, the maximum determined target parameter is 9, that is, the maximum value of the specified size is 90%. Assuming the target processing stage is 20, the maximum determined target parameter is 19, then the maximum value of the specified size is 95%. Obviously, 95% is greater than 90%. That is to say, as the target processing stage increases, the determined specified size can also increase with a certain probability. That is, with a certain probability, more perturbation removal stages can be used to remove more perturbation information to improve reliability.

[0164] It should also be noted that for the above steps S110 - S140, to ensure the effective implementation of steps S110 - S140, it can be achieved through a corresponding neural network model. This neural network model can be a trained neural network model to learn the semantic mapping relationship in the corresponding training data. That is, before the steps of determining the target personnel trajectory data and target personnel monitoring images in the target power engineering area, and mining the target personnel trajectory data to form a target personnel trajectory vector, and mining the target personnel monitoring images to form a target personnel image vector, the power engineering safety monitoring method based on deep learning can further include:

[0165] First, the training personnel trajectory data and corresponding training personnel monitoring images can be determined, and the training personnel trajectory data can be mined to form a training personnel trajectory vector, and the training personnel monitoring images can be mined to form a training personnel image vector. The specific processing process of this step can refer to the relevant explanation of step S110 in the previous text;

[0166] Second, after aggregating the training personnel image vector into the training personnel trajectory vector, a perturbation can be applied using a perturbation forward processing network to form a training personnel perturbation vector, and the training personnel perturbation vector can be used to remove the perturbation using a perturbation backward processing network to form a training personnel restoration vector. The specific processing process of this step can refer to the relevant explanations of steps S120 - S140 in the previous text;

[0167] Third, the training safety monitoring result can be determined based on the training personnel restoration vector (which can refer to the relevant explanation of step S150 in the previous text), and based on the error between the training safety monitoring result and the corresponding safety label data (such as the cross - entropy error between the corresponding probability distributions), the perturbation forward processing network and the perturbation backward processing network can be trained to form a trained perturbation forward processing network and a trained perturbation backward processing network. For example, the network parameters of the perturbation forward processing network and the perturbation backward processing network can be updated and adjusted along the direction of reducing this error to make this error converge, such as being less than a preset value.

[0168] Among them, the calculation formula of the cross - entropy error is:

[0169] Cross - Entropy Loss = −(y * log(p)+(1−y) * log(q));

[0170] Among them, y is the probability corresponding to the security label data, such as 0 or 1, and p, q are the probabilities corresponding to the training security monitoring results (such as 0.8, 0.2). Thus, Cross-Entropy Loss = −(1*log(0.8)+(1 - 1)*log(0.2)) = −log(0.8) ≈ 0.2231.

[0171] In summary, for the power engineering security monitoring method and system based on deep learning provided by the present invention, first, the target personnel trajectory data is mined to form a target personnel trajectory vector, and the target personnel monitoring image is mined to form a target personnel image vector; second, the target personnel image vector is aggregated into the target personnel trajectory vector to form a target personnel global vector; then, the semantic vector parameters of a specified size in the target personnel global vector are hidden, so that perturbations are applied to the target personnel global vector to form a target personnel perturbation vector; further, the perturbations of the target personnel perturbation vector are removed to form a target personnel restoration vector; finally, the target security monitoring result is determined based on the target personnel restoration vector. Based on the above content, on the one hand, by mining the trajectory data and the monitoring image, potential semantic information can be mined, and compared with traditional data analysis techniques, potential behavioral semantic features can be captured. Therefore, the accuracy of judgment can be improved to a certain extent. In addition, after aggregating to form the target personnel global vector, through further perturbation application and removal, the fusion of potential semantic information in the two dimensions of trajectory and image can be achieved in this process (the semantics of trajectory information is simple but continuous, and the semantics of image information is rich but local. Therefore, through fusion, mutual reinforcement can be achieved, that is, both the richness and continuity of semantics are taken into account), so as to obtain a target personnel restoration vector with relatively higher representation accuracy, and the reliability of the determined target security monitoring result can be further improved, which can improve the problem of relatively low reliability of power engineering security monitoring existing in the prior art.

[0172] In several embodiments provided by the embodiments of the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0173] In addition, in each embodiment of the present invention, the various functional modules can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0174] If the above functions are implemented in the form of software function modules and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, etc., which can store program codes. It should be noted that in this article, the term "including", "comprising", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article, or device. Without further limitations, the element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device including the said element.

[0175] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A safety monitoring method for power engineering based on deep learning, characterized in that, Including: Determine the target personnel trajectory data and target personnel monitoring images of the target power engineering area, and mine the target personnel trajectory data to form a target personnel trajectory vector, and mine the target personnel monitoring images to form a target personnel image vector. Among them, the embedding process of the word embedding model is included in the mining process of the target personnel trajectory data, and the convolution process of the convolutional network is included in the mining process of the target personnel monitoring images; Aggregate the target personnel image vector into the target personnel trajectory vector to form a target personnel global vector. Among them, the target personnel global vector includes multiple semantic vector parameters, and the target personnel global vector is used to represent the global semantic information in the image dimension and the trajectory dimension; Hide the semantic vector parameters of the specified size in the target personnel global vector, so that perturbations are applied to the target personnel global vector to form a target personnel perturbation vector. Among them, the specified size is used to reflect the number of the hidden semantic vector parameters, and the hidden semantic vector parameters are determined after evaluating the possibility of being hidden for each semantic vector parameter in the target personnel global vector; Remove the perturbations from the target personnel perturbation vector to form a target personnel restoration vector. Among them, the perturbation removal is achieved by using the context-related semantic information in the target personnel perturbation vector, so that the context-related semantic information is interactively fused during the perturbation removal process; Determine the target security monitoring result based on the target personnel restoration vector. Among them, the process of determining the target security monitoring result includes a fully connected process, which is used to map the target personnel restoration vector to the probability of whether there is an abnormal construction situation. The target security monitoring result is used to reflect whether there is an abnormal construction situation in the target power engineering area.

2. The method for power engineering safety monitoring based on deep learning according to claim 1, characterized in that, The step of aggregating the target personnel image vector into the target personnel trajectory vector to form a target personnel global vector includes: Perform at least one segmentation on the target personnel trajectory vector to form at least one segmentation position; Perform at least one segmentation on the target personnel image vector to form at least two local personnel image vectors, or use the target personnel image vector as the local personnel image vector. Among them, the number of the formed local personnel image vectors is equal to the number of the segmentation positions; Configure image semantic identifiers at both the head and tail ends of each of the local personnel image vectors to form new local personnel image vectors; Splice a corresponding new local personnel image vector at each of the segmentation positions to form a target personnel global vector.

3. The method for power engineering safety monitoring based on deep learning according to claim 1, wherein The step of hiding the semantic vector parameters of the specified size in the target personnel global vector, so that perturbations are applied to the target personnel global vector to form a target personnel perturbation vector includes: Determine the hidden coordinates of the global vector of the target person through the interference forward processing network, and output a coordinate set that matches the quantity ratio represented by the specified size, where the interference forward processing network is jointly trained with the interference backward processing network, and the interference backward processing network is used to remove the perturbation of the target person perturbation vector; Hide the semantic vector parameters corresponding to the coordinate set in the global vector of the target person, so that a perturbation is applied to the global vector of the target person to form a target person perturbation vector, where the hidden semantic vector parameters are replaced with random values or specific values, and the specific values include 0.

4. The method for power engineering safety monitoring based on deep learning according to claim 3, characterized in that The step of determining the hidden coordinates of the global vector of the target person through the interference forward processing network and outputting a coordinate set that matches the quantity ratio represented by the specified size includes: Load the global vector of the target person to be loaded into the interference forward processing network; Determine the coordinate distribution parameters corresponding to the global vector of the target person, where the coordinate distribution parameters are used to reflect the coordinates of multiple semantic vector parameters in the global vector of the target person; Cascade the global vector of the target person and the coordinate distribution parameters to form a cascaded vector; Perform saliency mining on the cascaded vector to form a saliency mining vector. The interference forward processing network includes a query matrix, a key matrix, and a value matrix. The process of saliency mining includes: multiplying the query matrix, the key matrix, and the value matrix with the cascaded vector respectively to obtain corresponding query vectors, key vectors, and value vectors, calculating the dot product between the query vector and the transposed result of the key vector, and based on this dot product, performing weighted summation on the value vectors to obtain the corresponding saliency mining vector; Evaluate the hidden coordinates based on the saliency mining vector to form a hidden coordinate evaluation parameter. The evaluation process includes: mapping the saliency mining vector through the fully connected network layer in the interference forward processing network to obtain a fully connected vector with the same size as the global vector of the target person, and outputting through the softmax function to obtain a probability distribution as the hidden coordinate evaluation parameter; Generate a coordinate set that matches the quantity ratio represented by the specified size based on the hidden coordinate evaluation parameter, where the hidden coordinate evaluation parameter is used to reflect the likelihood of each semantic vector parameter being hidden.

5. The method for power engineering safety monitoring based on deep learning according to claim 1, wherein The step of removing the perturbation of the target person perturbation vector to form a target person restoration vector includes: Remove the perturbation of the target person perturbation vector through the interference backward processing network and output the perturbation removal data of the current processing stage, where the perturbation applied to the global vector of the target person is achieved through the interference forward processing network; Remove the perturbation of the perturbation removal data of the current processing stage through the interference backward processing network, and when the current processing stage belongs to the target processing stage, output the target person restoration vector.

6. The method for power engineering safety monitoring based on deep learning according to claim 5, characterized in that The step of removing the perturbation of the target person perturbation vector through the interference backward processing network and outputting the perturbation removal data of the current processing stage includes: Loading the target person perturbation vector into the interference backward processing network; Calculating the first mapping parameter, the second mapping parameter, and the third mapping parameter of each semantic vector parameter; Polling multiple semantic vector parameters, and performing a dot product operation on the first mapping parameter of the currently polled semantic vector parameter and the second mapping parameters of each other semantic vector parameter, and outputting the significance parameter between the currently polled semantic vector parameter and each other semantic vector parameter; Polling multiple semantic vector parameters, and using the significance parameters of the currently polled semantic vector parameter as weight coefficients to fuse the third mapping parameters of each other semantic vector parameter, and outputting the significance fusion parameter of the currently polled semantic vector parameter, where the fusion process refers to using the significance parameter as a weight parameter to perform a weighted sum calculation on the third mapping parameters of each semantic vector parameter; Combining the significance fusion parameters of multiple semantic vector parameters to obtain the perturbation removal data of the current processing stage formed after perturbation removal.

7. The method for power engineering safety monitoring based on deep learning according to claim 6, wherein, The step of calculating the first mapping parameter, the second mapping parameter, and the third mapping parameter of each semantic vector parameter includes: Determining the first transformation parameter, the second transformation parameter, and the third transformation parameter of each semantic vector parameter from the interference backward processing network; For each semantic vector parameter, multiplying the semantic vector parameter by the first transformation parameter to output the first mapping parameter of the semantic vector parameter, multiplying the semantic vector parameter by the second transformation parameter to output the second mapping parameter of the semantic vector parameter, and multiplying the semantic vector parameter by the third transformation parameter to output the third mapping parameter of the semantic vector parameter.

8. The method for power engineering safety monitoring based on deep learning according to claim 5, characterized in that, Before the step of hiding the semantic vector parameters of a specified size in the target person global vector to apply a perturbation to the target person global vector to form a target person perturbation vector, the deep learning-based power engineering safety monitoring method further includes: Determining a target parameter based on the target processing stage, where the target parameter is a randomly generated parameter smaller than the target processing stage; Obtaining the specified size based on the ratio between the target parameter and the target processing stage, where the specified size is used to reflect the proportion of the number of semantic vector parameters hidden in the target person global vector.

9. The method for power engineering safety monitoring based on deep learning according to any one of claims 1-8, characterized in that, Before the steps of determining the target person trajectory data and the target person monitoring image of the target power engineering area, mining the target person trajectory data to form a target person trajectory vector, and mining the target person monitoring image to form a target person image vector, the deep learning-based power engineering safety monitoring method further includes: Determine the training personnel trajectory data and the corresponding training personnel monitoring images, and mine the training personnel trajectory data to form a training personnel trajectory vector, and mine the training personnel monitoring images to form a training personnel image vector; After aggregating the training personnel image vector into the training personnel trajectory vector, use the interference forward processing network to apply perturbations to form a training personnel perturbation vector, and use the interference backward processing network to remove the perturbations from the training personnel perturbation vector to form a training personnel restoration vector; Determine the training safety monitoring result based on the training personnel restoration vector, and train the interference forward processing network and the interference backward processing network based on the error between the training safety monitoring result and the corresponding safety label data to form a trained interference forward processing network and a trained interference backward processing network.

10. A power engineering safety monitoring system based on deep learning, characterized in that, Comprising: A memory for storing computer programs; A processor connected to the memory for executing the computer programs stored in the memory to implement the deep learning-based power engineering safety monitoring method according to any one of claims 1-9.

Images