Electric power engineering safety monitoring method and system based on deep learning

Abstract

The invention provides an electric power engineering safety monitoring method and system based on deep learning, and relates to the technical field of deep learning. According to the method, firstly, target person trajectory data are mined to form a target person trajectory vector, and a target person monitoring image is mined to form a target person image vector; secondly, target person image vectors are aggregated into target person trajectory vectors, and target person global vectors are formed; then, semantic vector parameters of a specified size in the global vector of the target personnel are hidden, so that disturbance is applied to the global vector of the target personnel, and a disturbance vector of the target personnel is formed; further, disturbance removal is carried out on the target person disturbance vector, and a target person reduction vector is formed; and finally, determining a target safety monitoring result according to the target person restoration vector. Based on the above content, the problem that the reliability of electric power engineering safety monitoring is relatively low in the prior art can be improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a method and system for power engineering safety monitoring based on deep learning. Background Art

[0002] With the rapid development of power engineering, the safety management issues of power construction have gradually become more complex and urgent. The field of power engineering often involves complex equipment and high-voltage power systems, which pose a great threat to the safety of personnel during operation. Therefore, the safety monitoring of power engineering has become an important part of ensuring the safety of the project. Among them, traditional security monitoring systems mostly rely on fixed cameras and manual intervention, which makes it difficult to achieve accurate safety warnings, easily miss some abnormal behaviors or potential safety hazards, and cannot fully understand the behavior patterns of the target. For example, in high-voltage power areas, the behavior of staff may be somewhat ambiguous, and these behaviors are difficult to accurately identify and judge through traditional methods, resulting in relatively low reliability of safety monitoring. Summary of the invention

[0003] In view of this, an object of the present invention is to provide a method and system for power engineering safety monitoring based on deep learning, so as to improve the problem of relatively low reliability of power engineering safety monitoring in the prior art.

[0004] To achieve the above object, the present invention adopts the following technical solution: A power engineering safety monitoring method based on deep learning, comprising: Determine the target personnel trajectory data and the target personnel monitoring image of the target power engineering area, and mine the target personnel trajectory data to form a target personnel trajectory vector, and mine the target personnel monitoring image to form a target personnel image vector, wherein the mining process of the target personnel trajectory data at least includes embedding processing of a word embedding model, and the mining process of the target personnel monitoring image at least includes convolution processing of a convolutional network; Aggregating the target person image vector into the target person trajectory vector to form a target person global vector, wherein the target person global vector includes a plurality of semantic vector parameters, and the target person global vector is used to represent global semantic information in an image dimension and a trajectory dimension; Hiding the semantic vector parameters of a specified size in the global vector of the target person, so that a disturbance is applied to the global vector of the target person to form a target person disturbance vector, wherein the specified size is used to reflect the number of the hidden semantic vector parameters, and the hidden semantic vector parameters are determined by evaluating the probability of each semantic vector parameter in the global vector of the target person being hidden; Performing disturbance removal on the target person disturbance vector to form a target person restoration vector, wherein the disturbance removal is achieved by utilizing context-related semantic information in the target person disturbance vector, so that interactive fusion of context-related semantic information is achieved in the process of disturbance removal; A target safety monitoring result is determined based on the target personnel restoration vector, wherein the process of determining the target safety monitoring result includes at least full connection processing, which is used to map the target personnel restoration vector to the probability of whether there is an abnormal construction situation, and the target safety monitoring result is used to reflect whether there is an abnormal construction situation in the target power engineering area.

[0005] In a preferred embodiment of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, the step of aggregating the target person image vector into the target person trajectory vector to form a target person global vector includes: Segmenting the target person's trajectory vector at least once to form at least one segmentation position; Perform at least one segmentation on the target person image vector to form at least two partial person image vectors, or use the target person image vector as a partial person image vector, wherein the number of the partial person image vectors formed is equal to the number of the segmentation positions; respectively configuring image semantic identifiers at the beginning and the end of each of the local person image vectors to form a new local person image vector; A new local person image vector corresponding to each segmentation position is spliced ​​to form a global vector of the target person.

[0006] In a preferred embodiment of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, the step of hiding the semantic vector parameters of the specified size in the global vector of the target person so as to apply disturbance to the global vector of the target person to form a disturbance vector of the target person includes: Determine the hidden coordinates of the global vector of the target person through an interference forward processing network, and output a coordinate set that matches the quantity ratio of the specified size representation, wherein the interference forward processing network is jointly trained with the interference backward processing network, and the interference backward processing network is used to remove the disturbance of the target person disturbance vector; The semantic vector parameters corresponding to the coordinate set in the global vector of the target person are hidden, so that disturbance is applied to the global vector of the target person to form a target person disturbance vector, wherein the hidden semantic vector parameters are replaced with random values ​​or specific values, and the specific value includes 0.

[0007] In a preferred embodiment of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, the step of determining the hidden coordinates of the target personnel global vector by interfering with the forward processing network and outputting a coordinate set matching the quantity ratio of the specified size representation includes: Loading the target person global vector to load into the interference forward processing network; Determine a coordinate distribution parameter corresponding to the global vector of the target person, wherein the coordinate distribution parameter is used to reflect the coordinates of the plurality of semantic vector parameters in the global vector of the target person; Cascading the target person's global vector and the coordinate distribution parameter to form a cascade vector; The cascade vector is saliency mined to form a saliency mining vector, wherein the interference forward processing network includes a query matrix, a key matrix and a value matrix, and the saliency mining process includes: multiplying the query matrix, the key matrix and the value matrix with the cascade vector respectively to obtain a corresponding query vector, a key vector and a value vector, calculating a dot product between the query vector and the transposed result of the key vector, and based on the dot product, performing a weighted summation on the value vector to obtain a corresponding saliency mining vector; Evaluate the hidden coordinates according to the saliency mining vector to form hidden coordinate evaluation parameters, wherein the evaluation process includes: mapping the saliency mining vector through the fully connected network layer in the interference forward processing network to obtain a fully connected vector with the same size as the global vector of the target person, and outputting the fully connected vector through a softmax function to obtain a probability distribution as the hidden coordinate evaluation parameter; A coordinate set matching the quantity ratio of the specified size representation is generated according to the hidden coordinate evaluation parameter, wherein the hidden coordinate evaluation parameter is used to reflect the possibility of each of the semantic vector parameters being hidden.

[0008] In a preferred embodiment of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, the step of removing disturbance from the target personnel disturbance vector to form a target personnel restoration vector includes: De-disturb the target person's disturbance vector through an interference backward processing network, and output disturbance removal data of the current processing stage, wherein the perturbation applied to the target person's global vector is achieved through an interference forward processing network; The disturbance removal data of the current processing stage is disturbed by the interference backward processing network, and when the current processing stage belongs to the target processing stage, the target person restoration vector is output.

[0009] In a preferred embodiment of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, the step of removing disturbance from the target personnel disturbance vector through the interference backward processing network and outputting the disturbance removal data of the current processing stage includes: Loading the target person disturbance vector to load into the interference backward processing network; Calculating a first mapping parameter, a second mapping parameter and a third mapping parameter of each of the semantic vector parameters; Polling a plurality of the semantic vector parameters, and performing a dot multiplication operation based on the first mapping parameter of the currently polled semantic vector parameter and the second mapping parameter of each of the other semantic vector parameters, and outputting a significance parameter between the currently polled semantic vector parameter and each of the other semantic vector parameters; Polling a plurality of the semantic vector parameters, and using each of the significance parameters of the currently polled semantic vector parameters as a weight coefficient to fuse the third mapping parameters of the other semantic vector parameters, and outputting the significance fusion parameter of the currently polled semantic vector parameter, wherein the fusion process refers to using the significance parameter as a weight parameter to perform a weighted sum calculation on the third mapping parameters of the various semantic vector parameters; The saliency fusion parameters of the plurality of semantic vector parameters are combined to obtain disturbance removal data of the current processing stage formed after disturbance removal.

[0010] In a preferred embodiment of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, the step of calculating the first mapping parameter, the second mapping parameter and the third mapping parameter of each of the semantic vector parameters includes: Determining a first transformation parameter, a second transformation parameter and a third transformation parameter of each of the semantic vector parameters from the interference backward processing network; For each of the semantic vector parameters, the semantic vector parameter is multiplied by the first transformation parameter to output the first mapping parameter of the semantic vector parameter, and the semantic vector parameter is multiplied by the second transformation parameter to output the second mapping parameter of the semantic vector parameter, and the semantic vector parameter is multiplied by the third transformation parameter to output the third mapping parameter of the semantic vector parameter.

[0011] In a preferred embodiment of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, before the step of hiding the semantic vector parameters of the specified size in the target personnel global vector so as to apply disturbance to the target personnel global vector to form the target personnel disturbance vector, the power engineering safety monitoring method based on deep learning also includes: Based on the target processing stage, determining a target parameter, wherein the target parameter is a randomly generated parameter that is less than the target processing stage; The specified size is obtained based on the ratio between the target parameter and the target processing stage, wherein the specified size is used to reflect the proportion of the number of semantic vector parameters to be hidden in the global vector of the target person.

[0012] In a preferred embodiment of the present invention, in the above-mentioned power engineering safety monitoring method based on deep learning, before the steps of determining the target personnel trajectory data and the target personnel monitoring image of the target power engineering area, and mining the target personnel trajectory data to form a target personnel trajectory vector, and mining the target personnel monitoring image to form a target personnel image vector, the power engineering safety monitoring method based on deep learning also includes: Determine trainee trajectory data and corresponding trainee monitoring images, and mine the trainee trajectory data to form a trainee trajectory vector, and mine the trainee monitoring images to form a trainee image vector; After the training person image vector is aggregated into the training person trajectory vector, disturbance is applied using a disturbance forward processing network to form a training person disturbance vector, and disturbance is removed from the training person disturbance vector using a disturbance backward processing network to form a training person restoration vector; Based on the training personnel restoration vector, a training safety monitoring result is determined, and based on the error between the training safety monitoring result and the corresponding safety label data, the interference forward processing network and the interference backward processing network are trained to form a trained interference forward processing network and a trained interference backward processing network.

[0013] On the basis of the above, the present invention also provides a power engineering safety monitoring system based on deep learning, comprising: Memory for storing computer programs; A processor connected to the memory is used to execute the computer program stored in the memory to implement the above-mentioned power engineering safety monitoring method based on deep learning.

[0014] The power engineering safety monitoring method and system based on deep learning provided by the present invention first mine the target personnel trajectory data to form a target personnel trajectory vector, and mine the target personnel monitoring image to form a target personnel image vector; secondly, the target personnel image vector is aggregated into the target personnel trajectory vector to form a target personnel global vector; then, the semantic vector parameters of the specified size in the target personnel global vector are hidden, so that disturbance is applied to the target personnel global vector to form a target personnel disturbance vector; further, the target personnel disturbance vector is disturbed and removed to form a target personnel restoration vector; finally, the target safety monitoring result is determined according to the target personnel restoration vector. Based on the above content, on the one hand, by mining the trajectory data and monitoring images, the potential semantic information therein can be mined. Compared with traditional data analysis technology, the potential behavioral semantic features can be captured, and thus, the accuracy of judgment can be improved to a certain extent. In addition, after aggregating to form a global vector of the target person, through further disturbance application and removal, the fusion of the potential semantic information of the two dimensions of trajectory and image can be achieved in this process (the semantics of trajectory information is simple but continuous, and the semantics of image information is rich but local. Therefore, mutual reinforcement can be achieved through fusion, that is, both semantic richness and continuity are taken into account), so as to obtain a target person restoration vector with relatively higher representation accuracy, so that the reliability of the determined target safety monitoring results can be further improved, which can improve the relatively low reliability of power engineering safety monitoring in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings.

[0016] Figure 1 A structural block diagram of a deep learning-based power engineering safety monitoring system provided in an embodiment of the present invention.

[0017] Figure 2 A block diagram of a deep learning-based power engineering safety monitoring device provided in an embodiment of the present invention.

[0018] Figure 3 A schematic flow chart of a method for power engineering safety monitoring based on deep learning provided in an embodiment of the present invention.

[0019] Figure 4 A schematic diagram of a convolution process based on edge filling provided by an embodiment of the present invention.

[0020] Figure 5 A schematic diagram of the target person image vector mining process provided by an embodiment of the present invention.

[0021] Figure 6 A schematic diagram of vector segmentation and extraction provided by an embodiment of the present invention.

[0022] Figure 7 A schematic diagram of vector aggregation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0024] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] like Figure 1 As shown, an embodiment of the present invention provides a power engineering safety monitoring system based on deep learning. The power engineering safety monitoring system based on deep learning may include a memory, a processor, and a power engineering safety monitoring device based on deep learning.

[0026] In detail, the memory and the processor are electrically connected directly or indirectly to achieve data transmission or interaction. For example, the memory and the processor can be electrically connected through one or more communication buses or signal lines. The deep learning-based power engineering safety monitoring device includes at least one software function module stored in the memory in the form of software or firmware. The processor is used to execute an executable computer program stored in the memory, for example, the software function modules and computer programs included in the deep learning-based power engineering safety monitoring device, so as to implement the deep learning-based power engineering safety monitoring method provided in an embodiment of the present invention.

[0027] Optionally, the memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0028] Optionally, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0029] Optionally, combined Figure 2 In a specific implementation, the power engineering safety monitoring device based on deep learning may include: A semantic mining module, for determining target personnel trajectory data and target personnel monitoring images in a target power engineering area, and mining the target personnel trajectory data to form a target personnel trajectory vector, and mining the target personnel monitoring images to form a target personnel image vector; A vector aggregation module, used for aggregating the target person image vector into the target person trajectory vector to form a target person global vector, wherein the target person global vector includes a plurality of semantic vector parameters; A disturbance applying module, used for hiding the semantic vector parameter of a specified size in the global vector of the target person, so that a disturbance is applied to the global vector of the target person to form a disturbance vector of the target person; A disturbance removal module, used to remove disturbance from the target person disturbance vector to form a target person restoration vector; The abnormality analysis module is used to determine the target safety monitoring result based on the target personnel restoration vector, wherein the target safety monitoring result is used to reflect whether there is any abnormal construction situation in the target power engineering area.

[0030] Understandably, Figure 1The structure shown is for illustration only. The power engineering safety monitoring system based on deep learning may also include Figure 1 More or fewer components as shown, or with Figure 1 The different configurations shown, for example, may also include a communication unit for information exchange with other devices. In addition, the power engineering safety monitoring system based on deep learning may be an electronic device, such as a server or a cluster of multiple servers.

[0031] Combination Figure 3 The embodiment of the present invention also provides a deep learning-based power engineering safety monitoring method that can be applied to the deep learning-based power engineering safety monitoring system. The method steps defined in the process related to the deep learning-based power engineering safety monitoring method can be implemented by the deep learning-based power engineering safety monitoring system (hereinafter referred to as the monitoring system). Figure 3 The specific process shown is explained in detail.

[0032] Step S110, determining the target personnel trajectory data and the target personnel monitoring image in the target power engineering area, and mining the target personnel trajectory data to form a target personnel trajectory vector, and mining the target personnel monitoring image to form a target personnel image vector.

[0033] In an embodiment of the present invention, the monitoring system can determine the target personnel trajectory data of the target power engineering area (exemplarily, the trajectory data of each staff member can be collected by setting a positioning device, such as an RFID tag, on the work clothes of each staff member working in the target power engineering area) and the target personnel monitoring image (exemplarily, the image can be collected by a device such as a camera set in the target power engineering area), and the target personnel trajectory data is mined to form a target personnel trajectory vector, and the target personnel monitoring image is mined to form a target personnel image vector. Among them, the mining process of the target personnel trajectory data at least includes the embedding processing of a word embedding model, and the mining process of the target personnel monitoring image at least includes the convolution processing of a convolutional network. Exemplarily, the target personnel trajectory data can be embedded by a word embedding model to form a corresponding target personnel trajectory vector. For example, the word embedding vectors of each staff member can be spliced ​​to form a target personnel trajectory vector. In addition, the target personnel monitoring image can be convolved by a convolutional network to form a corresponding target personnel image vector (such as Figure 4As shown, the size of the target person image vector formed by convolution can be the same as the size of the target person monitoring image. That is to say, for each pixel value in the target person monitoring image, a region can be determined according to the size of the convolution kernel of the convolution network (for example, the size can be 3*3). Then, the pixel values in this region are weighted and summed with the parameters in the convolution kernel. In this way, a convolution parameter corresponding to the center point can be obtained. Thus, for each pixel value in the target person monitoring image, a corresponding convolution parameter can be obtained. Then, the pixel value can be replaced with the corresponding convolution parameter to form the target person image vector. It should be noted that for the pixel points at the edges, edge padding can be performed (for example, all filled with 0). For example, when there are multiple target person monitoring images (formed by multiple cameras monitoring multiple local areas, or one camera monitoring one local area for multiple time periods, and the specific selection can be made according to actual needs), the convolution vectors (matrices) corresponding to each target person monitoring image can be directly expanded or mapped to row vectors or column vectors through a fully connected network. Then, the row vectors or column vectors corresponding to each target person monitoring image are concatenated to form the corresponding target person image vector. In addition, for the embedding processing of the target person trajectory data, assuming that part of the content in the target person trajectory data is "the trajectory coordinates of the staff are in sequence", then, the "the trajectory coordinates of the staff are in sequence" can be tokenized first to obtain the corresponding words "work", "person", "of", "trajectory", "coordinate", "in sequence", "are", and then, word embedding processing is performed on each word to obtain word vectors, such as: Word vector of "work": [0.3, 0.3, -0.1, 0.2, -0.4, 0.7, -0.8, 0.2,..., 0.9]; Word vector of "person": [0.1, 0.3, -0.2, 0.5, -0.4, 0.6, -0.1, 0.2,..., 0.3]; Word vector of "of": [0.5, -0.1, 0.2, -0.3, 0.4, 0.1, 0.3, -0.2,..., 0.1]; Word vector of "trajectory": [0.4, -0.2, 0.1, 0.3, -0.5, 0.2, -0.4, 0.6,..., 0.5]; Word vector of "coordinate": [-0.3, 0.1, 0.4, 0.2, 0.3, -0.1, 0.4, 0.5,..., -0.2]; Word vector of "in sequence": [0.2, -0.3, 0.4, -0.2, 0.1, 0.3, -0.1, 0.4,..., 0.6]; Word vector of "is": [0.1, 0.2, -0.3, 0.4, -0.1, 0.5, -0.3, 0.2,..., 0.3].

[0034] Then, the word vectors of the above-mentioned various words and the word vectors of other words can be concatenated to form the word embedding vector of the corresponding staff member, as follows: .

[0035] Step S120, aggregate the target person image vector into the target person trajectory vector to form a target person global vector.

[0036] In the embodiment of the present invention, after forming the target person image vector and the target person trajectory vector, the monitoring system can aggregate the target person image vector into the target person trajectory vector to form a target person global vector. In this way, the target person global vector can represent the global semantic information in the image dimension and the trajectory dimension. Among them, the target person global vector includes multiple semantic vector parameters, such as M*N.

[0037] Step S130, hide the semantic vector parameters of the specified size in the target person global vector, so that perturbations are imposed on the target person global vector to form a target person perturbation vector.

[0038] In an embodiment of the present invention, after forming the global vector of the target person, the monitoring system may hide the semantic vector parameters of the specified size in the global vector of the target person, so that a disturbance is applied to the global vector of the target person to form a disturbance vector of the target person, wherein the specified size is used to reflect the number of the hidden semantic vector parameters, and the hidden semantic vector parameters are determined by evaluating the possibility of hiding each semantic vector parameter in the global vector of the target person, so as to improve the reliability of the disturbance application. It should be noted that both the trajectory data and the monitoring image have certain distortions in the representation of the real situation, that is, there are disturbances or noises. Therefore, by applying disturbances, the robustness and effective representation of the real situation can be increased. Exemplarily, the specified size can be a certain ratio (such as 35%, 40%, etc.), or an arbitrary ratio, or a ratio configured according to actual needs, for example, based on the number of semantic vector parameters in the global vector of the target person, this ratio determines how many semantic vector parameters will be hidden. According to the specified size, a hidden matrix with the same size as the global vector of the target person is randomly generated. The parameters in this hidden matrix can be 0 or 1, where 1 means that the semantic vector parameters at the corresponding position will be hidden (i.e., perturbation is applied), 0 means that it remains unchanged (or vice versa), and the proportion of parameters equal to 1 in this hidden matrix is ​​equal to the specified size. Then, the generated hidden matrix is ​​applied to the global vector of the target person. For the position with a value of 1 in the hidden matrix, the corresponding semantic vector parameters are replaced with random values ​​or specific values ​​(such as 0, etc.), or perturbation marks can be added to these corresponding semantic vector parameters. At this time, these semantic vector parameters with perturbation marks do not contain any valid semantic information, and the vector after the hidden processing is the perturbation vector of the target person. For example, if the specified size is 20%, 20% of the semantic vector parameters in the global vector of the target person will be hidden, that is, 20% of the elements in the corresponding hidden matrix are 1. Assume that the global vector of the target person contains 10 semantic vector parameters [X0, X1, X2, X3, X4, X5, X6, X7, X8, X9]. Based on 20% of the specified size, a hidden matrix with the same size as the target person’s global vector can be randomly generated, such as [0, 1, 0, 0, 0, 0, 1, 0, 0, 0], and then, for the position with a value of 1 in the hidden matrix, the corresponding semantic vector parameter in the target person’s global vector is replaced with 0, that is, [X0, 0, X2, X3, X4, X5, 0, X7, X8, X9] is obtained.

[0039] Step S140, removing disturbance from the target person disturbance vector to form a target person restoration vector.

[0040] In an embodiment of the present invention, after obtaining the target person disturbance vector, the monitoring system can remove the disturbance of the target person disturbance vector to form a target person restoration vector. Since the applied disturbance information does not have an effective semantic representation function, it is still necessary to remove it. In this way, the disturbance can be removed by using the context-related semantic information in the target person disturbance vector, so that the interactive fusion of context-related semantic information can be achieved in the process of removing the disturbance, thereby obtaining a target person restoration vector with better representation ability. In this way, interactive fusion is performed in the process of disturbance application and removal. When the interactive fusion of context-related semantic information is achieved, the overfitting problem caused by the interactive fusion of context-related semantic information directly on the target person global vector can also be avoided. In addition, since the disturbance information is removed by mining the context-related semantic information, the irrelevant disturbance information can be effectively removed, and other useless information that is originally irrelevant in the target person global vector can also be removed, which can further improve the semantic representation accuracy of the target person restoration vector.

[0041] Step S150, determining a target safety monitoring result based on the target person restoration vector.

[0042] In an embodiment of the present invention, after forming the target personnel restoration vector, the monitoring system can determine the target safety monitoring result based on the target personnel restoration vector. Among them, the target safety monitoring result is used to reflect whether there is an abnormal construction situation in the target power engineering area. Exemplarily, the target personnel restoration vector can be fully connected so that the target personnel restoration vector can be mapped to a vector of the target size, such as 1*2, that is, including two vector parameters. Then, this vector can be mapped through classification functions such as softmax to obtain a probability distribution of size 1*2, that is, the probability of the existence of abnormal construction and the probability of the absence of abnormal construction. The larger value is taken as the target safety monitoring result. For example, there are a large number of staff trajectories in some dangerous areas, indicating that there may be abnormalities. Then, if the content in the monitoring image shows that the work content includes wearing safety equipment and construction tools, it means that normal construction is in progress, so it does not belong to abnormal conditions.

[0043] Exemplarily, the target person restoration vector may be a vector of size 1*n, such as X=(x1, x2, ..., x n), during the full connection processing, the target person restoration vector can be matrix multiplied with a weight parameter distribution of size n*2 to obtain a vector of size 1*2 (such as y in the following matrix multiplication formula), and then the vector can be added to a bias vector of size 1*2 to complete the full connection processing, and then the added vector is mapped through classification functions such as softmax to obtain a probability distribution of size 1*2.

[0044] Among them, the weight parameter distribution of n*2 can be expressed as: W= .

[0045] Among them, each parameter in W can be equal to 0 in the initial stage, and then, during the training process, it can be updated accordingly to obtain the final specific value.

[0046] The bias vector of size 1*2 can be expressed as: B=(b1,b2).

[0047] Among them, b1 and b2 can also be equal to 0 in the initial stage, and then, during the training process, they can be updated accordingly to obtain the final specific values.

[0048] In addition, the matrix multiplication above is as follows: .

[0049] Based on the above content, on the one hand, by mining the trajectory data and monitoring images, the potential semantic information therein can be mined. Compared with traditional data analysis technology, the potential behavioral semantic features can be captured, and thus, the accuracy of judgment can be improved to a certain extent. In addition, after aggregating to form a global vector of the target person, through further disturbance application and removal, the fusion of the potential semantic information of the two dimensions of trajectory and image can be achieved in this process (the semantics of trajectory information is simple but continuous, and the semantics of image information is rich but local. Therefore, mutual reinforcement can be achieved through fusion, that is, both semantic richness and continuity are taken into account), so as to obtain a target person restoration vector with relatively higher representation accuracy, so that the reliability of the determined target safety monitoring results can be further improved, which can improve the relatively low reliability of power engineering safety monitoring in the prior art.

[0050] It should be further explained that, in the above step S110, the specific process of mining the target person monitoring image to form the target person image vector is not limited and can be selected according to actual application requirements.

[0051] For example, in a specific implementation, in order to improve the semantic representation reliability of the mined target person image vector, the above step S110 may further include the following contents: In the first step, the target person monitoring image can be convolved to obtain the monitoring image convolution vector corresponding to the target person monitoring image. Figure 5 As shown; In the second step, the target person surveillance image may be subjected to contour extraction (for example, the Canny algorithm may be used to extract the contour) to obtain a corresponding contour image, wherein the value corresponding to the pixel points belonging to the contour in the contour image may be 1 (or, in other embodiments, may also be 255), and the value corresponding to the pixel points not belonging to the contour may be 0; In a third step, the contour image may be convolved to obtain a contour image convolution vector corresponding to the contour image, wherein the contour image convolution vector and the monitoring image convolution vector may have the same size; In the fourth step, the monitoring image convolution vector may be enhanced based on the contour image convolution vector to obtain the target person image vector.

[0052] In a specific implementation, the above-mentioned strengthening process may include the following contents: In the first step, the contour image may be segmented to form at least one contour segmentation image, wherein the principle of segmentation may be to obtain as many contour segmentation images as possible, and on this basis, it is necessary to ensure that the pixels on the same contour are in one contour segmentation image; In the second step, for each of the contour segmentation images, a first local vector having the same position coordinates as the contour segmentation image can be extracted from the monitoring image convolution vector, and a second local vector having the same position coordinates as the contour segmentation image can be extracted from the contour image convolution vector. Figure 6 , wherein the target person monitoring image, the monitoring image convolution vector, the contour image and the contour image convolution vector all have the same size; In the third step, for each of the second local vectors, based on the second local vector, cross-attention processing can be performed on the corresponding first local vector to obtain the first local enhancement vector corresponding to the second local vector, and, based on the distribution relationship between the corresponding contour segmentation images, the first local enhancement vectors corresponding to each of the second local vectors can be spliced ​​to form a spliced ​​local enhancement vector, and the spliced ​​local enhancement vector and the surveillance image convolution vector can be added or the mean is calculated to obtain the target person image vector.

[0053] It should be further explained that, in the above step S120, the specific process of aggregating the target person image vector into the target person trajectory vector is not limited and can be selected according to actual application requirements.

[0054] For example, in a specific implementation, in order to improve the efficiency of vector aggregation, the above step S120 may include the following content: concatenating the target person image vector and the target person trajectory vector end to end to form a global vector of the target person.

[0055] For example, in another specific implementation, in order to improve the reliability of vector aggregation, the above step S120 may further include the following contents (combined with Figure 7 ): In the first step, the target person trajectory vector may be segmented at least once to form at least one segmentation position; illustratively, the target person trajectory vector may be expanded into a row vector (or the target person trajectory vector processed in the aforementioned step is a row vector), such as [Y0, Y1, Y2, Y3, Y4, Y5, Y6, Y7, Y8, Y9]. For example, a segmentation may be performed once to form a segmentation position between Y4 and Y5. In a second step, the target person image vector may be segmented at least once to form at least two local person image vectors, or the target person image vector may be used as a local person image vector, wherein the number of the formed local person image vectors is equal to the number of the segmented positions, for example, one segmented position and one local person image vector are formed; In the third step, image semantic identifiers may be configured at the beginning and the end of each of the local personnel image vectors to form a new local personnel image vector, such as the image semantic identifier at the beginning is start, and the image semantic identifier at the end is finish, such as [start, U0, U1, U2, U3, U4, finish]. By way of example, in other implementations, image semantic identifiers may not be configured. In the fourth step, a new local person image vector corresponding to each of the segmented positions can be spliced ​​to form a global vector of the target person, such as [Y0, Y1, Y2, Y3, Y4, start, U0, U1, U2, U3, U4, finish, Y5, Y6, Y7, Y8, Y9].

[0056] It should be further explained that, in the above step S130, the specific process of hiding the semantic vector parameters of the specified size in the global vector of the target person is not limited and can be selected according to actual application requirements.

[0057] For example, in a specific implementation, the semantic vector parameters of the target size may be randomly selected from the target person global vector for hiding.

[0058] For another example, in another specific implementation, in order to improve the reliability of semantic vector parameter hiding, the above step S130 may further include step S131 and step S132, and the specific implementation process of each step is described as follows.

[0059] Step S131, determining the hidden coordinates of the global vector of the target person by interfering with the forward processing network, and outputting a coordinate set that matches the quantity ratio of the specified size representation.

[0060] In an embodiment of the present invention, the hidden coordinates of the global vector of the target person can be determined by an interference forward processing network, and a coordinate set matching the quantitative proportion of the specified size representation can be output, that is, the coordinates to be hidden are determined by the interference forward processing network, so that the reliability of the determined hidden coordinates is relatively high. Among them, the interference forward processing network is formed by joint training with the interference backward processing network, and the interference backward processing network is used to remove disturbances from the disturbance vector of the target person. It can be understood that both the interference forward processing network and the interference backward processing network are neural networks, which can be trained together with the word embedding model and the convolutional network used to execute step S110.

[0061] Step S132: hiding the semantic vector parameters corresponding to the coordinate set in the global vector of the target person, so that disturbance is applied to the global vector of the target person to form a disturbance vector of the target person.

[0062] In an embodiment of the present invention, after obtaining the coordinate set, the semantic vector parameters corresponding to the coordinate set in the target person's global vector can be hidden, so that disturbance is applied to the target person's global vector to form a target person's disturbance vector, wherein the specific hiding method can refer to the relevant description in the previous text and will not be repeated here.

[0063] It is necessary to further explain the above step S131 that, in the above step S131, the specific process of determining the hidden coordinates of the global vector of the target person by interfering with the forward processing network is not limited and can be selected according to actual needs.

[0064] For example, in a specific implementation, since the target person global vector is a semantic vector obtained by aggregating semantic vectors of different dimensions, the interference forward processing network can assign a weight to each semantic vector parameter in the target person global vector, and this weight can reflect the importance of different semantic vector parameters for hidden coordinate prediction. By adjusting these weights, it can help the interference forward processing network to balance the impact of hiding semantic vector parameters of different dimensions on subsequent processing. Based on this, the above step S131 can further include the following content: In the first step, the global vector of the target person may be loaded into the interference forward processing network, that is, the interference forward processing network may be used for subsequent processing; In the second step, the coordinate distribution parameters corresponding to the global vector of the target person can be determined, wherein the coordinate distribution parameters are used to reflect the coordinates of the plurality of semantic vector parameters in the global vector of the target person, and the size of the coordinate distribution parameters can be the same as the size of the global vector of the target person, so that each parameter in the coordinate distribution parameters is used to reflect the coordinates of a semantic vector parameter in the global vector of the target person. Specifically, a coordinate value can be mapped to a word vector, and then the mean or maximum value of each parameter in the word vector is calculated as a parameter corresponding to the coordinate value, thereby forming the corresponding coordinate distribution parameters; In the third step, the target person's global vector and the coordinate distribution parameters may be cascaded to form a cascade vector. For example, the coordinate distribution parameters may be connected to the back end of the target person's global vector, i.e., spliced, to obtain a large-sized cascade vector. In the fourth step, the cascade vector may be saliency mined to form a saliency mining vector; illustratively, the interference forward processing network may include a query matrix, a key matrix and a value matrix, and then the query matrix, the key matrix and the value matrix may be multiplied with the cascade vector respectively to obtain the corresponding query vector, the key vector and the value vector, and then the dot product between the query vector and the transposed result of the key vector may be calculated, and finally, based on the dot product, the value vector may be weighted summed to obtain the corresponding saliency mining vector; In the fifth step, the hidden coordinates can be evaluated based on the saliency mining vector to form hidden coordinate evaluation parameters; illustratively, the saliency mining vector can be mapped through the fully connected network layer in the interference forward processing network to obtain a fully connected vector with the same size as the target person global vector, and then the fully connected vector can be output (such as through a softmax function) to obtain a probability distribution, i.e., a hidden coordinate evaluation parameter, wherein each probability value can represent the probability of the corresponding coordinate being a hidden coordinate; In the sixth step, a coordinate set matching the percentage of the number of representations of the specified size can be generated based on the hidden coordinate evaluation parameters, wherein the hidden coordinate evaluation parameters are used to reflect the possibility of each of the semantic vector parameters being hidden. For example, the coordinates of the percentage of the number of representations of the specified size that have the highest probability of being hidden coordinates can be combined to form a coordinate set.

[0065] It should be further explained that, in step S140 in the above embodiment, the specific process of removing the disturbance of the target person disturbance vector is not limited and can be selected according to actual application requirements.

[0066] For example, in a specific implementation, in order to remove disturbance from the target person disturbance vector according to information learned from the training data to improve the reliability of disturbance removal, the above-mentioned step S140 may further include step S141 and step S142, the specific contents of which are as follows.

[0067] Step S141, removing disturbance from the target person disturbance vector through an interference backward processing network, and outputting disturbance removal data of the current processing stage.

[0068] In an embodiment of the present invention, the disturbance vector of the target person can be disturbed by the interference backward processing network, and the disturbance removal data of the current processing stage can be output. The disturbance imposed on the global vector of the target person is achieved by the interference forward processing network, as described above.

[0069] Step S142, removing disturbance from the disturbance removal data of the current processing stage through the interference backward processing network, and outputting a target person restoration vector when the current processing stage belongs to the target processing stage.

[0070] In an embodiment of the present invention, after obtaining the disturbance removal data of the current processing stage, the disturbance removal data of the current processing stage can be disturbed by the interference backward processing network (i.e., the disturbance removal data of the current processing stage is used as a new target person disturbance vector), and when the current processing stage belongs to the target processing stage, the target person restoration vector is output. In other words, the target person disturbance vector can be subjected to at least two stages of disturbance removal, so that the reliability of the removal of the disturbance information is higher, thereby obtaining a reliable target person restoration vector. Among them, the specific number of stages of the target processing stage is not limited and can be selected according to actual conditions. For example, based on the efficiency requirement, the smaller the number of stages, based on the accuracy requirement, the larger the number of stages.

[0071] It should be further explained that in the above step S141, the specific process of removing the disturbance of the target person disturbance vector through the interference backward processing network is not limited and can be selected according to the actual situation.

[0072] For example, in a specific implementation, in order to fully remove the disturbance information in the target person disturbance vector by capturing context-related semantic information, the above-mentioned step S141 may further include step S141a, step S141b, step S141c, step S141d and step S141e, and the specific content of each step is described as follows.

[0073] Step S141a, loading the target person disturbance vector to load into the interference backward processing network.

[0074] In the embodiment of the present invention, the target person disturbance vector may be loaded to be loaded into an interference backward processing network, that is, subsequent interference removal processing is performed in the interference backward processing network. The specific processing process is described below.

[0075] Step S141b, calculating the first mapping parameter, the second mapping parameter and the third mapping parameter of each of the semantic vector parameters.

[0076] In an embodiment of the present invention, the first mapping parameter, the second mapping parameter and the third mapping parameter of each of the semantic vector parameters can be calculated in the interference backward processing network. For example, the interference backward processing network includes the first mapping vector, the second mapping vector and the third mapping vector formed by training, and the sizes of the three mapping vectors can be consistent with the size of the target person disturbance vector. In this way, for each semantic vector parameter in the target person disturbance vector, the semantic vector parameter can be multiplied with the parameters of the corresponding positions in the first mapping vector, the second mapping vector and the third mapping vector to obtain the corresponding first mapping parameter, the second mapping parameter and the third mapping parameter.

[0077] Step S141c, polling multiple semantic vector parameters, and performing a dot multiplication operation based on the first mapping parameter of the currently polled semantic vector parameter and the second mapping parameter of each other semantic vector parameter, outputting a significance parameter between the currently polled semantic vector parameter and each other semantic vector parameter.

[0078] In an embodiment of the present invention, after obtaining the first mapping parameter, the second mapping parameter and the third mapping parameter of each semantic vector parameter, the plurality of semantic vector parameters can be polled, and a dot multiplication operation is performed based on the first mapping parameter of the currently polled semantic vector parameter and the second mapping parameter of each other semantic vector parameter, and the significance parameter between the currently polled semantic vector parameter and each other semantic vector parameter, i.e., the result of the dot multiplication, is output. In this way, each semantic vector parameter can be polled, so that the significance parameter between each semantic vector parameter and each other semantic vector parameter can be obtained. For example, the target person disturbance vector may be [a1, a2, a3]. Thus, for a1, the product of the first mapping parameter corresponding to a1 and the second mapping parameter corresponding to a2, and the product of the first mapping parameter corresponding to a1 and the second mapping parameter corresponding to a3 may be calculated respectively; for a2, the product of the first mapping parameter corresponding to a2 and the second mapping parameter corresponding to a1, and the product of the first mapping parameter corresponding to a2 and the second mapping parameter corresponding to a3 may be calculated respectively; for a3, the product of the first mapping parameter corresponding to a3 and the second mapping parameter corresponding to a1, and the product of the first mapping parameter corresponding to a3 and the second mapping parameter corresponding to a1 may be calculated respectively. Thus, the significance parameters may be obtained.

[0079] Step S141d, polling the multiple semantic vector parameters, and using the respective significance parameters of the currently polled semantic vector parameters as weight coefficients to fuse the third mapping parameters of the other semantic vector parameters, and outputting the significance fusion parameters of the currently polled semantic vector parameters.

[0080] In an embodiment of the present invention, after obtaining the corresponding significance parameter, multiple semantic vector parameters can be polled, and each significance parameter of the currently polled semantic vector parameter can be used as a weight coefficient to fuse the third mapping parameters of other semantic vector parameters, and output the significance fusion parameter of the currently polled semantic vector parameter. For example, for a1, the corresponding significance fusion parameter = (the first mapping parameter corresponding to a1 * the second mapping parameter corresponding to a2) * the third mapping parameter corresponding to a2 + (the first mapping parameter corresponding to a1 * the second mapping parameter corresponding to a3) * the third mapping parameter corresponding to a2.

[0081] Step S141e, merging the saliency fusion parameters of the plurality of semantic vector parameters to obtain disturbance removal data of the current processing stage formed after disturbance removal.

[0082] In an embodiment of the present invention, after obtaining each saliency fusion parameter, the saliency fusion parameters of multiple semantic vector parameters can be merged to obtain disturbance removal data of the current processing stage formed after disturbance removal, such as [saliency fusion parameter corresponding to a1, saliency fusion parameter corresponding to a2, saliency fusion parameter corresponding to a3].

[0083] Regarding the above step S141b, in a specific implementation, the following contents may be included: In the first step, the first transformation parameter, the second transformation parameter and the third transformation parameter of each of the semantic vector parameters can be determined from the interference backward processing network. For example, the first transformation parameter, the second transformation parameter and the third transformation parameter can be determined based on the first mapping vector, the second mapping vector and the third mapping vector formed by training in the interference backward processing network, that is, the parameters of the positions corresponding to the semantic vector parameters in the first mapping vector, the second mapping vector and the third mapping vector are used as the first transformation parameter, the second transformation parameter and the third transformation parameter, respectively; In the second step, for each of the semantic vector parameters, the semantic vector parameter is multiplied by the first transformation parameter to output the first mapping parameter of the semantic vector parameter, and the semantic vector parameter is multiplied by the second transformation parameter to output the second mapping parameter of the semantic vector parameter, and the semantic vector parameter is multiplied by the third transformation parameter to output the third mapping parameter of the semantic vector parameter.

[0084] In conjunction with the content of the above step S140, it is also necessary to explain the above step S130 that, in order to ensure the effective implementation of the above step S130, the specified size can also be determined first, that is, before the step of hiding the semantic vector parameters of the specified size in the global vector of the target person so as to apply disturbance to the global vector of the target person to form a disturbance vector of the target person, the power engineering safety monitoring method based on deep learning also includes: In the first step, a target parameter may be determined based on the target processing stage, wherein the target parameter is a randomly generated parameter less than the target processing stage. For example, when the target processing stage is 10, the target parameter may be 2, 3, 4, etc.; In the second step, the specified size can be obtained based on the ratio between the target parameter and the target processing stage, wherein the specified size is used to reflect the proportion of the number of semantic vector parameters hidden in the global vector of the target person, such as 20%, 30%, 40%, etc.

[0085] It should be noted that, assuming that the target processing stage is 10, the maximum target parameter determined is 9, that is, the maximum value of the specified size is 90%; assuming that the target processing stage is 20, the maximum target parameter determined is 19, and the maximum value of the specified size is 95%. It is very obvious that 95% is greater than 90%, that is, as the target processing stage increases, the determined specified size can also increase with a certain probability, that is, with a certain probability, more disturbance information can be removed through more disturbance removal stages to improve reliability.

[0086] It should also be noted that for the above-mentioned steps S110 to S140, in order to ensure the effective implementation of steps S110 to S140, they can be implemented by a corresponding neural network model, and the neural network model can be a trained neural network model to learn the semantic mapping relationship in the corresponding training data, that is, before the steps of determining the target personnel trajectory data and the target personnel monitoring image of the target power engineering area, and mining the target personnel trajectory data to form a target personnel trajectory vector, and mining the target personnel monitoring image to form a target personnel image vector, the power engineering safety monitoring method based on deep learning can also include: In the first step, the trainee trajectory data and the corresponding trainee monitoring image can be determined, and the trainee trajectory data is mined to form a trainee trajectory vector, and the trainee monitoring image is mined to form a trainee image vector. The specific processing process of this step can refer to the relevant explanation of step S110 above. In the second step, after the training personnel image vector is aggregated into the training personnel trajectory vector, disturbance is applied by using the interference forward processing network to form a training personnel disturbance vector, and the interference backward processing network is used to remove the disturbance of the training personnel disturbance vector to form a training personnel restoration vector. The specific processing process of this step can refer to the relevant explanations of steps S120 to S140 in the previous text; In the third step, a training safety monitoring result can be determined based on the training personnel restoration vector (refer to the relevant explanation of step S150 in the previous text), and based on the error between the training safety monitoring result and the corresponding safety label data (such as the cross entropy error between the corresponding probability distributions), the interference forward processing network and the interference backward processing network are trained to form a trained interference forward processing network and a trained interference backward processing network. For example, the network parameters of the interference forward processing network and the interference backward processing network can be updated and adjusted in the direction of reducing the error so that the error converges, such as being less than a preset value.

[0087] Among them, the calculation formula of cross entropy error is: Cross-Entropy Loss=−(y*log(p)+(1−y)*log(q)); Among them, y is the probability corresponding to the security label data, such as 0 or 1, p and q are the probabilities corresponding to the training security monitoring results (such as 0.8 and 0.2). In this way, Cross-Entropy Loss=−(1*log(0.8)+(1-1)*log(0.2))=−log(0.8)≈0.2231.

[0088] In summary, the power engineering safety monitoring method and system based on deep learning provided by the present invention first mine the target personnel trajectory data to form a target personnel trajectory vector, and mine the target personnel monitoring image to form a target personnel image vector; secondly, the target personnel image vector is aggregated into the target personnel trajectory vector to form a target personnel global vector; then, the semantic vector parameters of the specified size in the target personnel global vector are hidden, so that disturbance is applied to the target personnel global vector to form a target personnel disturbance vector; further, the target personnel disturbance vector is disturbed and removed to form a target personnel restoration vector; finally, the target safety monitoring result is determined based on the target personnel restoration vector. Based on the above content, on the one hand, by mining the trajectory data and monitoring images, the potential semantic information therein can be mined. Compared with traditional data analysis technology, the potential behavioral semantic features can be captured, and thus, the accuracy of judgment can be improved to a certain extent. In addition, after aggregating to form a global vector of the target person, through further disturbance application and removal, the fusion of the potential semantic information of the two dimensions of trajectory and image can be achieved in this process (the semantics of trajectory information is simple but continuous, and the semantics of image information is rich but local. Therefore, mutual reinforcement can be achieved through fusion, that is, both semantic richness and continuity are taken into account), so as to obtain a target person restoration vector with relatively higher representation accuracy, so that the reliability of the determined target safety monitoring results can be further improved, which can improve the relatively low reliability of power engineering safety monitoring in the prior art.

[0089] In several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the apparatus, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0090] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0091] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code. It should be noted that in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0092] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A power engineering safety monitoring method based on deep learning, characterized in that: include: Determine the target personnel trajectory data and the target personnel monitoring image of the target power engineering area, and mine the target personnel trajectory data to form a target personnel trajectory vector, and mine the target personnel monitoring image to form a target personnel image vector, wherein the mining process of the target personnel trajectory data at least includes embedding processing of a word embedding model, and the mining process of the target personnel monitoring image at least includes convolution processing of a convolutional network; Aggregating the target person image vector into the target person trajectory vector to form a target person global vector, wherein the target person global vector includes a plurality of semantic vector parameters, and the target person global vector is used to represent global semantic information in an image dimension and a trajectory dimension; Hiding the semantic vector parameters of a specified size in the global vector of the target person, so that a disturbance is applied to the global vector of the target person to form a target person disturbance vector, wherein the specified size is used to reflect the number of the hidden semantic vector parameters, and the hidden semantic vector parameters are determined by evaluating the probability of each semantic vector parameter in the global vector of the target person being hidden; Performing disturbance removal on the target person disturbance vector to form a target person restoration vector, wherein the disturbance removal is achieved by utilizing context-related semantic information in the target person disturbance vector, so that interactive fusion of context-related semantic information is achieved in the process of disturbance removal; A target safety monitoring result is determined based on the target personnel restoration vector, wherein the process of determining the target safety monitoring result includes at least full connection processing, which is used to map the target personnel restoration vector to the probability of whether there is an abnormal construction situation, and the target safety monitoring result is used to reflect whether there is an abnormal construction situation in the target power engineering area.

2. The power engineering safety monitoring method based on deep learning according to claim 1 is characterized in that: The step of aggregating the target person image vector into the target person trajectory vector to form a target person global vector comprises: Segmenting the target person's trajectory vector at least once to form at least one segmentation position; Perform at least one segmentation on the target person image vector to form at least two partial person image vectors, or use the target person image vector as a partial person image vector, wherein the number of the partial person image vectors formed is equal to the number of the segmentation positions; respectively configuring image semantic identifiers at the beginning and the end of each of the local person image vectors to form a new local person image vector; A new local person image vector corresponding to each segmentation position is spliced ​​to form a global vector of the target person.

3. The power engineering safety monitoring method based on deep learning according to claim 1 is characterized in that: The step of hiding the semantic vector parameters of a specified size in the global vector of the target person so as to apply disturbance to the global vector of the target person to form a disturbance vector of the target person includes: Determine the hidden coordinates of the global vector of the target person through an interference forward processing network, and output a coordinate set that matches the quantity ratio of the specified size representation, wherein the interference forward processing network is jointly trained with the interference backward processing network, and the interference backward processing network is used to remove the disturbance of the target person disturbance vector; The semantic vector parameters corresponding to the coordinate set in the global vector of the target person are hidden, so that disturbance is applied to the global vector of the target person to form a target person disturbance vector, wherein the hidden semantic vector parameters are replaced with random values ​​or specific values, and the specific value includes 0.

4. The power engineering safety monitoring method based on deep learning according to claim 3 is characterized in that: The step of determining the hidden coordinates of the target person's global vector by interfering with the forward processing network and outputting a coordinate set that matches the quantity ratio of the specified size representation includes: Loading the target person global vector to load into the interference forward processing network; Determine a coordinate distribution parameter corresponding to the global vector of the target person, wherein the coordinate distribution parameter is used to reflect the coordinates of the plurality of semantic vector parameters in the global vector of the target person; Cascading the target person's global vector and the coordinate distribution parameter to form a cascade vector; The cascade vector is saliency mined to form a saliency mining vector, wherein the interference forward processing network includes a query matrix, a key matrix and a value matrix, and the saliency mining process includes: multiplying the query matrix, the key matrix and the value matrix with the cascade vector respectively to obtain a corresponding query vector, a key vector and a value vector, calculating a dot product between the query vector and the transposed result of the key vector, and based on the dot product, performing a weighted summation on the value vector to obtain a corresponding saliency mining vector; Evaluate the hidden coordinates according to the saliency mining vector to form hidden coordinate evaluation parameters, wherein the evaluation process includes: mapping the saliency mining vector through the fully connected network layer in the interference forward processing network to obtain a fully connected vector with the same size as the global vector of the target person, and outputting the fully connected vector through a softmax function to obtain a probability distribution as the hidden coordinate evaluation parameter; A coordinate set matching the quantity ratio of the specified size representation is generated according to the hidden coordinate evaluation parameter, wherein the hidden coordinate evaluation parameter is used to reflect the possibility of each of the semantic vector parameters being hidden.

5. The power engineering safety monitoring method based on deep learning according to claim 1 is characterized in that: The step of removing disturbance from the target person disturbance vector to form a target person restoration vector comprises: De-disturb the target person's disturbance vector through an interference backward processing network, and output disturbance removal data of the current processing stage, wherein the perturbation applied to the target person's global vector is achieved through an interference forward processing network; The disturbance removal data of the current processing stage is disturbed by the interference backward processing network, and when the current processing stage belongs to the target processing stage, the target person restoration vector is output.

6. The power engineering safety monitoring method based on deep learning according to claim 5 is characterized in that: The step of removing disturbance from the target person disturbance vector through the interference backward processing network and outputting disturbance removal data of the current processing stage includes: Loading the target person disturbance vector to load into the interference backward processing network; Calculating a first mapping parameter, a second mapping parameter and a third mapping parameter of each of the semantic vector parameters; Polling a plurality of the semantic vector parameters, and performing a dot multiplication operation based on the first mapping parameter of the currently polled semantic vector parameter and the second mapping parameter of each of the other semantic vector parameters, and outputting a significance parameter between the currently polled semantic vector parameter and each of the other semantic vector parameters; Polling a plurality of the semantic vector parameters, and using each of the significance parameters of the currently polled semantic vector parameters as a weight coefficient to fuse the third mapping parameters of the other semantic vector parameters, and outputting the significance fusion parameter of the currently polled semantic vector parameter, wherein the fusion process refers to using the significance parameter as a weight parameter to perform a weighted sum calculation on the third mapping parameters of the various semantic vector parameters; The saliency fusion parameters of the plurality of semantic vector parameters are combined to obtain disturbance removal data of the current processing stage formed after disturbance removal.

7. The power engineering safety monitoring method based on deep learning according to claim 6 is characterized in that: The step of calculating the first mapping parameter, the second mapping parameter and the third mapping parameter of each of the semantic vector parameters comprises: Determining a first transformation parameter, a second transformation parameter and a third transformation parameter of each of the semantic vector parameters from the interference backward processing network; For each of the semantic vector parameters, the semantic vector parameter is multiplied by the first transformation parameter to output the first mapping parameter of the semantic vector parameter, and the semantic vector parameter is multiplied by the second transformation parameter to output the second mapping parameter of the semantic vector parameter, and the semantic vector parameter is multiplied by the third transformation parameter to output the third mapping parameter of the semantic vector parameter.

8. The power engineering safety monitoring method based on deep learning according to claim 5 is characterized in that: Before the step of hiding the semantic vector parameters of the specified size in the global vector of the target person so as to apply disturbance to the global vector of the target person to form a disturbance vector of the target person, the power engineering safety monitoring method based on deep learning further includes: Based on the target processing stage, determining a target parameter, wherein the target parameter is a randomly generated parameter that is less than the target processing stage; The specified size is obtained based on the ratio between the target parameter and the target processing stage, wherein the specified size is used to reflect the proportion of the number of semantic vector parameters to be hidden in the global vector of the target person.

9. The power engineering safety monitoring method based on deep learning according to any one of claims 1 to 8, characterized in that: Before the steps of determining the target personnel trajectory data and the target personnel monitoring image in the target power engineering area, mining the target personnel trajectory data to form a target personnel trajectory vector, and mining the target personnel monitoring image to form a target personnel image vector, the power engineering safety monitoring method based on deep learning also includes: Determine trainee trajectory data and corresponding trainee monitoring images, and mine the trainee trajectory data to form a trainee trajectory vector, and mine the trainee monitoring images to form a trainee image vector; After the training person image vector is aggregated into the training person trajectory vector, disturbance is applied using a disturbance forward processing network to form a training person disturbance vector, and disturbance is removed from the training person disturbance vector using a disturbance backward processing network to form a training person restoration vector; Based on the training personnel restoration vector, a training safety monitoring result is determined, and based on the error between the training safety monitoring result and the corresponding safety label data, the interference forward processing network and the interference backward processing network are trained to form a trained interference forward processing network and a trained interference backward processing network.

10. A power engineering safety monitoring system based on deep learning, characterized in that: include: Memory for storing computer programs; A processor connected to the memory is used to execute a computer program stored in the memory to implement the deep learning-based power engineering safety monitoring method described in any one of claims 1 to 9.

Images