Human body dangerous behavior identification method and system based on generative adversarial network

Abstract

The invention relates to the field of image recognition, and provides a human body dangerous behavior recognition method and system based on a generative adversarial network, and the method comprises the steps: carrying out the region segmentation of a video image, and carrying out the segmentation of a human body behavior region and a background region in the image, so as to avoid the interference of background factors, reduce the difficulty of human body behavior recognition, and improve the recognition efficiency. The method improves the recognition accuracy, extracts the human behavior characteristics and motion information according to a direction gradient algorithm, enriches the dynamic characteristic information, enhances the context relation, improves the recognition accuracy in a time sequence, and generates predicted human behavior characteristics according to a generative adversarial network comprising an information enhancement module. According to the human body dangerous behavior identification method, the long-distance dependency relationship is enhanced, the prediction further conforms to the actual situation, and the accuracy of the human body dangerous behavior identification method is improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method and system for identifying dangerous human behaviors based on a generative adversarial network. Background Art

[0002] With the rapid development of the railway industry, the number of railway maintenance depots is increasing, and the safety requirements for railway maintenance depots are also increasing. Especially when maintenance personnel are carrying out maintenance work, any omissions or illegal operations in any link may cause serious safety accidents and lead to the occurrence of dangerous operations. Therefore, it is crucial to promptly discover the dangerous behaviors of maintenance personnel during the maintenance work.

[0003] In the prior art, the maintenance process of railway maintenance stations is usually monitored by manual inspection and fixed video surveillance. Both manual inspection and video surveillance require the assistance of safety personnel, which has high manpower and time costs. In addition, manual monitoring cannot achieve real-time full coverage, resulting in insufficient real-time monitoring efficiency. Therefore, computer vision technology has become an important means of real-time monitoring and identification. However, the environment of railway maintenance depots is very complex, and there are many people working at the same time, which has caused great interference to the identification of dangerous behaviors. As a result, the image recognition method of existing computer vision technology cannot meet the requirements for dangerous behavior identification in railway maintenance depots.

[0004] Therefore, how to design a dangerous behavior identification method to avoid the complex environment and interference factors of the railway maintenance depot and improve the accuracy of dangerous behavior identification. Summary of the invention

[0005] Based on this, the present invention provides a method and system for identifying dangerous human behaviors based on a generative adversarial network. The method first performs regional segmentation on a video image, and then segments the human behavior region and the background region in the image to avoid interference from background factors, thereby reducing the difficulty of human behavior recognition and improving recognition accuracy. The human behavior characteristics and motion information are extracted according to a directional gradient algorithm, dynamic feature information is enriched, contextual connections are enhanced, and recognition accuracy in a time series is improved. The predicted human behavior characteristics are then generated according to a generative adversarial network including an information enhancement module to obtain a final recognition result, thereby enhancing long-distance dependencies and making the prediction further conform to actual conditions. The present invention improves the accuracy of the method for identifying dangerous human behaviors.

[0006] The present invention proposes a method for identifying dangerous human behaviors based on a generative adversarial network, comprising: Acquire real-time surveillance video images and perform preprocessing; Performing human behavior region segmentation on the preprocessed video image to obtain a human behavior region image, wherein the human behavior region segmentation refers to segmenting a foreground region and a background region in the video image, wherein the foreground region includes all moving human targets, and the background region includes all fixed environmental targets; Extracting human behavior features according to the human behavior region image, wherein the extracted human behavior features include directional gradient feature extraction and motion information extraction; Predicted human behavior features are generated according to a generative adversarial network to obtain a final recognition result according to the predicted human behavior features. The generative adversarial network includes an information enhancement module and calculates a dangerousness score.

[0007] In summary, according to the above-mentioned method for identifying dangerous human behaviors based on a generative adversarial network, by first performing regional segmentation on the video image, the human behavior area and the background area in the image are segmented to avoid interference from background factors, reduce the difficulty of human behavior recognition, and improve the accuracy of recognition. According to the directional gradient algorithm, human behavior characteristics and motion information are extracted, dynamic feature information is enriched, and the context connection is enhanced, thereby improving the recognition accuracy in the time series. Then, according to the generative adversarial network including the information enhancement module, predicted human behavior characteristics are generated to obtain the final recognition result, which enhances the long-distance dependency and makes the prediction further conform to the actual situation. The present invention improves the accuracy of the method for identifying dangerous human behaviors. Specifically, real-time monitoring video images are acquired and preprocessed, and human behavior area segmentation is performed on the preprocessed video images to acquire human behavior area images, wherein the human behavior area segmentation refers to segmenting the foreground area and the background area in the video image, wherein the foreground area includes all moving human targets, and the background area includes all fixed environmental targets, so as to avoid interference from background factors, reduce the difficulty of human behavior recognition, and improve recognition accuracy, extract human behavior features according to the human behavior area image, wherein the extracted human behavior features include directional gradient feature extraction and motion information extraction, enrich dynamic feature information, enhance contextual connection, and thereby improve recognition accuracy in time series, generate predicted human behavior features according to a generative adversarial network, and obtain final recognition results according to the predicted human behavior features, wherein the generative adversarial network includes an information enhancement module, and calculates a risk score, enhances long-distance dependencies, and makes predictions further conform to actual conditions, and the present invention improves the accuracy of a method for identifying dangerous human behaviors.

[0008] Furthermore, the step of performing human behavior region segmentation on the preprocessed video image to obtain a human behavior region image specifically includes: Segment the foreground area and background area of ​​the video image according to the Gaussian mixture model; The specific algorithm of the Gaussian mixture model is as follows: , , , , , , , in, represents the probability density function of pixel color, represents the pixel color of a random pixel, Represents the pixel color value of a random pixel, represents the number of Gaussian distributions, represents the ordinal number of the Gaussian distribution, represents the weight of Gaussian distribution, Indicates the time, represents the Gaussian distribution function, represents the mean of the Gaussian distribution, represents the covariance matrix, represents a constant, represents transpose, The dimension representing the pixel color, represents a random pixel value, Indicates the change in pixel value. represents the variance of the Gaussian distribution, represents the parameter learning rate, represents an adjustable learning rate, Represents the ordinal number of the Gaussian distribution arranged by size, Indicates the weight threshold.

[0009] Furthermore, the step of extracting human behavior features according to the human behavior area image specifically includes: All pixel gradients are calculated based on the human behavior area image. The specific algorithm for calculating all pixel gradients is as follows: , , , in, Represents a single-frame human behavior area image, and denote the horizontal and vertical components of the finite difference approximation, represents the pixel gradient size, Indicates the pixel gradient direction; The human behavior area image is divided into a plurality of interval blocks, the number of the interval blocks is , each of the interval blocks includes pixels, each interval block is divided into multiple cell units, each cell unit includes pixels, a directional gradient feature is generated and normalized according to the gradient size and corresponding direction of the interval block. The dimension of the directional gradient feature is , the specific algorithm for generating directional gradient features and normalizing them is as follows: , in, represents the normalized directional gradient feature, represents the directional gradient feature, represents the norm; Calculate the average pixel value of each interval block; The specific algorithm for calculating the average pixel value of each interval block is as follows: , in, Represents the average pixel value of the interval block, Indicates the ordinal number of the interval block, represents the ordinal number of the pixel value, Indicates the size of the interval block. Represents the pixel value of a single pixel in the interval block; After normalizing the average pixel value, the feature weight is obtained. The specific algorithm for obtaining the directional gradient feature weight is as follows: , in, Represents the directional gradient feature weight; The directional gradient features are weighted according to the directional gradient feature weights to obtain the total directional gradient features of the single-frame human behavior region image. The specific algorithm for obtaining the total directional gradient features of the single-frame human behavior region image is as follows: , in, Represents the total directional gradient features of a single-frame human action area image, Indicates the time, Indicates interval blocks; According to the total directional gradient feature of the single-frame human behavior region image, the human behavior feature is obtained. The specific algorithm for obtaining the human behavior feature is as follows: , in, Represents human behavioral characteristics.

[0010] Furthermore, the step of generating predicted human behavior features according to the generative adversarial network to obtain the final recognition result according to the predicted human behavior features specifically includes: Generate predicted human behavior features according to a generative adversarial network, wherein the generative adversarial network includes a feature generation submodule and a feature discrimination submodule; The feature generation submodule includes a Transformer encoding block, an information enhancement block and a Transformer decoding block; The feature generation submodule generates predicted human behavior features based on human behavior features; The feature identification submodule performs dangerous behavior identification based on the predicted human behavior features and human behavior features to obtain a final identification result.

[0011] Furthermore, the information enhancement module specifically includes: The information enhancement module performs cosine similarity calculation based on the encoding features obtained after the Transformer encoding block is processed to obtain the feature similarity weight. The specific algorithm for obtaining the feature similarity weight is as follows: , in, represents the feature similarity weight, represents the cosine similarity function, represents the encoding feature, represents the storage features in the information enhancement module, O represents the total number of stored features, The ordinal number representing the stored feature; Weighted reconstruction is performed according to the feature similarity weights to obtain the reconstructed features. The specific algorithm for obtaining the reconstructed features is as follows: , in, Represents reconstruction features; Concatenate the encoded features and the reconstructed features to input into a Transformer decoding block; The storage features in the information enhancement module are updated according to the gating mechanism. The specific algorithm of the update is as follows: , , in, represents the output of the forget gate, represents the activation function, and represents the weight and bias of the forget gate, represents the normalization function, represents an element-wise multiplication operation, Indicates the matching index between the stored features and the encoded features.

[0012] Furthermore, the step of obtaining the final recognition result according to the predicted human behavior characteristics further includes: The loss of predicting human behavior characteristics is calculated and generated, and the algorithm for generating the loss of predicting human behavior characteristics is as follows: , in, represents the loss of generating predicted human behavior features, and They represent the predicted human behavior features and the input Transformer encoding block human behavior features, respectively. is the number of interval blocks, Indicates the ordinal number of the interval block, t Indicates the moment; The loss of information enhancement is calculated, and the algorithm of the loss of information enhancement is as follows: , , , in, represents the basic loss of information enhancement, represents the sparsity L1 regularization term, represents the regularization coefficient, represents the norm, represents the total loss of information enhancement, Represents the weight parameter of the sparsity L1 regularization term; The loss of the feature discrimination submodule is calculated. The specific algorithm of the loss of the feature discrimination submodule is as follows: , , , in, Represents resistance to loss, Indicates that it is judged as a real operation. represents the weighted discriminant loss, , and They represent the total loss weights of generating the predicted human behavior features, the total loss of information enhancement, and the total loss of discrimination loss, respectively. Represents the final loss function of the feature discriminant submodule.

[0013] Furthermore, the step of calculating the risk score specifically includes: The specific algorithm for calculating the risk score is as follows: , in, represents the risk score, represents the normalized risk score, and They represent the predicted human behavior features and the input Transformer encoding block human behavior features, respectively. is the number of interval blocks, Indicates the ordinal number of the interval block, t Indicates time.

[0014] The present invention proposes a human dangerous behavior recognition system based on a generative adversarial network, comprising: A preprocessing module is used to obtain real-time monitoring video images and perform preprocessing; A region segmentation module is used to perform human behavior region segmentation on the preprocessed video image to obtain a human behavior region image, wherein the human behavior region segmentation refers to segmenting a foreground region and a background region in the video image, wherein the foreground region includes all moving human targets, and the background region includes all fixed environmental targets; A feature extraction module, used for extracting human behavior features according to the human behavior region image, wherein the extracted human behavior features include directional gradient feature extraction and motion information extraction; The recognition module is used to generate predicted human behavior features according to a generative adversarial network to obtain a final recognition result according to the predicted human behavior features, wherein the generative adversarial network includes an information enhancement module and calculates a dangerousness score.

[0015] The present invention also provides a storage medium, which stores one or more programs, and when the programs are executed by a processor, the method for identifying dangerous human behaviors based on a generative adversarial network as described above is implemented.

[0016] The present invention also provides a computer device, the computer device comprising a memory and a processor, wherein: The memory is used to store computer programs; When the processor is used to execute the computer program stored in the memory, it implements the above-mentioned method for identifying dangerous human behaviors based on a generative adversarial network. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1A flowchart of a method for identifying dangerous human behaviors based on a generative adversarial network proposed in the first embodiment of the present invention; Figure 2 A flowchart of a method for identifying dangerous human behaviors based on a generative adversarial network proposed in a second embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a human dangerous behavior recognition system based on a generative adversarial network proposed in the third embodiment of the present invention.

[0018] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0019] In order to facilitate understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0020] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0022] See also Figure 1 , which is a flow chart of a method for identifying dangerous human behaviors based on a generative adversarial network according to a first embodiment of the present invention, wherein the method for identifying dangerous human behaviors based on a generative adversarial network comprises steps S01 to S04, wherein: Step S01: Acquire real-time monitoring video images and perform preprocessing; Step S02: performing human behavior region segmentation on the pre-processed video image to obtain a human behavior region image; It should be noted that in this embodiment, the foreground area and the background area of ​​the video image are segmented according to the Gaussian mixture model; The specific algorithm of the Gaussian mixture model is as follows: , , , , , , , in, represents the probability density function of pixel color, represents the pixel color of a random pixel, Represents the pixel color value of a random pixel, represents the number of Gaussian distributions, represents the ordinal number of the Gaussian distribution, represents the weight of Gaussian distribution, Indicates the time, represents the Gaussian distribution function, represents the mean of the Gaussian distribution, represents the covariance matrix, represents a constant, represents transpose, The dimension representing the pixel color, represents a random pixel value, Indicates the change in pixel value. represents the variance of the Gaussian distribution, represents the parameter learning rate, represents an adjustable learning rate, Represents the ordinal number of the Gaussian distribution arranged by size, Indicates the weight threshold.

[0023] Step S03: extracting human behavior features according to the human behavior area image; It should be noted that in this embodiment, all pixel gradients are calculated based on the human behavior area image. The specific algorithm for calculating all pixel gradients is as follows: , , , in, Represents a single-frame human behavior area image, and denote the horizontal and vertical components of the finite difference approximation, represents the pixel gradient size, Indicates the pixel gradient direction; The human behavior area image is divided into a plurality of interval blocks, the number of the interval blocks is , each of the interval blocks includes pixels, each interval block is divided into multiple cell units, each cell unit includes pixels, a directional gradient feature is generated and normalized according to the gradient size and corresponding direction of the interval block. The dimension of the directional gradient feature is , the specific algorithm for generating directional gradient features and normalizing them is as follows: , in, represents the normalized directional gradient feature, represents the directional gradient feature, represents the norm; Calculate the average pixel value of each interval block; The specific algorithm for calculating the average pixel value of each interval block is as follows: , in, Represents the average pixel value of the interval block, Indicates the ordinal number of the interval block, represents the ordinal number of the pixel value, Indicates the size of the interval block. Represents the pixel value of a single pixel in the interval block; After normalizing the average pixel value, the feature weight is obtained. The specific algorithm for obtaining the directional gradient feature weight is as follows: , in, Represents the directional gradient feature weight; The directional gradient features are weighted according to the directional gradient feature weights to obtain the total directional gradient features of the single-frame human behavior region image. The specific algorithm for obtaining the total directional gradient features of the single-frame human behavior region image is as follows: , in, Represents the total directional gradient features of a single-frame human action area image, Indicates the time, Indicates interval blocks; According to the total directional gradient feature of the single-frame human behavior region image, the human behavior feature is obtained. The specific algorithm for obtaining the human behavior feature is as follows: , in, Represents human behavioral characteristics.

[0024] Step S04: Generate predicted human behavior features according to the generative adversarial network, so as to obtain a final recognition result and calculate a risk score according to the predicted human behavior features; It should be noted that in this embodiment, the predicted human behavior features are generated according to the generative adversarial network, and the generative adversarial network includes a feature generation submodule and a feature discrimination submodule; The feature generation submodule includes a Transformer encoding block, an information enhancement block and a Transformer decoding block; The feature generation submodule generates predicted human behavior features based on human behavior features; The feature identification submodule performs dangerous behavior identification based on the predicted human behavior features and human behavior features to obtain a final identification result; The information enhancement module performs cosine similarity calculation based on the encoding features obtained after the Transformer encoding block is processed to obtain the feature similarity weight. The specific algorithm for obtaining the feature similarity weight is as follows: , in, represents the feature similarity weight, represents the cosine similarity function, represents the encoding feature, represents the storage features in the information enhancement module, O represents the total number of stored features, The ordinal number representing the stored feature; Weighted reconstruction is performed according to the feature similarity weights to obtain the reconstructed features. The specific algorithm for obtaining the reconstructed features is as follows: , in, Represents reconstruction features; Concatenate the encoded features and the reconstructed features to input into a Transformer decoding block; The storage features in the information enhancement module are updated according to the gating mechanism. The specific algorithm of the update is as follows: , , in, represents the output of the forget gate, represents the activation function, and represents the weight and bias of the forget gate, represents the normalization function, represents an element-wise multiplication operation, Represents the matching index between the stored features and the encoded features; The loss of predicting human behavior characteristics is calculated and generated, and the algorithm for generating the loss of predicting human behavior characteristics is as follows: , in, represents the loss of generating predicted human behavior features, and They represent the predicted human behavior features and the input Transformer encoding block human behavior features, respectively. is the number of interval blocks, Indicates the ordinal number of the interval block, t Indicates the moment; The loss of information enhancement is calculated, and the algorithm of the loss of information enhancement is as follows: , , , in, represents the basic loss of information enhancement, represents the sparsity L1 regularization term, represents the regularization coefficient, represents the norm, represents the total loss of information enhancement, Represents the weight parameter of the sparsity L1 regularization term; The loss of the feature discrimination submodule is calculated. The specific algorithm of the loss of the feature discrimination submodule is as follows: , , , in, Represents resistance to loss, Indicates that it is judged as a real operation. represents the weighted discriminant loss, , and They represent the total loss weights of generating the predicted human behavior features, the total loss of information enhancement, and the total loss of discrimination loss, respectively. Represents the final loss function of the feature discrimination submodule; The specific algorithm for calculating the risk score is as follows: , in, represents the risk score, represents the normalized risk score, and They represent the predicted human behavior features and the input Transformer encoding block human behavior features, respectively. is the number of interval blocks, Indicates the ordinal number of the interval block, t Indicates time.

[0025] In summary, according to the above-mentioned method for identifying dangerous human behaviors based on a generative adversarial network, by first performing regional segmentation on the video image, the human behavior area and the background area in the image are segmented to avoid interference from background factors, reduce the difficulty of human behavior recognition, and improve the accuracy of recognition. According to the directional gradient algorithm, human behavior characteristics and motion information are extracted, dynamic feature information is enriched, and the context connection is enhanced, thereby improving the recognition accuracy in the time series. Then, according to the generative adversarial network including the information enhancement module, predicted human behavior characteristics are generated to obtain the final recognition result, which enhances the long-distance dependency and makes the prediction further conform to the actual situation. The present invention improves the accuracy of the method for identifying dangerous human behaviors. Specifically, real-time monitoring video images are acquired and preprocessed, and human behavior area segmentation is performed on the preprocessed video images to acquire human behavior area images, wherein the human behavior area segmentation refers to segmenting the foreground area and the background area in the video image, wherein the foreground area includes all moving human targets, and the background area includes all fixed environmental targets, so as to avoid interference from background factors, reduce the difficulty of human behavior recognition, and improve recognition accuracy, extract human behavior features according to the human behavior area image, wherein the extracted human behavior features include directional gradient feature extraction and motion information extraction, enrich dynamic feature information, enhance contextual connection, and thereby improve recognition accuracy in time series, generate predicted human behavior features according to a generative adversarial network, and obtain final recognition results according to the predicted human behavior features, wherein the generative adversarial network includes an information enhancement module, and calculates a risk score, enhances long-distance dependencies, and makes predictions further conform to actual conditions, and the present invention improves the accuracy of a method for identifying dangerous human behaviors.

[0026] See also Figure 2 , which is a flow chart of a method for identifying dangerous human behaviors based on a generative adversarial network according to a second embodiment of the present invention, wherein the method for identifying dangerous human behaviors based on a generative adversarial network comprises steps S11 to S15, wherein: Step S11: segmenting the foreground area and the background area of ​​the video image according to the Gaussian mixture model; Step S12: calculating all pixel gradients according to the human behavior region image, dividing the human behavior region image into a plurality of interval blocks, dividing each interval block into a plurality of cell units, generating and normalizing directional gradient features according to the gradient size and corresponding direction of the interval block, calculating the average pixel value of each interval block, obtaining feature weights after normalizing the average pixel value, and weighting the directional gradient features according to the directional gradient feature weights to obtain the total directional gradient features of the single-frame human behavior region image, and obtaining the human behavior features according to the total directional gradient features of the single-frame human behavior region image; Step S13: Generate predicted human behavior features according to a generative adversarial network, the generative adversarial network includes a feature generation submodule and a feature discrimination submodule, the feature generation submodule generates predicted human behavior features according to human behavior features, and the feature discrimination submodule discriminates dangerous behaviors according to the predicted human behavior features and human behavior features to obtain a final recognition result; Step S14: Calculate the loss of generating predicted human behavior features, calculate the loss of information enhancement, and calculate the loss of feature discrimination submodule; Step S15: Calculate the risk score.

[0027] In summary, according to the above-mentioned method for identifying dangerous human behaviors based on a generative adversarial network, by first performing regional segmentation on the video image, the human behavior area and the background area in the image are segmented to avoid interference from background factors, reduce the difficulty of human behavior recognition, and improve the accuracy of recognition. According to the directional gradient algorithm, human behavior characteristics and motion information are extracted, dynamic feature information is enriched, and the context connection is enhanced, thereby improving the recognition accuracy in the time series. Then, according to the generative adversarial network including the information enhancement module, predicted human behavior characteristics are generated to obtain the final recognition result, which enhances the long-distance dependency and makes the prediction further conform to the actual situation. The present invention improves the accuracy of the method for identifying dangerous human behaviors. Specifically, real-time monitoring video images are acquired and preprocessed, and human behavior area segmentation is performed on the preprocessed video images to acquire human behavior area images, wherein the human behavior area segmentation refers to segmenting the foreground area and the background area in the video image, wherein the foreground area includes all moving human targets, and the background area includes all fixed environmental targets, so as to avoid interference from background factors, reduce the difficulty of human behavior recognition, and improve recognition accuracy, extract human behavior features according to the human behavior area image, wherein the extracted human behavior features include directional gradient feature extraction and motion information extraction, enrich dynamic feature information, enhance contextual connection, and thereby improve recognition accuracy in time series, generate predicted human behavior features according to a generative adversarial network, and obtain final recognition results according to the predicted human behavior features, wherein the generative adversarial network includes an information enhancement module, and calculates a risk score, enhances long-distance dependencies, and makes predictions further conform to actual conditions, and the present invention improves the accuracy of a method for identifying dangerous human behaviors.

[0028] See also Figure 3 , which is a schematic diagram of the structure of a human dangerous behavior recognition system based on a generative adversarial network proposed in the third embodiment of the present invention, and the system includes: A preprocessing module 10 is used to obtain real-time monitoring video images and perform preprocessing; A region segmentation module 20 is used to perform human behavior region segmentation on the preprocessed video image to obtain a human behavior region image, wherein the human behavior region segmentation refers to segmenting a foreground region and a background region in the video image, wherein the foreground region includes all moving human targets, and the background region includes all fixed environmental targets; A feature extraction module 30, for extracting human behavior features according to the human behavior region image, wherein the extracted human behavior features include directional gradient feature extraction and motion information extraction; The recognition module 40 is used to generate predicted human behavior features according to a generative adversarial network to obtain a final recognition result according to the predicted human behavior features, wherein the generative adversarial network includes an information enhancement module and calculates a risk score.

[0029] The present invention also proposes a computer storage medium on which one or more programs are stored, which, when executed by a processor, implement the above-mentioned method for identifying dangerous human behaviors based on a generative adversarial network.

[0030] The present invention also proposes a computer device, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the above-mentioned method for identifying dangerous human behaviors based on a generative adversarial network.

[0031] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, "computer-readable medium" may be any device that can contain storage, communication, propagation or transmission of a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0032] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0033] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0034] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0035] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A human dangerous behavior recognition method based on generative adversarial network, characterized in that: include: Acquire real-time surveillance video images and perform preprocessing; Performing human behavior region segmentation on the preprocessed video image to obtain a human behavior region image, wherein the human behavior region segmentation refers to segmenting a foreground region and a background region in the video image, wherein the foreground region includes all moving human targets, and the background region includes all fixed environmental targets; Extracting human behavior features according to the human behavior region image, wherein the extracted human behavior features include directional gradient feature extraction and motion information extraction; Predicted human behavior features are generated according to a generative adversarial network to obtain a final recognition result according to the predicted human behavior features. The generative adversarial network includes an information enhancement module and calculates a dangerousness score.

2. The method for identifying dangerous human behaviors based on a generative adversarial network according to claim 1 is characterized in that: The step of performing human behavior region segmentation on the preprocessed video image to obtain a human behavior region image specifically includes: Segment the foreground area and background area of ​​the video image according to the Gaussian mixture model; The specific algorithm of the Gaussian mixture model is as follows: ， ， ， ， ， ， ， in, represents the probability density function of pixel color, represents the pixel color of a random pixel, Represents the pixel color value of a random pixel, represents the number of Gaussian distributions, represents the ordinal number of the Gaussian distribution, represents the weight of Gaussian distribution, Indicates the time, represents the Gaussian distribution function, represents the mean of the Gaussian distribution, represents the covariance matrix, represents a constant, represents transpose, The dimension representing the pixel color, represents a random pixel value, Indicates the change in pixel value. represents the variance of the Gaussian distribution, represents the parameter learning rate, represents an adjustable learning rate, Represents the ordinal number of the Gaussian distribution arranged by size, Indicates the weight threshold.

3. The method for identifying dangerous human behaviors based on a generative adversarial network according to claim 1 is characterized in that: The step of extracting human behavior features according to the human behavior region image specifically comprises: All pixel gradients are calculated based on the human behavior area image. The specific algorithm for calculating all pixel gradients is as follows: ， ， ， in, Represents a single-frame human behavior area image, and denote the horizontal and vertical components of the finite difference approximation, represents the pixel gradient size, Indicates the pixel gradient direction; The human behavior area image is divided into a plurality of interval blocks, the number of the interval blocks is , each of the interval blocks includes pixels, each interval block is divided into multiple cell units, each cell unit includes pixels, a directional gradient feature is generated and normalized according to the gradient size and corresponding direction of the interval block. The dimension of the directional gradient feature is , the specific algorithm for generating directional gradient features and normalizing them is as follows: ， in, represents the normalized directional gradient feature, represents the directional gradient feature, represents the norm; Calculate the average pixel value of each interval block; The specific algorithm for calculating the average pixel value of each interval block is as follows: ， in, Represents the average pixel value of the interval block, Indicates the ordinal number of the interval block, represents the ordinal number of the pixel value, Indicates the size of the interval block. Represents the pixel value of a single pixel in the interval block; After normalizing the average pixel value, the feature weight is obtained. The specific algorithm for obtaining the directional gradient feature weight is as follows: ， in, Represents the directional gradient feature weight; The directional gradient features are weighted according to the directional gradient feature weights to obtain the total directional gradient features of the single-frame human behavior region image. The specific algorithm for obtaining the total directional gradient features of the single-frame human behavior region image is as follows: ， in, Represents the total directional gradient features of a single-frame human action area image, Indicates the time, Indicates interval blocks; According to the total directional gradient feature of the single-frame human behavior region image, the human behavior feature is obtained. The specific algorithm for obtaining the human behavior feature is as follows: ， in, Represents human behavioral characteristics.

4. The method for identifying dangerous human behaviors based on a generative adversarial network according to claim 1, characterized in that: The step of generating predicted human behavior features according to a generative adversarial network to obtain a final recognition result according to the predicted human behavior features specifically includes: Generate predicted human behavior features according to a generative adversarial network, wherein the generative adversarial network includes a feature generation submodule and a feature discrimination submodule; The feature generation submodule includes a Transformer encoding block, an information enhancement block and a Transformer decoding block; The feature generation submodule generates predicted human behavior features based on human behavior features; The feature identification submodule performs dangerous behavior identification based on the predicted human behavior features and human behavior features to obtain a final identification result.

5. The method for identifying dangerous human behaviors based on a generative adversarial network according to claim 4 is characterized in that: The information enhancement module specifically includes: The information enhancement module performs cosine similarity calculation based on the encoding features obtained after the Transformer encoding block is processed to obtain the feature similarity weight. The specific algorithm for obtaining the feature similarity weight is as follows: ， in, represents the feature similarity weight, represents the cosine similarity function, represents the encoding feature, represents the storage features in the information enhancement module, O represents the total number of stored features, The ordinal number representing the stored feature; Weighted reconstruction is performed according to the feature similarity weights to obtain the reconstructed features. The specific algorithm for obtaining the reconstructed features is as follows: ， in, Represents reconstruction features; Concatenate the encoded features and the reconstructed features to input into a Transformer decoding block; The storage features in the information enhancement module are updated according to the gating mechanism. The specific algorithm of the update is as follows: ， ， in, represents the output of the forget gate, represents the activation function, and represents the weight and bias of the forget gate, represents the normalization function, represents an element-wise multiplication operation, Indicates the matching index between the stored features and the encoded features.

6. The method for identifying dangerous human behaviors based on a generative adversarial network according to claim 1, characterized in that: The step of obtaining the final recognition result according to the predicted human behavior characteristics further includes: The loss of predicting human behavior characteristics is calculated and generated, and the algorithm for generating the loss of predicting human behavior characteristics is as follows: ， in, represents the loss of generating predicted human behavior features, and They represent the predicted human behavior features and the input Transformer encoding block human behavior features, respectively. is the number of interval blocks, Indicates the ordinal number of the interval block, t Indicates the moment; The loss of information enhancement is calculated, and the algorithm of the loss of information enhancement is as follows: ， ， ， in, represents the basic loss of information enhancement, represents the sparsity L1 regularization term, represents the regularization coefficient, represents the norm, represents the total loss of information enhancement, Represents the weight parameter of the sparsity L1 regularization term; The loss of the feature discrimination submodule is calculated. The specific algorithm of the loss of the feature discrimination submodule is as follows: ， ， ， in, Represents resistance to loss, Indicates that it is judged as a real operation. represents the weighted discriminant loss, , and They represent the total loss weights of generating the predicted human behavior features, the total loss of information enhancement, and the total loss of discrimination loss, respectively. Represents the final loss function of the feature discriminant submodule.

7. The method for identifying dangerous human behaviors based on a generative adversarial network according to claim 1, characterized in that: The step of calculating the risk score specifically includes: The specific algorithm for calculating the risk score is as follows: ， in, represents the risk score, represents the normalized risk score, and They represent the predicted human behavior features and the input Transformer encoding block human behavior features, respectively. is the number of interval blocks, Indicates the ordinal number of the interval block, t Indicates time.

8. A human dangerous behavior recognition system based on generative adversarial network, characterized in that: include: A preprocessing module is used to obtain real-time monitoring video images and perform preprocessing; A region segmentation module is used to perform human behavior region segmentation on the preprocessed video image to obtain a human behavior region image, wherein the human behavior region segmentation refers to segmenting a foreground region and a background region in the video image, wherein the foreground region includes all moving human targets, and the background region includes all fixed environmental targets; A feature extraction module, used for extracting human behavior features according to the human behavior region image, wherein the extracted human behavior features include directional gradient feature extraction and motion information extraction; The recognition module is used to generate predicted human behavior features according to a generative adversarial network to obtain a final recognition result according to the predicted human behavior features, wherein the generative adversarial network includes an information enhancement module and calculates a dangerousness score.

9. A storage medium, characterized in that: The storage medium stores one or more programs, which, when executed by the processor, implement the method for identifying dangerous human behaviors based on a generative adversarial network as described in any one of claims 1 to 7.

10. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein: The memory is used to store computer programs; When the processor is used to execute the computer program stored in the memory, it implements the method for identifying dangerous human behaviors based on a generative adversarial network as described in any one of claims 1 to 7.

Images