Safety psychological stress identification method and system based on neural network

Abstract

The invention relates to the technical field of image recognition, in particular to a safety psychological stress recognition method and system based on a neural network, and the method comprises the steps: collecting a first image and a second image; acquiring key point coordinates of the first image and the second image; obtaining a high-quality first image; determining expression expressions of the key points; obtaining an improved loss function of the discriminator by using the expression expressivity of the key points; acquiring an expanded image based on the improved loss function; taking the extended image and the high-quality first image as input, and training an expression recognition neural network by using a cross entropy loss function to obtain a trained expression recognition neural network; inputting the third image into a trained expression recognition neural network, and outputting a probability sequence of miner expressions; and determining the pressure level of the miner based on the probability sequence of the miner expression. According to the method, the emotion recognition network with better recognition capability can be obtained through training, and a more accurate pressure recognition result is obtained.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically to a method and system for recognizing psychological stress based on neural networks. Background Technology

[0002] The underground environment of coal mines is unique, with problems such as insufficient lighting, continuous noise, high temperature and humidity, and the accumulation of dust and harmful gases. Complex production processes, frequent shift work, and actual working hours often exceeding the scheduled working hours all contribute to the continuous work pressure on miners. Prolonged exposure to high pressure can easily lead to physical and mental discomfort and behavioral abnormalities, such as insomnia, anxiety, burnout, and increased unsafe behaviors. This not only affects the individual health of miners but also threatens the safe production and overall operational efficiency of the coal mine. Therefore, timely and accurate identification of miners' psychological stress is a crucial aspect of ensuring the safety and health of personnel in underground operations.

[0003] Existing methods typically acquire images of miners' faces and use neural networks to identify stress levels. Since psychological stress is often indirectly reflected through facial expressions, these methods generally first identify facial expressions and then infer the degree of stress based on those expressions, rather than directly identifying the stress state.

[0004] Due to poor lighting and low visibility in coal mines, the contrast between facial areas and background in the acquired images is weak, and miners' skin tones are often dark, resulting in a limited number of high-quality image samples for various facial expressions. Therefore, existing methods typically employ data augmentation techniques to enhance training data, i.e., expanding a small number of high-quality facial expression images. Currently, Generative Adversarial Networks (GANs) are commonly used for data augmentation. However, traditional GANs do not consider the different contributions of different facial regions to expression recognition during the data augmentation process, leading to the inability to obtain satisfactory augmented images and thus affecting the performance of subsequent expression recognition models. Summary of the Invention

[0005] This invention provides a method and system for identifying psychological stress based on neural networks to solve existing problems.

[0006] The safety psychological stress recognition method and system based on neural networks of the present invention adopts the following technical solution: One embodiment of the present invention provides a method for identifying psychological stress related to safety based on neural networks, the method comprising the following steps: Acquire a first image and a second image; wherein the first image represents the facial image of each miner underground in the coal mine, and the second image represents the neutral facial expression image of each miner before entering the coal mine. The first image and the second image are respectively input into the face key point detection model to obtain the key point coordinates of the first image and the second image. Based on the key point coordinates of the first image, obtain a high-quality first image; Based on the key point coordinates of high-quality first and second images, the expressiveness of each key point is determined. By utilizing the facial expression intensity of each key point, the improved loss function of the cGAN network discriminator is obtained; An augmented image is obtained based on a loss function improved from the cGAN network discriminator. Using the augmented image and the high-quality first image as input, the facial expression recognition neural network is trained using the cross-entropy loss function to obtain the trained facial expression recognition neural network. The third image is input into the trained facial expression recognition neural network, which outputs a probability sequence of various facial expressions of the miner; where the third image represents a real-time captured image of the miner's face. The stress level of miners is determined based on the probability sequence of various facial expressions.

[0007] Furthermore, the specific steps for obtaining a high-quality first image based on the key point coordinates of the first image are as follows: Based on the key point coordinates of each first image, locate the set of key points corresponding to each facial feature in each first image; Based on the set of key points corresponding to each facial feature, construct the minimum bounding rectangle of each facial feature, and determine the region corresponding to each facial feature; The same region corresponding to each facial feature is cropped from the first image, and each cropped region is determined as a local image block for each facial feature; Several local image blocks are extracted from all local image blocks of each facial feature, and the minimum variance among the Laplacian variances of each extracted local image block is determined as the variance of the local image block of each facial feature. The minimum variance among the variances of all local image patches of the facial features is determined as the variance threshold. The first image in which the Laplacian variance of all local image patches of the facial features is greater than the variance threshold is identified as a high-quality first image.

[0008] Furthermore, the specific steps for determining the expressiveness of each key point based on the key point coordinates of the high-quality first and second images are as follows: Based on the key point coordinates of the high-quality first and second images, a triangulation method is used to construct a first face key point map and a second face key point map; wherein the first face key point map represents the face key point map of the high-quality first image, and the second face key point map represents the face key point map of the second image. Based on the first and second facial key point maps, determine the density variation relationship and direction of change value of each key point; By utilizing the density variation relationship and direction value of each key point, the associated key points of each key point are obtained; Based on the key point coordinates of the high-quality first and second images, obtain the texture descriptor and neutral texture descriptor for each key point; By utilizing the texture descriptor, neutral texture descriptor, and associated keypoints for each keypoint, the expressiveness of each keypoint is obtained.

[0009] Furthermore, the specific steps for determining the density variation relationship and direction of change of each key point based on the first and second facial key point maps are as follows: For each key point in the first face key point map, obtain the number of nodes directly connected to the key point and the area enclosed by the nodes directly connected to the key point, and determine the density of each key point in the first face key point map by the ratio of the area to the number of nodes. For each key point in the second face key point map, obtain the number of nodes directly connected to the key point and the area enclosed by the nodes directly connected to the key point, and determine the density of each key point in the second face key point map by the ratio of the area to the number of nodes. For the same key point in the first face key point map and the second face key point map, the frequency of the key point having a higher density in the first face key point map than its density in the second face key point map, and the frequency of the key point having a lower density in the first face key point map than its density in the second face key point map are counted. The density relationship corresponding to the frequency of the two frequencies that is greater than a preset frequency threshold is determined as the density change relationship of each key point. For the same key point in the high-quality first and second images corresponding to each miner, obtain the direction of motion of the key point from its position in the second image to its position in the high-quality first image, and determine the angle of the direction of motion as the direction value of each key point; The frequency of each direction value of each key point is counted, and the direction value corresponding to the frequency greater than the preset frequency threshold is determined as the change direction value of each key point.

[0010] Furthermore, the specific steps for obtaining the associated key points of each key point by utilizing the density change relationship and change direction value of each key point are as follows: For any two key points among all key points, compare whether the density change relationship between the two key points is consistent; When the density change relationship between two key points is consistent, count the number of times the change direction values ​​of the two key points appear in the same high-quality first image, and calculate the sum of the number of times the change direction values ​​of the two key points appear in the high-quality first image respectively; The correlation between two keypoints is determined by the ratio of the number of times the change direction values ​​of two keypoints appear in the same high-quality first image to the sum of the number of times the change direction values ​​of the two keypoints appear in each high-quality first image. When the correlation between two key points is greater than a preset correlation threshold, the two key points are identified as related key points. Based on all the associated key points obtained, determine the associated key points for each key point.

[0011] Furthermore, the specific steps for obtaining the texture descriptor and neutral texture descriptor for each key point based on the key point coordinates of the high-quality first and second images are as follows: For a high-quality first image, a fixed-size image region is cropped with the coordinates of each key point as the center, and the cropped image region is determined as the region corresponding to each key point in the high-quality first image; Calculate the LBP histogram of the region corresponding to each key point in the high-quality first image, generate the corresponding LBP feature vector, and determine the texture descriptor of each key point using the LBP feature vector; For the second image, a fixed-size image region is cropped with the coordinates of each key point as the center, and the cropped image region is determined as the region corresponding to each key point in the second image; Calculate the LBP histogram of the region corresponding to each key point in the second image, generate the corresponding LBP feature vector, and determine the LBP feature vector as the neutral texture descriptor for each key point.

[0012] Furthermore, the specific steps for obtaining the facial expression degree of each key point using the texture descriptor, neutral texture descriptor, and associated key points are as follows: For the same key point in the high-quality first and second images corresponding to each miner, calculate the cosine similarity between the texture descriptor of the key point and its neutral texture descriptor, and determine the cosine similarity as the similarity of each key point; Calculate the mean of all similarities for each keypoint, and determine the self-expression of each keypoint by the difference between 1 and the mean. Obtain the average performance of the associated keypoints for each keypoint; The performance of each keypoint's associated keypoints is determined by multiplying the ratio of the number of associated keypoints to 68 and the average performance of each keypoint's associated keypoints. The average of the self-expression of each keypoint and the expression of each keypoint's associated keypoints is used to determine the expression of each keypoint to the current expression. The average value of the expressiveness of each keypoint for each expression is used to determine the expressiveness of each keypoint.

[0013] Furthermore, the specific steps for obtaining the improved loss function of the cGAN network discriminator using the facial expression performance of each key point are as follows: Obtain the key point coordinates of the face image input to the cGAN network discriminator; The nearest positive integer obtained by multiplying the facial expression score of each key point by 20 is used to determine the range of influence for each key point. The influence area of ​​each key point is defined as a circular region centered on the coordinates of each key point and with the radius of the influence range of each key point as the radius; the weight of each key point's influence area is the expression degree of each key point. The influence areas of all key points are superimposed to form a weight map of the same size as the face image input to the cGAN network discriminator. Based on the weight graph, obtain the improved loss function of the cGAN network discriminator; The improved loss function of the cGAN network discriminator is calculated as follows: the squared difference between the high-quality first image and the generated image is calculated pixel by pixel, then multiplied by the weight value of the corresponding position in the weight map, and finally the weighted result of all pixels is summed.

[0014] Furthermore, the specific steps for obtaining the augmented image based on the improved loss function of the cGAN network discriminator are as follows: Identify and label the facial expressions in each high-quality first image; Using a high-quality first image and its label as input, the cGAN network is trained using the improved loss function of the cGAN network discriminator to obtain the trained cGAN network. Based on the trained cGAN network, augmented images are obtained.

[0015] One embodiment of the present invention provides a safety psychological stress recognition system based on neural networks, the system comprising the following modules: The acquisition module is used to acquire a first image and a second image; wherein the first image represents the facial image of each miner in the coal mine, and the second image represents the neutral expression image of each miner before entering the coal mine. The analysis module is used to input the first image and the second image into the face keypoint detection model respectively to obtain the keypoint coordinates of the first image and the second image; based on the keypoint coordinates of the first image, a high-quality first image is obtained; based on the keypoint coordinates of the high-quality first image and the second image, the facial expression degree of each keypoint is determined; using the facial expression degree of each keypoint, the improved loss function of the cGAN network discriminator is obtained; based on the improved loss function of the cGAN network discriminator, the augmented image is obtained. The recognition module takes the augmented image and the high-quality first image as input, and trains the facial expression recognition neural network using the cross-entropy loss function to obtain a trained facial expression recognition neural network. The third image is then input into the trained facial expression recognition neural network, and the output is a probability sequence of various facial expressions of the miner. The third image represents a real-time captured image of the miner's face. Based on the probability sequence of various facial expressions of the miner, the stress level of the miner is determined.

[0016] The beneficial effects of the technical solution of the present invention are as follows: The embodiments of the present invention propose a method and system for safe psychological stress recognition based on neural networks. First, the collected facial data is screened to obtain high-quality data. Then, the high-quality data is analyzed to obtain the expression degree of each key point. Based on the expression degree of each key point, the loss function of the discriminator in the existing cGAN network is improved, and these high-discrimination key points are given greater weights in the loss function. This results in an expanded image containing more features that are helpful for expression recognition. That is, the quality of the expanded facial data is higher, thereby training an emotion recognition network with better recognition ability and obtaining more accurate stress recognition results. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the steps of the neural network-based method for identifying psychological stress in this invention. Figure 2 This is a schematic diagram of the cGAN network architecture of the present invention; Figure 3 This is a block diagram of the neural network-based safety psychological stress recognition system of the present invention. Detailed Implementation

[0019] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the neural network-based safety psychological stress recognition method and system proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0021] The specific solution of the neural network-based safety psychological stress recognition method and system provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0022] This invention collects facial data of coal miners underground using installed cameras, then selects high-quality data from different expressions. This data is then augmented using a cGAN network to obtain training data, which is used to train an expression recognition network. A mapping relationship between expression data and stress levels is then established. Actual miner expression data is input into the expression recognition neural network to obtain the probabilities of different expression categories. Based on the mapping relationship, the miner's stress level is then determined.

[0023] This invention improves the loss function of the discriminator, so that the images retained by the discriminator contain more features that are helpful for facial expression recognition, thereby improving the usability of the expanded data.

[0024] Please see Figure 1 The diagram illustrates a flowchart of a neural network-based method for identifying psychological stress in safety, provided by an embodiment of the present invention. The method includes the following steps: Step S001: Acquire the first image and the second image; wherein the first image represents the facial image of each miner in the coal mine, and the second image represents the neutral expression image of each miner before entering the coal mine.

[0025] It should be noted that: a neutral expression image indicates that the miner is in a calm and relaxed state at this moment, and the miner's face shows no obvious emotional expression characteristics.

[0026] Specifically, intrinsically safe or explosion-proof high-resolution cameras are installed in key areas of the underground coal mine working area, such as the front of the tunneling machinery operating platform, the interior and entrance of the loading and transport vehicle cab, and the area around the engineering vehicle operating cab. These cameras are responsible for capturing the facial expressions of miners from a third-person perspective during equipment operation, shift changes, and short breaks, especially capturing their natural expressions when focused on work, thus acquiring several first images. In a quiet environment on the surface, miners are guided into a relaxed state, and images of their natural facial expressions—facing the camera directly, with even lighting and no obvious emotional expression—are captured, thus acquiring several second images.

[0027] Step S002: Input the first image and the second image into the face key point detection model respectively to obtain the key point coordinates of the first image and the second image.

[0028] It should be noted that the facial landmark detection model is a deep learning tool based on computer vision. Its main function is to automatically locate and output a series of semantically meaningful facial feature point coordinates from an input facial image. The facial landmark detection model in this embodiment is the Dlib model. This embodiment obtains 68 landmark coordinates, of which numbers 1-17 correspond to the facial contour, 18-27 correspond to the eyebrows, 28-36 correspond to the nose, 37-42 correspond to the left eye, 43-48 correspond to the right eye, and 49-68 correspond to the mouth.

[0029] Specifically, two facial images (the first image and the second image) are input into the Dlib model. The model first detects and locates the facial region in the image, and then automatically identifies the two-dimensional coordinates of a total of 68 key points, including the eyes, corners of the eyes, brow ridges, tip of the nose, nostrils, corners of the mouth, cupid's bow, and facial contours, based on the pixels in that region. The Dlib model is a well-known technology and will not be described in detail here.

[0030] Step S003: Obtain a high-quality first image based on the key point coordinates of the first image.

[0031] It should be noted that, considering that there is relatively little high-quality data among the collected facial expression data, high-quality data is first selected from each type of facial expression data, and then the data is augmented using a cGAN (Conditional Generative Adversarial Network).

[0032] Step S003 further includes steps S0031-S0036: Step S0031: Based on the key point coordinates of each first image, locate the set of key points corresponding to each facial feature in each first image.

[0033] Specifically, based on the numbering rules of the facial key point model, key point sets corresponding to the left eye, right eye, eyebrows, nose and mouth are extracted from the coordinates of the 68 detected key points.

[0034] Step S0032: Based on the set of key points corresponding to each facial feature, construct the minimum bounding rectangle of each facial feature, and determine the region corresponding to each facial feature.

[0035] For example, for the set of keypoints for the left eye, its minimum bounding rectangle is constructed, and this minimum bounding rectangle is used as the corresponding region for the left eye: First, the minimum and maximum values ​​of the coordinates of all keypoints in the horizontal and vertical directions are obtained to determine the coordinates of the upper left and lower right corners of the rectangle. Optionally, the boundary is expanded to finally obtain the rectangular region corresponding to the left eye. The construction of the minimum bounding rectangle is a well-known technique and will not be elaborated on here.

[0036] Step S0033: Crop out the regions that correspond to each facial feature from the first image, and determine each cropped region as a local image block of each facial feature.

[0037] It should be noted that each local image block of the facial features includes five local image blocks: left eye, right eye, eyebrows, nose, and mouth. That is, each first image contains five local image blocks.

[0038] For example, a region identical to the region corresponding to the left eye is cropped from the first image, and the cropped region is used as a local image block of the left eye.

[0039] Step S0034: Extract several local image blocks from all local image blocks of each facial feature, and determine the variance of the local image block of each facial feature by taking the minimum variance among the Laplacian variances of each extracted local image block.

[0040] It should be noted that Laplacian variance is an evaluation metric used to quantify image sharpness. It reflects the sharpness of image edges and textures by calculating the variance of pixel values ​​after the image has been processed by the Laplacian operator. A higher value usually indicates a sharper image.

[0041] The Laplacian variance of each extracted local image patch is calculated as follows: First, each local image patch is converted to a grayscale image. Next, the grayscale image is convolved using the Laplacian operator (a 3×3 convolution kernel) to obtain a Laplacian response map, which highlights edges and high-frequency texture regions in the image. Then, the variance of all pixel values ​​in this response map is calculated; this variance is the Laplacian variance of the local image patch. The calculation of the Laplacian variance is a well-known technique and will not be elaborated upon here.

[0042] Step S0035: Determine the minimum variance among the variances of all local image patches of the facial features as the variance threshold.

[0043] Step S0036: The first image in which the Laplacian variance of all local image patches of the facial features is greater than the variance threshold is determined as a high-quality first image.

[0044] For example, there are 2000 first images, and each first image has 5 local image patches. Therefore, there are 2000 local image patches for each facial feature. 1000 local image patches are randomly selected from the 2000 local image patches of the left eye. The Laplacian variance of the selected 1000 local image patches is calculated, and the minimum variance is selected as the variance of the local image patches of the left eye.

[0045] The variances of the local image patches corresponding to the left eye, right eye, eyebrows, nose, and mouth can be obtained in the above manner, and the smallest variance among these five variances can be selected as the variance threshold.

[0046] Then, the five local image patches in each first image are judged. If the Laplacian variance of these five local image patches is greater than the variance threshold, the first image is judged to be a high-quality first image.

[0047] Step S004: Based on the key point coordinates of the high-quality first and second images, determine the facial expression level of each key point.

[0048] It's important to note that facial muscles work in tandem. We focus on groups of points that interact and pull on each other during facial expressions. Therefore, for each keypoint, we need to define a set of neighboring keypoints with direct muscle connections or motor associations.

[0049] Step S004 further includes steps S0041-S0045: Step S0041: Based on the key point coordinates of the high-quality first image and the second image, a first face key point map and a second face key point map are constructed using the triangulation method; wherein the first face key point map represents the face key point map of the high-quality first image, and the second face key point map represents the face key point map of the second image.

[0050] It should be noted that triangulation is a geometric partitioning method that connects planar point sets into non-overlapping triangular meshes, aiming to cover and describe the overall shape and topology of a target region using the simplest geometric units (triangles). This embodiment uses the Delaunay triangulation method.

[0051] A triangulation method is used to construct a first and a second facial keypoint map. Sixty-eight keypoints in a high-quality first or second image are considered as a planar point set. A triangulation algorithm automatically generates a set of triangles, each with these points as vertices, that completely cover the face region. The edges of each triangle connect spatially adjacent keypoints, thus forming a mesh diagram that characterizes the facial geometric topology, i.e., a triangular mesh structure. This triangular mesh structure is denoted as the facial keypoint map. Triangulation is a well-known technique and will not be elaborated upon here.

[0052] Step S0042: Based on the first and second facial key point maps, determine the density variation relationship and direction of change of each key point.

[0053] It should be noted that this embodiment obtains neighboring keypoints associated with the movement of each keypoint by determining whether the density of keypoints in the miner's face image changes in a block and whether the keypoints always change in a fixed direction. For example, a block of density increasing corresponds to frowning under high pressure; a block of density decreasing corresponds to relaxing the brow under low pressure.

[0054] The key points with the same number in different expressions are key points in the same position on the face.

[0055] Step S0042 further includes steps S0421-S0425: Step S0421: For each key point in the first face key point map, obtain the number of nodes directly connected to the key point and the area enclosed by the nodes directly connected to the key point, and determine the density of each key point in the first face key point map by the ratio of the area to the number of nodes.

[0056] It should be noted that nodes directly connected to keypoints are the other keypoints connected to keypoints. Let n be the number of nodes directly connected to keypoints, and h be the area enclosed by the nodes directly connected to keypoints. Then, the density of each keypoint in the first face keypoint map is denoted as m1: m1 = h / n.

[0057] Step S0422: For each key point in the second face key point map, obtain the number of nodes directly connected to the key point and the area enclosed by the nodes directly connected to the key point, and determine the density of each key point in the second face key point map by the ratio of the area to the number of nodes.

[0058] Specifically, similarly, the density of each keypoint in the second face keypoint map is denoted as m2.

[0059] Step S0423: For the same key point in the first facial key point map and the second facial key point map, count the frequency that the density of the key point in the first facial key point map is greater than its density in the second facial key point map, and the frequency that the density of the key point in the first facial key point map is less than its density in the second facial key point map. Determine the density change relationship corresponding to each key point based on the density relationship corresponding to the frequency greater than the preset frequency threshold among the two frequencies.

[0060] It should be noted that: The preset frequency threshold is mainly based on: the statistical confidence required by the sample size (number of miners), the judgment of the density change relationship of key points, and the strictness requirements for judging the movement direction of key points. For example, when the sample size is sufficient (such as more than 30 people) and individual accidental differences are expected to be excluded, the threshold can be set to 0.8 to ensure that only when most miners show the same density change trend (contraction or stretching) at the same key point, it is determined that the key point has a stable density change relationship. In this embodiment, it is preferably 0.8.

[0061] Specifically, for miner A, for the same key point in his first facial key point map and the second facial key point map: If the frequencies of the cases of m1>m2 and m1<m2 appear respectively, and if one of the frequencies is greater than 0.8, it is considered that the key point changes in this way under this underground expression, and the corresponding relationship between m1 and m2 is obtained, denoted as the density change relationship (here the density change relationship is relative to the neutral expression). When there is no frequency greater than 0.8, it means that there is no density change relationship for this key point.

[0062] Exemplarily, for miner A, for the same key point (number 12) in his first facial key point map and the second facial key point map: The frequency of m1>m2 appears greater than 0.8, then the density change relationship of the key point numbered 12 is: m1>m2.

[0063] Step S0424: For the same key point in the high-quality first image and the second image corresponding to each miner, obtain the movement direction of the key point changing from its position in the second image to its position in the high-quality first image, and determine the angle of the movement direction as the direction value of each key point.

[0064] Specifically, for each miner, extract the two-dimensional coordinates of the same numbered key point in his high-quality first image (underground expression image) and the second image (neutral expression image) respectively. Calculate the displacement vector of the key point from its position in the second image to its position in the first image, which represents the spatial movement direction of the key point during the expression change process. Quantify the angle of the displacement vector (usually with the horizontal right as 0°, and the counterclockwise rotation as the positive direction) as the direction value. Thus, the movement direction of each key point is obtained.

[0065] For ease of calculation, this embodiment uses the nearest multiple of 5 to the angle of the displacement vector as its direction value. For example, if the angle of the displacement vector is 43°, the nearest multiple of 5 is 45°, so the direction value is 45°. Therefore, this calculation yields 72 types of direction values.

[0066] Step S0425: Calculate the frequency of each direction value of each key point, and determine the direction value corresponding to the frequency greater than the preset frequency threshold as the change direction value of each key point.

[0067] It should be noted that for a certain key point, the calculated direction value may be the same or different in the high-quality first and second images corresponding to different miners. Therefore, the probability of occurrence of each direction value of a certain key point is statistically analyzed. The preset frequency threshold is the same as the frequency threshold in step S0423.

[0068] For each keypoint, obtain the direction value of that keypoint for all miners, and statistically determine the frequency of occurrence of each direction value. The direction value corresponding to the frequency greater than 0.8 is taken as the change direction value of that keypoint. That is, when a person's expression changes from neutral to non-neutral, the keypoint always moves along the direction of the change direction value.

[0069] Step S0043: Using the density change relationship and change direction value of each key point, obtain the associated key points of each key point.

[0070] Step S0043 further includes steps S0431-S0435: Step S0431: For any two key points among all key points, compare whether the density change relationship between the two key points is consistent.

[0071] Specifically, for any two key points out of the 68 key points, first determine whether the density change relationship between the two key points is consistent. If one of the key points does not have a density change relationship, then directly use the direction values ​​of the two key points to make the judgment, that is, directly proceed to step S0432.

[0072] Step S0432: When the density change relationship of the two key points is consistent, count the number of times the change direction values ​​of the two key points appear in the same high-quality first image, and calculate the sum of the number of times the change direction values ​​of the two key points appear in the high-quality first image respectively.

[0073] Step S0433: The ratio of the number of times the change direction values ​​of two key points appear in the same high-quality first image to the sum of the number of times the change direction values ​​of the two key points appear in the high-quality first images respectively is used to determine the correlation between the two key points.

[0074] Specifically, for any two key points, to determine whether they are motion-related points, first compare the density change relationship. If the magnitudes of m1 and m2 in the density change relationship are consistent, then compare the directions of change: The correlation between two keypoints is denoted by the ratio v of the number of times the change direction values ​​of two keypoints appear in the same high-quality first image to the sum of the number of times the change direction values ​​of the two keypoints appear in each high-quality first image.

[0075] Step S0434: When the correlation between two key points is greater than the preset correlation threshold, the two key points are determined to be related key points.

[0076] It should be noted that the larger v is, the more likely the two key points are to always move in this way. If v is greater than 0.8, then the two key points are recorded as related key points.

[0077] The preset correlation threshold is set based on the significance requirements of group data statistics and prior physiological knowledge of facial muscle synergistic movements. In this embodiment, a threshold of 0.8 is preferred.

[0078] Step S0435: Based on all the acquired associated key points, determine the associated key points of each key point.

[0079] It should be noted that by comparing the density change relationship and change direction values ​​of any two key points, all associated key points can be obtained, and thus the associated key points of each key point can be obtained.

[0080] The key points that are associated with each other can be mapped onto facial expressions. For example, when smiling, the key points of the mouth, eyes, and corners of the eyes are likely to form associated key points, corresponding to the harmonious and pleasant expression of "upturned corners of the mouth, slightly narrowed eyelids, and crow's feet at the corners of the eyes" when smiling. When surprised, the key points between the eyebrows and the mouth are likely to form associated key points, corresponding to the instantaneous shocked expression of "raised eyebrows, wide-open eyelids, and mouth involuntarily opening into a circle" when surprised.

[0081] The associated keypoints of each keypoint were obtained through calculation; these are groups of points that pull and interact with each other during facial expression changes. Then, by analyzing the changes in each keypoint itself and the changes in associated keypoints in each type of underground expression (the expression corresponding to the high-quality first image), the expressiveness of each keypoint for that expression needs to be obtained.

[0082] Step S0044: Based on the key point coordinates of the high-quality first and second images, obtain the texture descriptor and neutral texture descriptor for each key point.

[0083] It should be noted that when a person is under significant psychological stress, their facial expressions will exhibit a complex state of "tension, inhibition, and exhaustion," specifically manifested in the typical characteristics of three areas: Eyebrow and eye area (serious and absent-minded): Frowning eyebrows, dull or wandering eyes, and tense muscles around the eyes.

[0084] Lip area (inhibiting tension): Lips are pursed, corners of the mouth droop, jaw is stiff, accompanied by slight tremors.

[0085] Overall condition (stiff and tired): facial muscles are stiff and lack natural micro-movements; skin color may be abnormal, as if wearing a "mask".

[0086] Expressions under high pressure are not a single emotion, but a complex state characterized by furrowed brows, dull eyes, downturned lips, and facial stiffness, reflecting an individual's state of tension, exhaustion, and self-control. Based on this, this embodiment describes these stress characteristics using the texture of each person's facial key points, obtaining the texture of each of the 68 key points for each individual. Furthermore, it compares other expressions with neutral expressions to determine which key points' textures and texture relationships have changed. Key points exhibiting significant changes are then used as key points representing facial expression features. For example, when angry, one often furrows their brow; therefore, key points in the eyebrow area are used as key points representing facial expression features. When the discriminator judges the generated image, it pays greater attention to these key points.

[0087] Comparing other expressions with a neutral expression is equivalent to comparing them with a standard expressionless state to determine how the texture of the facial features has changed in other expressions, thereby obtaining the facial expression characteristics.

[0088] Step S0044 further includes steps S0441-S0444: Step S0441: For the high-quality first image, take the coordinates of each key point as the center, crop a fixed-size image region, and determine the cropped image region as the region corresponding to each key point in the high-quality first image.

[0089] It should be noted that the fixed size setting is based on factors including the normalized size of the face detection bounding box, the local range of the physiological structure where each key point is located, and the requirements of subsequent texture feature operators (such as LBP) for the effective neighborhood. In this embodiment, a 7×7 pixel area is set, which achieves a good balance between feature discrimination and computational efficiency.

[0090] For example, in a high-quality first image, for a key point numbered 12, a 7×7 pixel image region is cropped centered on the coordinates of the key point as the region corresponding to the key point.

[0091] Step S0442: Calculate the LBP histogram of the region corresponding to each key point in the high-quality first image, generate the corresponding LBP feature vector, and determine the LBP feature vector as the texture descriptor for each key point.

[0092] It should be noted that LBP histogram (Local Binary Pattern Histogram) is a statistical representation method used to describe the local texture features of an image. It forms a fixed-dimensional histogram vector (LBP feature vector) by statistically analyzing the distribution of LBP encoded values ​​generated by comparing each pixel with its neighboring pixels, thus effectively representing the texture patterns and structural features of an image region. LBP histograms are a well-known technique and will not be elaborated upon further here.

[0093] To obtain the texture descriptor for each keypoint: After converting the region corresponding to each keypoint into a grayscale image, a unified LBP (Local Binary Pattern) operator is used to calculate its texture encoding map. Then, the LBP value distribution of all pixels in the encoding map is statistically analyzed, forming a 256-dimensional histogram. This histogram is normalized to obtain a standard LBP feature vector representing the local texture structure of the keypoint; this vector is the texture descriptor for that keypoint.

[0094] Step S0443: For the second image, take the coordinates of each key point as the center, crop a fixed-size image region, and determine the cropped image region as the region corresponding to each key point in the second image.

[0095] It should be noted that the fixed size in this step is the same as the fixed size in step S0441.

[0096] Step S0444: Calculate the LBP histogram of the region corresponding to each key point in the second image, generate the corresponding LBP feature vector, and determine the LBP feature vector as the neutral texture descriptor for each key point.

[0097] Step S0045: Use the texture descriptor, neutral texture descriptor and associated key points of each key point to obtain the expression degree of each key point.

[0098] Step S0045 further includes steps S0451-S0456: Step S0451: For the same key point in the high-quality first and second images corresponding to each miner, calculate the cosine similarity between the texture descriptor of the key point and its neutral texture descriptor, and determine the cosine similarity as the similarity of each key point.

[0099] It should be noted that cosine similarity is an index that measures the consistency of the directions of two vectors. It assesses their similarity by calculating the cosine of the angle between them. Cosine similarity is a well-known technique and will not be elaborated upon here.

[0100] For example, for miner A, the corresponding high-quality first image represents the miner's facial expression underground, and the corresponding second image represents the miner in a calm and relaxed state. Then, for keypoint number 12, the similarity between the texture features of this keypoint in the two images is calculated. The smaller the similarity s, the greater the variation of the keypoint, and the greater the role that keypoint can play in recognizing the expression in the high-quality first image.

[0101] Calculate the cosine similarity between the texture descriptor of a key point and its neutral texture descriptor: Calculate the cosine similarity between two LBP feature vectors.

[0102] Step S0452: Calculate the mean of all similarities for each keypoint, and determine the difference between 1 and the mean as the self-expression of each keypoint.

[0103] For example, for miner A, the similarity of key point number 12 is calculated as s1; for miner B, the similarity of key point number 12 is calculated as s2; then the self-expression degree c of key point number 12 is: c = 1 - (s1 + s2) / 2.

[0104] Step S0453: Obtain the mean of the self-performance of the associated keypoints of each keypoint.

[0105] For example, the self-expressiveness of keypoint number 12 is c1, and its associated keypoints are number 13 and number 11. The self-expressiveness of keypoint number 13 is c2, and the self-expressiveness of keypoint number 11 is c3. Then the mean w of the self-expressiveness of the associated keypoints of keypoint number 12 is: w = (c2 + c3) / 2.

[0106] Step S0454: The product of the ratio of the number of associated keypoints of each keypoint to 68 and the mean of the performance of the associated keypoints of each keypoint is used to determine the performance of the associated keypoints of each keypoint.

[0107] For example, if the number of associated keypoints of keypoint number 12 is 2, then the performance degree q of the associated keypoints of keypoint number 12 is: q = (2 / 68) × w.

[0108] Step S0455: Determine the performance level of each key point in relation to the current expression by taking the average of the performance level of each key point's own key point and the performance level of each key point's associated key points.

[0109] For example, the representation degree d of key point number 12 to the current expression (the expression corresponding to the high-quality first image) is: d = (c1 + q) / 2.

[0110] Step S0456: Determine the expression level of each key point by the mean of the expression level of each expression.

[0111] It should be noted that the expressiveness of each key point indicates the importance of that key point in recognizing facial expressions.

[0112] For example, if the expressiveness of key point number 12 for the expression of happiness is d1, the expressiveness of key point number 12 for the expression of anger is d2, and the expressiveness of key point number 12 for the expression of frustration is d3, then the expressiveness k of key point number 12 is: k = (d1 + d2 + d3) / 3.

[0113] Step S005: Utilize the facial expression performance of each key point to obtain the improved loss function of the cGAN network discriminator.

[0114] It should be noted that when the discriminator makes a judgment, it first obtains 68 key points of the generated image through key point recognition, and then calculates the loss weight of each key point and each surrounding key point on a high-quality dataset, which is used as the loss weight of the improved discriminator.

[0115] After obtaining the facial expression performance at each key point, the loss function of the existing discriminator for recognizing images forged by the generator is improved, specifically as follows: Step S005 further includes steps S0051-S0055: Step S0051: Obtain the key point coordinates of the face image input to the cGAN network discriminator.

[0116] It should be noted that this embodiment uses a conditional GAN ​​network, i.e., cGAN, to generate images with different expressions.

[0117] Figure 2 This is a schematic diagram of the cGAN network architecture of the present invention. Figure 2 As shown, in cGAN, the inputs to both the generator and the discriminator are supplemented with additional conditional information, namely the vector representation of the category label (e.g., happy, sad, surprised, etc.).

[0118] The generator's input is random noise z plus a conditional label u; the output is a specific facial expression image corresponding to the label u.

[0119] The generator hopes that the image it produces based on a certain condition label will be mistakenly identified as real by the discriminator (with the same label) when it is input to the discriminator for evaluation, thus giving it a high score.

[0120] Therefore, the generator's loss is defined as the negative logarithm of the scores given by the discriminator to these forged images. Through training, the generator strives to minimize this loss value, meaning that the images it generates will consistently receive higher scores from the discriminator.

[0121] The discriminator's input is an image x plus a condition label u; its output is a judgment on whether the image is both realistic and matches the expression described by the label.

[0122] The goal of the discriminator is to maximize its discriminative power, and its loss consists of the sum of two parts: Part 1 (Identifying Real Images): When the input is a real image with the correct conditional label, the discriminator should award a high score, indicating that "this is undoubtedly a real image that matches the label." This part of the loss is calculated as the logarithm of the discriminator's scores for these "real and correct" samples. The discriminator wants this value to be as high as possible.

[0123] Part Two (Detecting Forged Images): When the input is an image forged by the generator based on a certain condition label, the discriminator should give a very low score, indicating "this is a fake image." This part of the loss is calculated as the logarithm of the complement of the discriminator's judgment score for these "forged" samples. Again, the discriminator wants this value to be as large as possible. The complement is the difference between 1 and the score.

[0124] The overall goal of the discriminator is to improve its accuracy in both aspects through training, that is, to strive to increase the sum of the loss values ​​of the two parts mentioned above.

[0125] Specifically, facial landmark recognition is first performed on the face image input to the discriminator to obtain 68 landmarks.

[0126] Step S0052: Multiply the facial expression score of each key point by 20 and find the nearest positive integer to determine the influence range of each key point.

[0127] Step S0053: Determine the influence area of ​​each key point as the circular area centered on the coordinates of each key point and with the influence range of each key point as the radius; the weight in the influence area of ​​each key point is the expression degree of each key point.

[0128] Step S0054: Overlay the influence regions of all key points to form a weight map of the same size as the face image input to the cGAN network discriminator.

[0129] It should be noted that the higher the weight of a pixel, the more important that location is for facial expression recognition.

[0130] Step S0055: Based on the weight graph, obtain the improved loss function of the cGAN network discriminator.

[0131] The improved loss function of the cGAN network discriminator is calculated as follows: the squared difference between the high-quality first image and the generated image is calculated pixel by pixel, then multiplied by the weight value of the corresponding position in the weight map, and finally the weighted result of all pixels is summed.

[0132] Specifically, the loss function for the discriminator to identify images forged by the generator is changed from the sum of squares of (real image - generated image) to: the sum of squares of weight map × (real image features - generated image features). The improved loss function expression is: ; in, The value of the loss function after the improvement of the cGAN network discriminator is represented; H and W are the image height and width, respectively. Represents the pixel value at position (i,j) in the real image (the first high-quality image). This represents the pixel value at position (i,j) in the generated image; This represents the weight value at position (i,j) in the weighted graph.

[0133] Step S006: Obtain the augmented image based on the improved loss function of the cGAN network discriminator.

[0134] Step S006 further includes steps S0061-S0063: Step S0061: Identify and label the facial expressions in each high-quality first image.

[0135] It should be noted that the facial expressions in this embodiment are divided into 7 categories: angry, disgusted, fearful, happy, frustrated, surprised, and neutral.

[0136] Specifically, in this embodiment, the labels are applied manually.

[0137] Step S0062: Take the high-quality first image and its label as input, and train the cGAN network using the improved loss function of the cGAN network discriminator to obtain the trained cGAN network.

[0138] Specifically, a high-quality first image and its corresponding expression category label are used as training samples and input into a Conditional Generative Adversarial Network (cGAN). The generator receives random noise and conditional labels and attempts to synthesize a face image with the corresponding expression. The discriminator is optimized based on an improved weighted loss function: this loss function, by introducing a weight map generated from key point representation, assigns higher weights to the pixel differences between the real image and the generated image in key expression regions (such as between the eyebrows and corners of the mouth). During adversarial training, the discriminator minimizes the weighted loss to more accurately distinguish the authenticity of real and synthesized images in key expression features. At the same time, the generator continuously optimizes based on the discriminator's feedback, gradually generating synthesized images with more distinct features in key expression regions and higher quality, ultimately resulting in a fully trained cGAN network.

[0139] Step S0063: Obtain the augmented image based on the trained cGAN network.

[0140] Specifically, when generating augmented images using a trained cGAN network, the generator is first input with a conditional label specifying the expression category (e.g., "happy") and a random noise vector conforming to a preset distribution. Based on the facial feature distribution of underground miners and the generation patterns of key expression regions learned during training, the generator synthesizes facial images of the corresponding category with clear expression features. Through batch generation and filtering, a large number of high-quality, feature-rich augmented images in the key expression regions are ultimately obtained.

[0141] Step S007: Using the augmented image and the high-quality first image as input, train the facial expression recognition neural network using the cross-entropy loss function to obtain the trained facial expression recognition neural network.

[0142] Specifically, each generated image is judged by the loss function of the improved discriminator, and the resulting generated images contain more features that can be used to recognize facial expressions.

[0143] The first image, consisting of all images (enlarged images) generated by the cGAN network and a high-quality image, is used as input to the facial expression recognition neural network to obtain a trained facial expression recognition neural network. The training process is as follows: Input and Processing: The input consists of all images generated by the cGAN network and the original high-quality images. The ResNet50 model is used as the architecture for the facial expression recognition neural network.

[0144] Output and Comparison: The model outputs a 7-dimensional probability, representing the likelihood of anger, disgust, fear, happiness, frustration, surprise, and neutrality. The system compares this result with the actual expression labels and calculates the difference (cross-entropy loss).

[0145] Learning and Optimization: Based on the calculated differences, the Adam optimizer automatically adjusts millions of parameters within the model. This process is repeated hundreds of times on hundreds of thousands of images until the model can accurately identify emotions.

[0146] Step S008: Input the third image into the trained facial expression recognition neural network, and output the probability sequence of various facial expressions of the miner; wherein the third image represents the face image of the miner collected in real time.

[0147] It should be noted that the third image represents a real-time captured image of a miner's facial expression in a coal mine, i.e., the image to be identified.

[0148] Step S009: Determine the stress level of the miners based on the probability sequence of their various facial expressions.

[0149] Specifically, for real-time acquired facial images of miners, a probability sequence of various emotions of the miner is obtained based on an expression recognition neural network, and the stress level is determined according to a pre-set judgment rule base by security experts: If the emotion of "fear" or "anger" exceeds 40%, it should be immediately marked as a high-risk event.

[0150] If the combined feelings of "frustration" and "fear" account for more than half (50%) of the total, and this state lasts for more than 5 minutes, it is considered to be sustained high stress.

[0151] Every half minute, based on the above information, a stress level result of "low, medium, or high" is given.

[0152] Execute the corresponding alert: The system automatically triggers different actions based on the pressure level, and relevant personnel must cooperate: If the pressure is low, the system will only record the data and will not take any action.

[0153] If the pressure is medium, the system sends a command to vibrate the miner's wristband, prompting "Pay attention to adjustment," and sends a message to the shift leader: Team member XXX is fatigued, please pay attention. The shift leader must confirm the team member's condition within 10 minutes and click "Processed" on the device.

[0154] In the event of high pressure, the system sends a command to the miner's wristband, triggering a strong vibration and a voice announcement: "High risk, please stop work immediately and evacuate!" The dispatch center's large screen flashes red lights, sounds an alarm, displays the miner's location, and automatically notifies the safety officer. The dispatcher immediately urges evacuation via broadcast and dispatches a safety officer to the scene. The safety officer arrives to assist until the situation is resolved. The outcome is subsequently recorded in the system.

[0155] In keypoint-based facial expression recognition models, the information contribution of different facial regions varies significantly. This embodiment evaluates this difference through quantitative analysis: if the texture pattern of a keypoint region (such as between the eyebrows or at the corners of the mouth) shows greater differences under low, medium, and high pressure states, it indicates that the keypoint has a stronger ability to discriminate pressure states and carries higher information entropy. Therefore, when training the discriminator of the cGAN network, these high-discrimination keypoints should be assigned larger weights in the loss function, thereby guiding the model to pay more attention to these information-rich regions.

[0156] When using images generated by cGAN as training data, the facial expression recognition network is forced to learn how to extract effective features from images that may be blurry but whose facial features are discernible. This training enables the model to achieve good stress expression recognition even when faced with realistic, poorly lit faces of underground miners, as long as the core facial features are identified, thanks to its learned ability to "focus on key discriminative regions." This significantly improves the system's robustness and practicality in the underground coal mine environment.

[0157] Please see Figure 3 The diagram illustrates a block diagram of a neural network-based safety psychological stress recognition system according to an embodiment of the present invention. The system includes the following modules: The acquisition module 100 is used to acquire a first image and a second image; wherein the first image represents the facial image of each miner in the coal mine, and the second image represents the neutral expression image of each miner before entering the coal mine.

[0158] The analysis module 200 is used to input the first image and the second image into the face key point detection model respectively to obtain the key point coordinates of the first image and the second image; based on the key point coordinates of the first image, obtain a high-quality first image; based on the key point coordinates of the high-quality first image and the second image, determine the expression degree of each key point; use the expression degree of each key point to obtain the improved loss function of the cGAN network discriminator; and based on the improved loss function of the cGAN network discriminator, obtain the augmented image.

[0159] The recognition module 300 is used to take the augmented image and the high-quality first image as input, and train the facial expression recognition neural network using the cross-entropy loss function to obtain the trained facial expression recognition neural network; input the third image into the trained facial expression recognition neural network, and output the probability sequence of various facial expressions of the miner; wherein the third image represents the real-time acquired facial image of the miner; based on the probability sequence of various facial expressions of the miner, the stress level of the miner is determined.

[0160] In summary, in this embodiment of the invention, the collected facial data is first filtered to obtain high-quality data. Then, the high-quality data is analyzed to obtain the expression performance of each key point. Based on the expression performance of each key point, the loss function of the discriminator in the existing cGAN network is improved, and these high-discrimination key points are given greater weights in the loss function. This results in an expanded image containing more features that are helpful for expression recognition. In other words, the quality of the expanded facial data is higher, thereby training an emotion recognition network with better recognition capabilities and obtaining more accurate stress recognition results.

[0161] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying psychological stress related to safety based on neural networks, characterized in that, The method includes the following steps: Acquire a first image and a second image; wherein the first image represents the facial image of each miner underground in the coal mine, and the second image represents the neutral facial expression image of each miner before entering the coal mine. The first image and the second image are respectively input into the face key point detection model to obtain the key point coordinates of the first image and the second image. Based on the key point coordinates of the first image, obtain a high-quality first image; Based on the key point coordinates of high-quality first and second images, the expressiveness of each key point is determined. By utilizing the facial expression intensity of each key point, the improved loss function of the cGAN network discriminator is obtained; An augmented image is obtained based on a loss function improved from the cGAN network discriminator. Using the augmented image and the high-quality first image as input, the facial expression recognition neural network is trained using the cross-entropy loss function to obtain the trained facial expression recognition neural network. The third image is input into the trained facial expression recognition neural network, which outputs a probability sequence of various facial expressions of the miner; where the third image represents a real-time captured image of the miner's face. The stress level of miners is determined based on the probability sequence of various facial expressions.

2. The method for identifying psychological stress based on neural networks according to claim 1, characterized in that, The specific steps for obtaining a high-quality first image based on the key point coordinates of the first image are as follows: Based on the key point coordinates of each first image, locate the set of key points corresponding to each facial feature in each first image; Based on the set of key points corresponding to each facial feature, construct the minimum bounding rectangle of each facial feature, and determine the region corresponding to each facial feature; The same region corresponding to each facial feature is cropped from the first image, and each cropped region is determined as a local image block for each facial feature; Several local image blocks are extracted from all local image blocks of each facial feature, and the minimum variance among the Laplacian variances of each extracted local image block is determined as the variance of the local image block of each facial feature. The minimum variance among the variances of all local image patches of the facial features is determined as the variance threshold. The first image in which the Laplacian variance of all local image patches of the facial features is greater than the variance threshold is identified as a high-quality first image.

3. The method for identifying psychological stress based on neural networks according to claim 1, characterized in that, The specific steps for determining the expressiveness of each key point based on the key point coordinates of the high-quality first and second images are as follows: Based on the key point coordinates of the high-quality first and second images, a triangulation method is used to construct a first face key point map and a second face key point map; wherein the first face key point map represents the face key point map of the high-quality first image, and the second face key point map represents the face key point map of the second image. Based on the first and second facial key point maps, determine the density variation relationship and direction of change value of each key point; By utilizing the density variation relationship and direction value of each key point, the associated key points of each key point are obtained; Based on the key point coordinates of the high-quality first and second images, obtain the texture descriptor and neutral texture descriptor for each key point; By utilizing the texture descriptor, neutral texture descriptor, and associated keypoints for each keypoint, the expressiveness of each keypoint is obtained.

4. The method for identifying psychological stress based on neural networks according to claim 3, characterized in that, The specific steps for determining the density variation relationship and direction of change of each key point based on the first and second facial key point maps are as follows: For each key point in the first face key point map, obtain the number of nodes directly connected to the key point and the area enclosed by the nodes directly connected to the key point, and determine the density of each key point in the first face key point map by the ratio of the area to the number of nodes. For each key point in the second face key point map, obtain the number of nodes directly connected to the key point and the area enclosed by the nodes directly connected to the key point, and determine the density of each key point in the second face key point map by the ratio of the area to the number of nodes. For the same key point in the first face key point map and the second face key point map, the frequency of the key point having a higher density in the first face key point map than its density in the second face key point map, and the frequency of the key point having a lower density in the first face key point map than its density in the second face key point map are counted. The density relationship corresponding to the frequency of the two frequencies that is greater than a preset frequency threshold is determined as the density change relationship of each key point. For the same key point in the high-quality first and second images corresponding to each miner, obtain the direction of motion of the key point from its position in the second image to its position in the high-quality first image, and determine the angle of the direction of motion as the direction value of each key point; The frequency of each direction value of each key point is counted, and the direction value corresponding to the frequency greater than the preset frequency threshold is determined as the change direction value of each key point.

5. The method for identifying psychological stress based on neural networks according to claim 3, characterized in that, The specific steps involved in obtaining the associated key points for each key point by utilizing the density change relationship and change direction value of each key point are as follows: For any two key points among all key points, compare whether the density change relationship between the two key points is consistent; When the density change relationship between two key points is consistent, count the number of times the change direction values ​​of the two key points appear in the same high-quality first image, and calculate the sum of the number of times the change direction values ​​of the two key points appear in the high-quality first image respectively; The correlation between two keypoints is determined by the ratio of the number of times the change direction values ​​of two keypoints appear in the same high-quality first image to the sum of the number of times the change direction values ​​of the two keypoints appear in each high-quality first image. When the correlation between two key points is greater than a preset correlation threshold, the two key points are identified as related key points. Based on all the associated key points obtained, determine the associated key points for each key point.

6. The method for identifying psychological stress based on neural networks according to claim 3, characterized in that, The specific steps for obtaining the texture descriptor and neutral texture descriptor for each key point based on the key point coordinates of the high-quality first and second images are as follows: For a high-quality first image, a fixed-size image region is cropped with the coordinates of each key point as the center, and the cropped image region is determined as the region corresponding to each key point in the high-quality first image; Calculate the LBP histogram of the region corresponding to each key point in the high-quality first image, generate the corresponding LBP feature vector, and determine the texture descriptor of each key point using the LBP feature vector; For the second image, a fixed-size image region is cropped with the coordinates of each key point as the center, and the cropped image region is determined as the region corresponding to each key point in the second image; Calculate the LBP histogram of the region corresponding to each key point in the second image, generate the corresponding LBP feature vector, and determine the LBP feature vector as the neutral texture descriptor for each key point.

7. The method for identifying psychological stress based on neural networks according to claim 3, characterized in that, The specific steps for obtaining the facial expression degree of each key point by utilizing the texture descriptor, neutral texture descriptor, and associated key points are as follows: For the same key point in the high-quality first and second images corresponding to each miner, calculate the cosine similarity between the texture descriptor of the key point and its neutral texture descriptor, and determine the cosine similarity as the similarity of each key point; Calculate the mean of all similarities for each keypoint, and determine the self-expression of each keypoint by the difference between 1 and the mean. Obtain the average performance of the associated keypoints for each keypoint; The performance of each keypoint's associated keypoints is determined by multiplying the ratio of the number of associated keypoints to 68 and the average performance of each keypoint's associated keypoints. The average of the self-expression of each keypoint and the expression of each keypoint's associated keypoints is used to determine the expression of each keypoint to the current expression. The average value of the expressiveness of each keypoint for each expression is used to determine the expressiveness of each keypoint.

8. The method for identifying psychological stress based on neural networks according to claim 1, characterized in that, The specific steps involved in obtaining the improved loss function of the cGAN network discriminator by utilizing the facial expression degree of each key point are as follows: Obtain the key point coordinates of the face image input to the cGAN network discriminator; The nearest positive integer obtained by multiplying the facial expression score of each key point by 20 is used to determine the range of influence for each key point. The influence area of ​​each key point is defined as a circular region centered on the coordinates of each key point and with the radius of the influence range of each key point as the radius; the weight of each key point's influence area is the expression degree of each key point. The influence areas of all key points are superimposed to form a weight map of the same size as the face image input to the cGAN network discriminator. Based on the weight graph, obtain the improved loss function of the cGAN network discriminator; The improved loss function of the cGAN network discriminator is calculated as follows: the squared difference between the high-quality first image and the generated image is calculated pixel by pixel, then multiplied by the weight value of the corresponding position in the weight map, and finally the weighted result of all pixels is summed.

9. The method for identifying psychological stress based on neural networks according to claim 1, characterized in that, The specific steps for obtaining the augmented image using the improved loss function based on the cGAN network discriminator are as follows: Identify and label the facial expressions in each high-quality first image; Using a high-quality first image and its label as input, the cGAN network is trained using the improved loss function of the cGAN network discriminator to obtain the trained cGAN network. Based on the trained cGAN network, augmented images are obtained.

10. A safety psychological stress recognition system based on neural networks, characterized in that, The system includes the following modules: The acquisition module is used to acquire a first image and a second image; wherein the first image represents the facial image of each miner in the coal mine, and the second image represents the neutral expression image of each miner before entering the coal mine. The analysis module is used to input the first image and the second image into the face key point detection model respectively, obtain the key point coordinates of the first image and the second image; and obtain a high-quality first image based on the key point coordinates of the first image. Based on the key point coordinates of high-quality first and second images, the expressiveness of each key point is determined. By utilizing the facial expression intensity of each key point, the improved loss function of the cGAN network discriminator is obtained; An augmented image is obtained based on a loss function improved from the cGAN network discriminator. The recognition module is used to train the facial expression recognition neural network by taking the augmented image and the high-quality first image as input and using the cross-entropy loss function to obtain the trained facial expression recognition neural network. The third image is input into the trained facial expression recognition neural network, which outputs a probability sequence of various facial expressions of the miner. The third image represents a real-time captured image of a miner's face; the miner's stress level is determined based on the probability sequence of various facial expressions.

Images