Facial micro-expression image enhancement transmission method for remote psychological counseling

Abstract

The present invention relates to the field of image processing technology, and in particular to a facial micro-expression image enhancement transmission method for remote psychological consultation, the method comprising: obtaining continuous video frame images; determining the micro-expression area of ​​the facial image in each video frame image according to the changes of each part area in the continuous video frame images; performing fuzzy processing on the non-micro-expression area in each video frame image to obtain the processed continuous video frame images; the non-micro-expression area is the area in the video frame image except the micro-expression area; and performing image transmission for remote psychological consultation according to the processed continuous video frame images. The embodiment of the present invention makes the micro-expression more prominent on the whole image, and can more comprehensively collect and analyze the micro-expression information of the consultation object, so as to make more wise and accurate decisions and judgments.

Description

Technical Field

[0001] The embodiment of the present invention relates to the field of image processing technology, and in particular to a facial micro-expression image enhancement transmission method for remote psychological consultation. Background Art

[0002] At present, the importance of micro-expressions in the process of psychological counseling cannot be ignored. They provide important clues for counselors to gain a deeper understanding of the inner world of their clients. Micro-expressions are changes in facial expressions that are difficult for people to control in a very short period of time. They are often a true reflection of emotions and are not controlled by consciousness. In psychological counseling, micro-expressions can reveal the true emotions that clients may try to hide or are unwilling to express directly. These emotional states are of great significance for counselors to formulate counseling strategies, evaluate counseling progress, and predict counseling results.

[0003] Compared with face-to-face counseling, remote psychological counseling has limitations in communication, such as limited transmission of non-verbal signals. As an important non-verbal signal, accurate recognition of micro-expressions is crucial to understanding the true emotions and intentions of the counselor. However, since micro-expressions are tiny expressions produced by facial muscles in a very short time, usually lasting no more than 1 / 25 second, and difficult to be consciously controlled, their subtlety makes them easy to be missed or misjudged when observed directly. Summary of the invention

[0004] In order to solve the technical problem, the purpose of the present invention is to provide a facial micro-expression image enhancement transmission method for remote psychological consultation, and the technical solution adopted is as follows:

[0005] Acquire continuous video frame images;

[0006] Determine the micro-expression area of ​​the facial image in each video frame image according to the changes of each part area in the continuous video frame images;

[0007] Performing fuzzy processing on the non-micro-expression area in each video frame image to obtain processed continuous video frame images; the non-micro-expression area is the area in the video frame image other than the micro-expression area;

[0008] Based on the processed continuous video frame images, image transmission for remote psychological consultation is performed.

[0009] In an optional manner, determining the micro-expression area of ​​the facial image in each video frame image according to the changes between the various part areas in each video frame image includes:

[0010] Determine the facial image in each video frame image;

[0011] Using a part region recognition model to determine each part region in the facial image; the part region recognition model is obtained by pre-training a neural network model based on part recognition samples;

[0012] Analyze the motion expression of each area in the continuous video frame image;

[0013] According to the motion expression degree of each part area, determine the target part area belonging to the micro-expression area in each video frame image; the target part area is any part area belonging to the micro-expression area in each part area in the facial image;

[0014] According to each target part area in the video frame image, the micro-expression area of ​​the facial image in each video frame image is obtained.

[0015] In an optional manner, the analysis of the degree of motion expression in each part area in continuous video frame images includes: determining the degree of motion expression in each part area in each video frame image based on the difference in grayscale values ​​of each position in the part area on the continuous video frame images, and the difference in grayscale values ​​of the part area on the continuous video frame images on the current edge detection image; the edge detection image is an image obtained after edge detection is performed on the video frame image.

[0016] In an optional manner, determining the target part area belonging to the micro-expression area in each video frame image according to the motion expression degree in each part area includes:

[0017] When the motion expression degree in the part area is greater than a preset motion expression degree threshold, determining the part area as a target part area belonging to a micro-expression area;

[0018] When the motion expression of the surrounding part area is within a preset motion expression range, the surrounding part area is determined to be the target part area belonging to the micro-expression area; the surrounding part area is each of the part areas around the part area whose motion expression is greater than a preset motion expression threshold.

[0019] In an optional manner, determining the facial image in each video frame image includes:

[0020] Acquire the face recognition algorithm to perform face image recognition on each video frame image, and identify the initial face image in each video frame image;

[0021] Using a facial key point detection algorithm to perform key point detection on the initial facial image to obtain key point positions;

[0022] Determine the face rotation angle based on the key point position;

[0023] According to the face rotation angle, the initial face image is rotationally corrected to obtain a corrected video frame image;

[0024] The corrected video frame image is cropped according to the corrected video frame image and the key point positions to obtain the facial image corresponding to the face area.

[0025] In an optional manner, the blurring of the non-micro-expression regions in each video frame image to obtain processed continuous video frame images includes:

[0026] For the non-micro-expression area, fuzzy processing is performed on the first neighborhood area according to the similarity of each pixel point in the first neighborhood area;

[0027] According to the similarity between each pixel point in the second neighborhood area and the first neighborhood area, fuzzy processing is performed on the second neighborhood area; the second neighborhood area is an area composed of pixel points with a preset number of pixel points away from the first neighborhood area;

[0028] The second neighborhood area is used as the first neighborhood area, and the step of blurring the second neighborhood area according to the similarity between each pixel point in the second neighborhood area and each pixel point in the first neighborhood area is continued, until all the non-micro-expression areas are traversed to obtain the processed continuous video frame images.

[0029] In an optional manner, the first neighborhood area is a neighborhood area consisting of k pixels sampled from top to bottom and from left to right starting from the first pixel point in the upper left corner of the non-micro-expression area.

[0030] In an optional manner, the fuzzy processing is performed on the first neighborhood area according to the similarity of each pixel point in the first neighborhood area in the non-micro-expression area, including:

[0031] Calculating the pixel similarity of each pixel in the first neighborhood area according to the gradient value and the grayscale value of each pixel in the first neighborhood area;

[0032] When the pixel similarity of each pixel in the first neighborhood area is greater than or equal to a first similarity threshold, the average grayscale value of each pixel in the first neighborhood area is used as the grayscale value of the first neighborhood area;

[0033] When the similarity of each pixel in the first neighborhood area to the pixel in the first neighborhood area is less than a first similarity threshold, the first neighborhood area is not processed.

[0034] In an optional manner, the fuzzy processing is performed on the second neighborhood area according to the similarity between each pixel point in the second neighborhood area and the first neighborhood area, including:

[0035] Calculate the difference between the grayscale value of each pixel in the second neighborhood area and the average grayscale value of the first neighborhood area;

[0036] Counting the percentage of pixels in the second neighborhood whose difference value is less than a preset gray value threshold;

[0037] When the number ratio is greater than a preset ratio threshold, the grayscale value of the second neighborhood area is set to the average grayscale value.

[0038] In an optional manner, the acquiring of continuous video frame images further includes:

[0039] Use a high-definition camera to collect video images and obtain initial video frame images;

[0040] The initial video frame images are preprocessed to obtain the continuous video frame images.

[0041] The present invention has the following beneficial effects:

[0042] The embodiment of the present invention obtains continuous video frame images, determines the micro-expression area of ​​the facial image in each video frame image according to the changes of each part area in the continuous video frame images, performs fuzzy processing on the non-micro-expression area in each video frame image, obtains the processed continuous video frame images, and performs image transmission for remote psychological consultation. The embodiment of the present invention identifies the micro-expressions of the human face according to the specific pixel information shown in the continuous video frame images, and makes the micro-expressions more prominent in the whole image by fuzzy processing on the other areas of the facial image except the micro-expression area, so as to help the psychological counselor identify the facial micro-expressions of the counselor, and can more comprehensively collect and analyze the micro-expression information of the counselor, so as to make more wise and accurate decisions and judgments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A schematic diagram showing a flow chart of a facial micro-expression image enhancement transmission method for remote psychological consultation provided by an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of the structure of the first neighborhood area and the second neighborhood area in the facial micro-expression image enhancement transmission method for remote psychological consultation provided by an embodiment of the present invention is shown;

[0046] Figure 3 A schematic diagram of traversing a neighborhood area in a facial micro-expression image enhancement transmission method for remote psychological consultation provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0047] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the facial micro-expression image enhancement transmission method for remote psychological consultation proposed by the present invention, its specific implementation method, structure, features and effects, in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0048] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0049] The following specifically describes a method for enhancing and transmitting facial micro-expression images for remote psychological consultation provided by the present invention in conjunction with the accompanying drawings.

[0050] See also Figure 1 , which shows a flow chart of a facial micro-expression image enhancement transmission method for remote psychological consultation provided by an embodiment of the present invention, and the method is executed by a computer device. The computer device can be a computer, a terminal, an intelligent device, a wearable device, etc., and the embodiment of the present invention does not make specific restrictions. Figure 1 As shown, the method comprises the following steps:

[0051] Step 110: Acquire continuous video frame images.

[0052] The embodiment of the present invention uses a high-definition camera to collect video images to obtain an initial video frame image; the initial video frame image is preprocessed to obtain the continuous video frame image. The embodiment of the present invention needs to use the facial image data of the consultant for analysis. Since the duration of micro-expressions is relatively short, after using a high-definition camera to collect continuous video frame images during remote psychological consultation, the video frame images are preprocessed such as denoising to reduce the impact of abnormal factors during the shooting process.

[0053] Step 120: Determine the micro-expression area of ​​the facial image in each video frame image according to the changes of each part area in the continuous video frame images.

[0054] The embodiment of the present invention identifies the micro-expression of the consultant's face according to the pixel difference shown in the continuous video frame images. The consultant may turn his head and other behaviors during the remote consultation process. Since the presentation of micro-expressions is mainly distributed on the consultant's face, it is first necessary to use the existing technology to align the facial images on the continuous video frame images collected by the high-definition camera, so as to facilitate subsequent analysis and processing. Secondly, since micro-expressions appear briefly on the consultant's face image and the change amplitude is small, it is difficult for the psychological counselor to grasp these key factors during the remote consultation process. The embodiment of the present invention analyzes the difference between the same area on the consultant's face image in the previous and next frame images, and identifies the consultant's micro-expression changes through the pixel-level difference on the image, which is convenient for improving the contrast between the micro-expression area and other areas during the subsequent transmission process, so that the psychological counselor can quickly grasp the consultant's micro-expression changes.

[0055] Wherein, according to the changes between the video frame images, determining the micro-expression area of ​​the facial image in each video frame image specifically includes the following steps:

[0056] 1201: Determine the facial image in each video frame image.

[0057] First, the pixel point at the lower left corner of the collected image is used as the coordinate origin to construct an XY coordinate system. The face recognition algorithm is used to perform face image recognition on each video frame image to identify the initial face image in each video frame image. Specifically, the existing face detection technology is used to identify the face image of the consultant in the video frame image; for the detected face, the face key point detection algorithm (such as FAN) is used to detect the key points in the face, such as eyes, nose, etc.

[0058] Afterwards, the key point detection algorithm is used to detect the key points of the initial face image to obtain the key point positions, and the face rotation angle is determined according to the key point positions. Then, according to the face rotation angle, the initial face image is rotationally corrected to obtain a corrected video frame image. Finally, according to the corrected video frame image and the key point positions, the corrected video frame image is cropped to obtain the facial image corresponding to the face area. Specifically, according to the detected key point positions, the rotation angle of the face is calculated, and the image is rotationally corrected using affine transformation to ensure the consistency of the face direction. According to the rotation-corrected image and the key point positions, the face area is cropped and adjusted to a uniform size. Since the changes in the face image have little interference with the nose position, the present application utilizes the fact that the key points of the nose in different face images are roughly the same in the image, thereby achieving face alignment.

[0059] 1202: Determine each part region in the facial image using a part region recognition model.

[0060] Since micro-expressions are subtle changes in facial expressions and have the characteristics of local changes, the trained part area recognition model is used to identify the various parts of the consultant's face image, such as the eye area, nose area, etc. The part area recognition model is obtained by pre-training the neural network model based on the part recognition samples.

[0061] 1203: Analyze the motion expression in each area of ​​the continuous video frame images.

[0062] The degree of motion expression in each part area in each video frame image is determined based on the difference in grayscale values ​​of each position in the part area on the continuous video frame images and the difference in the part area on the current edge detection image on the continuous video frame images. The edge detection image is an image obtained after edge detection of the video frame images.

[0063] Specifically, analyze the consultant's continuous facial images in The moment Sports performance in different areas:

[0064]

[0065] In the formula, Indicates the number of consecutive face images in The moment Movement performance in each area, Indicates the face image Part area, Indicates the first The face image corresponding to the moment, represents the vth pixel, Indicates that the consultant The moment The first The gray value of each pixel is Indicates that the consultant The moment The first The gray value of each pixel; Represents the number of pixels in the i-th part area; Indicates the time t The number of pixels in the area. Indicates that the consultant The moment The motion on each part area represents the number of pixels with a gray value of 1 in the edge detection image of each part area; Indicates that the consultant The moment The motion performance on a part area is the number of pixels with a gray value of 1 in the edge detection image of the part area.

[0066] in, Indicates the first Moment and Same area at the same time The difference in gray value; Indicates the same area In the Moment and The difference in edge detection images of face images at different moments.

[0067] In the above formula, the difference in grayscale values ​​at the same position on consecutive video frames and the difference in edge detection images of the same area in different frames are used to determine the consultant's The moment Movement performance in each area . Since the analysis of the consultant's micro-expressions is based on the alignment of facial images on continuous video frame images, when the grayscale values ​​at the same position are very different, it means that the face may be in motion. And when there is motion on the face, for characteristic areas such as the eyes and mouth, they will also show certain differences in the edge detection image due to the influence of light and skin wrinkles. Therefore, the embodiment of the present invention uses the difference in grayscale values ​​at the same position on continuous video frame images, and the difference in the edge detection images of the same area in different frame images, to determine the consultant's micro-expressions. The moment The motion performance of each part area can be obtained by the above method.

[0068] 1204: Determine a target part region belonging to the micro-expression region in each video frame image according to the motion expression degree in each part region, wherein the target part region is any part region belonging to the micro-expression region in each part region in the facial image.

[0069] Among them, after determining the motion expression of each part area in the face image, when the motion expression degree in the part area is greater than the preset motion expression degree threshold, the part area is determined to be the target part area belonging to the micro-expression area; when the motion expression degree of the surrounding part area is in the preset motion expression degree interval, the surrounding part area is determined to be the target part area belonging to the micro-expression area. Among them, the surrounding part area is each of the part areas around the part area whose motion expression degree is greater than the preset motion expression degree threshold. In combination with the facial expression of a normal face, when a certain part area undergoes motion changes, it is often not only one part area that changes. For example, when raising eyebrows, the eyes will also change slightly. Therefore, the present application combines the motion expression of adjacent parts to further analyze whether each part area belongs to the micro-expression area. Specifically, the motion expression of each part area has been calculated in the above process. Therefore, a motion expression degree threshold is set. In the present application, the motion expression degree threshold is set to , when calculated , it means that the area is the main micro-expression change, and it belongs to the target area of ​​the micro-expression area. The motion performance of the surrounding areas is analyzed. If the surrounding areas have the calculated motion performance , then it is considered that this part also belongs to the target part area of ​​the micro-expression area, that is, the micro-expression area. So far, the micro-expression recognition on the face image is completed.

[0070] 1205: Obtaining micro-expression regions of facial images in each video frame image according to each target part region in the video frame image.

[0071] Among them, each target part area is a micro-expression area. After determining the target part area in the face image at each moment, the micro-expression area of ​​the face image in each video frame image can be obtained.

[0072] Step 130: performing blurring processing on the non-micro-expression regions in each video frame image to obtain processed continuous video frame images.

[0073] The non-micro-expression area is an area in the video frame image other than the micro-expression area.

[0074] In an embodiment of the present invention, the non-micro-expression area in each video frame image is blurred to obtain a processed continuous video frame image. Since the consultant's micro-expressions occur in a short time and with a small amplitude, they are difficult to be captured by the psychological counselor. In order to help the psychological counselor quickly capture the changes in the consultant's micro-expressions, the present application processes the image during the transmission process, and blurs the pixels in the non-micro-expression area outside the micro-expression area, so that the micro-expressions are more prominent on the facial image of the human face. However, since the psychological counselor still needs to obtain the overall appearance of the consultant during remote psychological counseling, the area where the non-micro-expression area is located is blurred to a certain extent according to the performance of the pixels in the non-micro-expression area, while ensuring that the human face can be observed by the psychological counselor, its clarity is appropriately reduced, thereby making the micro-expressions more prominent. Specifically, the following steps are included:

[0075] 1301: For the non-micro-expression area, according to the similarity of each pixel point in the first neighborhood area, perform fuzzy processing on the first neighborhood area.

[0076] a. Calculate the pixel similarity of each pixel in the first neighborhood area according to the gradient value and grayscale value of each pixel in the first neighborhood area.

[0077] Among them, Figure 2 As shown in the figure, the first pixel point in the non-micro-expression area (from left to right, from top to bottom) is selected as the starting point for judgment. The number of pixels in the neighborhood is similar to that of the neighboring pixels, so the application can set the first neighborhood area to , judged in The image collected at the time Pixels in Similarity of pixels in the neighborhood:

[0078]

[0079] In the formula, Indicated in The image collected at the time Pixels in The similarity of pixels in the neighborhood area; Indicates the first The face image corresponding to the moment, exist The first neighborhood area collected at the time pixels; In the non-micro-expression area Pixels Neighborhood, In the non-micro-expression area Pixels Neighborhood pixels; In the non-micro-expression area Pixels The first The pixel near pixels; Indicated in The non-micro-expression area collected at the moment Pixels in The gradient value of the pixel in the neighborhood; Indicated in The non-micro-expression area collected at the moment Pixels The variance of the grayscale values ​​of pixels in the neighborhood; Indicated in The non-micro-expression area collected at the moment Pixels Neighborhood The gray value of a pixel, Shown in The non-micro-expression area collected at the moment Pixels The first The pixel near The gray value of each pixel ( The maximum value is 3). Indicated in Pixels in The product of the gradient value and the gray value variance of the pixel in the neighborhood; Indicated in Pixels in The minimum difference between the grayscale values ​​of pixels in a neighborhood.

[0080] Among them, the above formula is mainly used in Pixels in The inverse of the product of the gradient value and the gray value variance of the pixel in the neighborhood , and in Pixels in The reciprocal of the minimum difference between the grayscale values ​​of pixels in the neighborhood When the pixel The smaller the gradient value of the pixel points in the neighborhood, the smaller the grayscale value variance, indicating that the grayscale values ​​of the pixels are similar to each other. However, in this neighborhood, there may be four pixels with grayscale values ​​that are small and gradually increase. For this case, this application uses the average minimum grayscale value between the pixels to further restrict. When the pixel points The average minimum grayscale value between pixels in the neighborhood is also small, indicating that the grayscale values ​​of the pixels are more similar.

[0081] b. When the similarity of each pixel in the first neighborhood area is greater than or equal to a first similarity threshold, the average grayscale value of each pixel in the first neighborhood area is used as the grayscale value of the first neighborhood area.

[0082] Among them, the first similarity threshold T is set to , when the calculated , without any processing, calculate the next pixel.

[0083] When the calculated , then it means The non-micro-expression area of ​​the face image collected at the moment Pixels in If the grayscale values ​​of pixels in the neighborhood are close, then their average grayscale value is calculated. , taking the average grayscale value of each pixel in the first neighborhood area as the first neighborhood area ( The grayscale value of the neighborhood area.

[0084] c. When the similarity of each pixel in the first neighborhood area to the pixel in the first neighborhood area is less than a first similarity threshold, the first neighborhood area is not processed.

[0085] 1302: Perform fuzzy processing on the second neighborhood area according to the similarity between each pixel in the second neighborhood area and the first neighborhood area, wherein the second neighborhood area is an area formed by pixels that are a preset number of pixels away from the first neighborhood area.

[0086] In this application, after determining the similarity of the pixels in the first neighborhood area, it is necessary to further consider the performance of the pixels outside the first neighborhood area. When the neighborhood area is ( ), this application is judged from left to right and from top to bottom, so the added row and column are located at Below the neighborhood. In this case, the above formula cannot be used to calculate The similarity of the image points in the neighborhood area, because The grayscale value of the pixel in the neighborhood area has been changed. If we continue to use the above formula to change it, then The grayscale value of the pixel points in the neighborhood area may be further changed, resulting in a large difference between it and the original grayscale value. Therefore, the present application calculates the difference between the grayscale value of each pixel point in the second neighborhood area and the average grayscale value of the first neighborhood area; counts the number of pixel points in the second neighborhood area whose difference is less than the preset grayscale value threshold; when the number ratio is greater than the preset ratio threshold, sets the grayscale value of the second neighborhood area to the average grayscale value. Specifically, we calculate according to the direction (two directions: downward and right). Calculate the first direction in each direction. The gray value of a pixel is The difference in size, when , it means that the gray value of this pixel is The grayscale values ​​in two directions are statistically The number of pixels that are closer . Calculate the gray value and The proportion of pixels that are closer When the calculated , then the grayscale values ​​in this direction are set to .

[0087] 1303: Use the second neighborhood area as the first neighborhood area, and continue to execute the step of blurring the second neighborhood area according to the similarity between each pixel point in the second neighborhood area and each pixel point in the first neighborhood area, until all the non-micro-expression areas are traversed to obtain the processed continuous video frame image.

[0088] Among them, Figure 3 As shown, repeat the above steps to continue calculating Pixels in the neighborhood. When both are less than 0.7, the calculation is terminated. When only one of the two results is greater than 0.7, the grayscale value in this direction is set to , and continue to calculate in this direction until all the non-micro-expression areas are traversed to obtain the processed continuous video frame images. In the above process, the grayscale value of the pixel points in the non-micro-expression area is blurred, and the area where the micro-expression is located is not processed, so that when the consultant shows a micro-expression, the image outside the micro-expression is relatively unclear, but still retains its general information, and the area where the micro-expression is located is better highlighted on the consultant's facial image, which is convenient for psychological counselors to timely understand the patient's micro-expression performance, comprehensively collect and analyze the micro-expression information of the consulting object, and make more wise and accurate decisions and judgments.

[0089] Step 140: Perform image transmission for remote psychological consultation based on the processed continuous video frame images.

[0090] Among them, after processing the continuous video frame images, they are transmitted to the remote psychological counseling end. Since the non-micro-expression area is blurred, it not only meets the requirements of enhancing the micro-expression image, but also compresses the image data during the transmission process, and the data transmission cost is further reduced.

[0091] The embodiment of the present invention obtains continuous video frame images, determines the micro-expression area of ​​the facial image in each video frame image according to the changes between the video frame images, enhances the contrast between the micro-expression area and the non-micro-expression area in each video frame image, and performs image transmission of remote psychological consultation according to the processed continuous video frame images. The time for micro-expressions to occur is often short, and the amplitude of facial changes is small, which may cause the psychological counselor to fail to capture the micro-expression changes of the consultant in time, making it impossible for the psychological counselor to analyze the potential emotions that the consultant may want to express. The embodiment of the present invention identifies the micro-expressions of the face of the human face according to the specific pixel information shown in the continuous video frame images, and blurs the other areas of the facial image except the micro-expression area, so that the micro-expressions are more prominent in the whole image, helping the psychological counselor to identify the facial micro-expressions of the consultant, and can more comprehensively collect and analyze the micro-expression information of the consulting object, so as to make more wise and accurate decisions and judgments.

[0092] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0093] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A facial micro-expression image enhancement transmission method for remote psychological counseling, characterized in that: The method comprises: Acquire continuous video frame images; Determine the micro-expression area of ​​the facial image in each video frame image according to the changes of each part area in the continuous video frame images; Performing fuzzy processing on the non-micro-expression area in each video frame image to obtain processed continuous video frame images; the non-micro-expression area is the area in the video frame image other than the micro-expression area; According to the processed continuous video frame images, image transmission of remote psychological consultation is performed; according to the changes between various parts and regions in the continuous video frame images, the micro-expression region of the facial image in each video frame image is determined, including: Determine the facial image in each video frame image; Using a part region recognition model to determine each part region in the facial image; the part region recognition model is obtained by pre-training a neural network model based on part recognition samples; Analyze the motion expression of each area in the continuous video frame image; According to the motion expression degree of each part area, determine the target part area belonging to the micro-expression area in each video frame image; the target part area is any part area belonging to the micro-expression area in each part area in the facial image; According to each target part area in the video frame image, obtaining the micro-expression area of ​​the facial image in each video frame image; The analyzing the motion expression of each region in the continuous video frame images includes: Determine the degree of motion expression in each part area in each video frame image according to the difference in grayscale values ​​of each position in the part area on the continuous video frame images and the difference in grayscale values ​​of each position in the part area on the continuous video frame images on the current edge detection image; the edge detection image is an image obtained after edge detection of the video frame images; Analyze the consultant's continuous facial images The moment Sports performance in different areas: In the formula, Indicates the number of consecutive face images in The moment Movement performance in each area, Indicates the face image Part area, Indicates the first The face image corresponding to the moment, represents the vth pixel, Indicates that the consultant The moment The first The gray value of each pixel is Indicates that the consultant The moment The first The gray value of each pixel; Represents the number of pixels in the i-th part area; Indicates the time t The number of pixels in each part area; Indicates that the consultant The moment The motion on each part area represents the number of pixels with a gray value of 1 in the edge detection image of each part area; Indicates that the consultant The moment The motion on the part area represents the number of pixels with a gray value of 1 in the edge detection image of the part area; among them, Indicates the first Moment and Same area at the same time The difference in gray value; Indicates the same area In the Moment and The difference in edge detection images of face images at different moments.

2. The facial micro-expression image enhancement transmission method for remote psychological consultation according to claim 1, characterized in that: Determining the target part area belonging to the micro-expression area in each video frame image according to the motion expression degree in each part area includes: When the motion expression degree in the part area is greater than a preset motion expression degree threshold, determining the part area as a target part area belonging to a micro-expression area; When the motion expression of the surrounding part area is within a preset motion expression range, the surrounding part area is determined to be the target part area belonging to the micro-expression area; the surrounding part area is each of the part areas around the part area whose motion expression is greater than a preset motion expression threshold.

3. The facial micro-expression image enhancement transmission method for remote psychological consultation according to claim 1, characterized in that: The step of determining the facial image in each video frame image comprises: Performing face image recognition on each video frame image using a face recognition algorithm to identify an initial face image in each video frame image; Using a facial key point detection algorithm to perform key point detection on the initial facial image to obtain key point positions; Determine the face rotation angle based on the key point position; According to the face rotation angle, the initial face image is rotationally corrected to obtain a corrected video frame image; The corrected video frame image is cropped according to the corrected video frame image and the key point positions to obtain the facial image corresponding to the face area.

4. The facial micro-expression image enhancement transmission method for remote psychological consultation according to any one of claims 1 to 3, characterized in that: The fuzzy processing is performed on the non-micro-expression area in each video frame image to obtain the processed continuous video frame images, including: For the non-micro-expression area, fuzzy processing is performed on the first neighborhood area according to the similarity of each pixel point in the first neighborhood area; According to the similarity between each pixel point in the second neighborhood area and the first neighborhood area, fuzzy processing is performed on the second neighborhood area; the second neighborhood area is an area composed of pixel points with a preset number of pixel points away from the first neighborhood area; The second neighborhood area is used as the first neighborhood area, and the step of blurring the second neighborhood area according to the similarity between each pixel point in the second neighborhood area and each pixel point in the first neighborhood area is continued, until all the non-micro-expression areas are traversed to obtain the processed continuous video frame images.

5. The facial micro-expression image enhancement transmission method for remote psychological consultation according to claim 4 is characterized in that: The first neighborhood area is a neighborhood area consisting of k pixels sampled from top to bottom and from left to right, starting from the first pixel point in the upper left corner of the non-micro-expression area.

6. The facial micro-expression image enhancement transmission method for remote psychological consultation according to claim 4, characterized in that: The fuzzy processing is performed on the first neighborhood area according to the similarity of each pixel point in the first neighborhood area in the non-micro-expression area, including: Calculating the pixel similarity of each pixel in the first neighborhood area according to the gradient value and the grayscale value of each pixel in the first neighborhood area; When the pixel similarity of each pixel in the first neighborhood area is greater than or equal to a first similarity threshold, the average grayscale value of each pixel in the first neighborhood area is used as the grayscale value of the first neighborhood area; When the similarity of each pixel in the first neighborhood area to the pixel in the first neighborhood area is less than a first similarity threshold, the first neighborhood area is not processed.

7. The facial micro-expression image enhancement transmission method for remote psychological consultation according to claim 5, characterized in that: The fuzzy processing of the second neighborhood area according to the similarity between each pixel point in the second neighborhood area and the first neighborhood area includes: Calculate the difference between the grayscale value of each pixel in the second neighborhood area and the average grayscale value of the first neighborhood area; Counting the percentage of pixels in the second neighborhood whose difference value is less than a preset gray value threshold; When the number ratio is greater than a preset ratio threshold, the grayscale value of the second neighborhood area is set to the average grayscale value.

8. The facial micro-expression image enhancement transmission method for remote psychological consultation according to any one of claims 1 to 3, characterized in that: The obtaining of continuous video frame images further comprises: Use a high-definition camera to collect video images and obtain initial video frame images; The initial video frame images are preprocessed to obtain the continuous video frame images.

Images