Neck wrinkle detection method, system, terminal device and storage medium

Abstract

The present invention provides a neck wrinkle detection method, system, terminal device, and storage medium. The method includes: inputting a sample neck image into a teacher model and a student model respectively for semantic segmentation to obtain a first segmented image and a second segmented image; training the student model according to the first segmented image and the second segmented image until convergence; sampling a video to be processed to obtain sample images, and determining an image association relationship between each sample image and the remaining images; inputting each sample image into the student model for neck wrinkle detection to obtain a neck wrinkle image, and setting the neck wrinkle image of each sample image as the neck wrinkle image of the corresponding remaining image according to the image association relationship. The present invention eliminates the need for the user to manually select a neck wrinkle image by inputting each sample image into the converged student model for neck wrinkle detection, and setting the neck wrinkle image of each sample image as the neck wrinkle image of the corresponding remaining image according to the image association relationship, thereby improving the accuracy of neck wrinkle detection.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a neck wrinkle detection method, system, terminal device and storage medium. Background Art

[0002] With neck wrinkles appearing at younger ages, more and more people, especially those who are addicted to their phones, are experiencing them. To improve the aesthetics of photos, neck wrinkle detection is gaining increasing attention. Users can remove neck wrinkles from images by deleting or overwriting detected neck wrinkles.

[0003] In the existing neck wrinkle processing process, the image manually selected by the user is set as the neck wrinkle detection result, which leads to low detection accuracy of the neck wrinkle image and reduces the user experience. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a neck wrinkle detection method, system, terminal device and storage medium, aiming to solve the problem of low neck wrinkle detection accuracy caused by setting the image manually selected by the user as the neck wrinkle detection result in the existing neck wrinkle processing process.

[0005] The embodiment of the present invention is implemented as follows: a neck wrinkle detection method, the method comprising:

[0006] The sample neck image is input into the teacher model and the student model for semantic segmentation respectively, and the first segmentation image and the second segmentation image are obtained;

[0007] Calculating the loss of the student model according to the first segmented image and the second segmented image to obtain a model loss value, and updating parameters of the student model according to the model loss value until the student model converges;

[0008] Sampling the video to be processed to obtain at least one sampled image, and determining an image association relationship between each sampled image and the remaining images in the video to be processed;

[0009] Each sampling image is input into the converged student model to perform neck wrinkle detection to obtain a neck wrinkle image of each sampling image, and according to the image association relationship, the neck wrinkle image of each sampling image is set as the neck wrinkle image of the corresponding remaining image.

[0010] Furthermore, the sample neck image is input into the teacher model and the student model respectively for semantic segmentation to obtain a first segmented image and a second segmented image, including:

[0011] Inputting the sample neck image into the teacher model for downsampling to obtain a first sampling result, and performing attention fusion on the first sampling result to obtain a first fusion feature;

[0012] Perform upsampling convolution on the first fusion feature and the first sampling result to obtain the first segmented image;

[0013] Inputting the sample neck image into the student model for downsampling to obtain a third sampling result, and downsampling the third sampling result to obtain a fourth sampling result;

[0014] The third sampling result and the fourth sampling result are subjected to feature fusion to obtain a second fused feature, and the second fused feature is subjected to upsampling convolution to obtain the second segmented image.

[0015] Furthermore, the step of inputting the sample neck image into the teacher model for downsampling to obtain a first sampling result includes:

[0016] Inputting the sample neck image into the teacher model for performing expanded channel downsampling and downsampling convolution respectively to obtain a first sampling feature and a second sampling feature;

[0017] Down-sampling convolution is performed on the first sampling feature to obtain a third sampling feature, wherein the first sampling result includes the first sampling feature, the second sampling feature, and the third sampling feature.

[0018] Furthermore, performing attention fusion on the first sampling result to obtain a first fusion feature includes:

[0019] Superimposing the third sampling feature and the second sampling feature to obtain a first superimposed feature;

[0020] Performing upsampling convolution on the second sampling feature to obtain a fourth sampling feature, and superimposing the fourth sampling feature with the third sampling feature to obtain a second superimposed feature;

[0021] The first superimposed feature and the second superimposed feature are fused to obtain a third fused feature, and the third fused feature is convolved to obtain the first fused feature.

[0022] Furthermore, performing upsampling convolution on the first fusion feature and the first sampling result to obtain the first segmented image includes:

[0023] Upsampling the first fused feature, and fusing the upsampled first fused feature with the first sampled feature to obtain a fourth fused feature;

[0024] Perform upsampling convolution on the fourth fusion feature to obtain the first segmented image.

[0025] Furthermore, the inputting the sample neck image into the student model for downsampling to obtain a third sampling result, and downsampling the third sampling result to obtain a fourth sampling result, includes:

[0026] Inputting the sample neck image into the student model for expanded channel downsampling to obtain the third sampling result;

[0027] Performing depth-wise separable downsampling convolution on the third sampling result to obtain the fourth sampling result.

[0028] Furthermore, the performing feature fusion on the third sampling result and the fourth sampling result to obtain a second fusion feature includes:

[0029] The fourth sampling result is upsampled, and feature fusion is performed on the upsampled fourth sampling result and the third sampling result to obtain the second fused feature.

[0030] Furthermore, the sampling of the video to be processed to obtain at least one sampled image and determining an image association relationship between each sampled image and the remaining images in the video to be processed includes:

[0031] Sampling the video to be processed at alternate frames to obtain at least one sampled image;

[0032] The relationship between each sampled image and the remaining images of the corresponding next frame is determined as the image association relationship.

[0033] Furthermore, the step of setting the neck wrinkle image of each sampled image as the neck wrinkle image of the corresponding remaining image according to the image association relationship includes:

[0034] respectively determining, in each image association relationship, a position displacement of the neck region between the sampled image and a corresponding remaining image of the next frame;

[0035] For each image association relationship, according to the position displacement of the neck region, the neck wrinkle image of the sampled image is overlaid onto the remaining image of the corresponding next frame to obtain the neck wrinkle image of each remaining image.

[0036] Furthermore, the method further comprises:

[0037] According to the image position of each neck wrinkle image, mask processing is performed on the corresponding sampled image and the remaining images in the video to be processed to obtain a neck wrinkle output video.

[0038] Another object of an embodiment of the present invention is to provide a neck wrinkle detection system, the system comprising:

[0039] A semantic segmentation module is used to input the sample neck image into the teacher model and the student model respectively for semantic segmentation to obtain a first segmented image and a second segmented image;

[0040] a student model training module, configured to calculate a loss of the student model according to the first segmented image and the second segmented image to obtain a model loss value, and update parameters of the student model according to the model loss value until the student model converges;

[0041] an association relationship determination module, configured to sample the video to be processed to obtain at least one sampled image, and determine an image association relationship between each sampled image and the remaining images in the video to be processed;

[0042] The neck wrinkle detection module is used to input each sampling image into the converged student model for neck wrinkle detection, obtain the neck wrinkle image of each sampling image, and set the neck wrinkle image of each sampling image as the neck wrinkle image of the corresponding remaining image according to the image association relationship.

[0043] Another object of an embodiment of the present invention is to provide a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0044] Another object of an embodiment of the present invention is to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0045] The embodiment of the present invention adopts the knowledge distillation technology to train the student model, so that the converged student model reduces the computational amount of the model while ensuring the accuracy of neck wrinkle detection for each sampled image, thereby improving the efficiency of neck wrinkle detection for each sampled image. By determining the image association relationship between each sampled image and the remaining images in the video to be processed, and setting the neck wrinkle image of each sampled image as the neck wrinkle image of the corresponding remaining image according to the image association relationship, it is avoided to use the converged student model for neck wrinkle detection for each frame of the image, thereby improving the efficiency of neck wrinkle detection for the image in the video to be processed, and improving the coherence between the sampled image and the corresponding remaining image. By inputting each sampled image into the converged student model for neck wrinkle detection, and setting the neck wrinkle image of each sampled image as the neck wrinkle image of the corresponding remaining image according to the image association relationship, it is unnecessary for the user to manually select the neck wrinkle image, thereby improving the accuracy of neck wrinkle detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flowchart of a neck wrinkle detection method provided by a first embodiment of the present invention;

[0047] Figure 2 is a flowchart of a neck wrinkle detection method provided by a second embodiment of the present invention;

[0048] Figure 3 for Figure 2 The specific implementation flow chart of step S10 provided in the embodiment;

[0049] Figure 4 is a structural diagram of a neck wrinkle detection system provided by a third embodiment of the present invention;

[0050] Figure 5 It is a schematic structural diagram of a terminal device provided in the fourth embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0052] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0053] Example 1

[0054] See also Figure 1 , is a flowchart of a neck wrinkle detection method provided by a first embodiment of the present invention. The neck wrinkle detection method can be applied to any terminal device, including a server, a mobile phone, a tablet, or a wearable smart device. The neck wrinkle detection method includes the following steps:

[0055] Step S10, inputting the sample neck image into the teacher model and the student model respectively for semantic segmentation to obtain a first segmented image and a second segmented image;

[0056] Among them, by collecting images of the face including the neck area, and annotating the collected images with neck lines as sample images, using the face detection algorithm to detect the face position in the sample image, the detected face position is translated downward by a preset distance to obtain a neck image, and the neck image is cropped and scaled to a fixed size to obtain the sample neck image. The preset distance and fixed size can be set according to needs. For example, the preset distance can be set to 5 cm, 6 cm or 7 cm, etc.

[0057] In this step, the sample neck image is input into the teacher model and the student model respectively for semantic segmentation to segment the neck image in the sample neck image, thereby obtaining a first segmented image and a second segmented image.

[0058] Step S20, calculating the loss of the student model according to the first segmented image and the second segmented image to obtain a model loss value, and updating the parameters of the student model according to the model loss value until the student model converges;

[0059] In the knowledge distillation framework, the teacher network model has greater capacity and computational complexity than the student network, and its prediction results are better than those of a model trained using only input images and annotations. However, its speed is slower than that of the student network, failing to achieve real-time performance. Therefore, the teacher network's predictions are given additional supervision by the student network to improve its prediction accuracy.

[0060] In this step, during training, the student network prediction result (the second segmented image) not only needs to be subjected to cross entropy loss with the annotation on the sample neck image to obtain a first loss value, but also needs to be subjected to cross entropy loss with the teacher network prediction result (the first segmented image) to obtain a second loss value. The sum of the first loss value and the second loss value is calculated to obtain the model loss value. By performing cross entropy loss on the second segmented image and the first segmented image, the teacher network prediction result can effectively provide additional supervision to the student network to improve the accuracy of the student network prediction.

[0061] Step S30: sampling the video to be processed to obtain at least one sampled image, and determining an image association relationship between each sampled image and the remaining images in the video to be processed;

[0062] Among them, by determining the image association relationship between each sampled image and the remaining images in the video to be processed, the accuracy of subsequent setting of the neck wrinkle image in the remaining images is effectively improved.

[0063] Optionally, in this step, sampling the video to be processed to obtain at least one sampled image, and determining an image association relationship between each sampled image and the remaining images in the video to be processed includes:

[0064] Sampling the video to be processed at alternate frames to obtain at least one sampled image;

[0065] Determine the relationship between each sampled image and the remaining image of the corresponding next frame as the image association relationship;

[0066] For example, when the video to be processed includes a first frame image, a second frame image, a third frame image, a fourth frame image, a fifth frame image and a sixth frame image, the sampled images obtained by performing intermittent frame sampling on the video to be processed include the first frame image, the third frame image and the fifth frame image, and the relationships between the first frame image and the second frame image, the third frame image and the fourth frame image, and the fifth frame image and the sixth frame image are respectively determined as image association relationships.

[0067] Step S40, inputting each sampling image into the converged student model to perform neck wrinkle detection to obtain a neck wrinkle image of each sampling image, and setting the neck wrinkle image of each sampling image as the neck wrinkle image of the corresponding remaining image according to the image association relationship;

[0068] Among them, the neck wrinkle detection is performed by inputting each sampling image into the converged student model to obtain the neck wrinkle image in each sampling image. According to the image association relationship, the neck wrinkle image of each sampling image is set as the neck wrinkle image of the corresponding remaining image, avoiding the use of the converged student model for neck wrinkle detection in each frame of the image, improving the efficiency of neck wrinkle detection of the image in the processed video, and improving the coherence between the sampling image and the corresponding remaining image.

[0069] Optionally, in this step, setting the neck wrinkle image of each sampled image as the neck wrinkle image of the corresponding remaining image according to the image association relationship includes:

[0070] respectively determining, in each image association relationship, a position displacement of the neck region between the sampled image and a corresponding remaining image of the next frame;

[0071] Among them, by respectively determining the position displacement of the neck area between the sampled image and the remaining image of the corresponding next frame in the association relationship of each image, the accuracy of the image coverage between the neck wrinkle image of each subsequent sampled image and the remaining image of the corresponding next frame is improved;

[0072] For each image association relationship, according to the position displacement of the neck area, the neck wrinkle image of the sampled image is overlaid onto the remaining image of the corresponding next frame to obtain the neck wrinkle image of each remaining image;

[0073] Among them, through the position displacement of the neck area, the neck wrinkle image of the sampled image is overlaid on the remaining images of the corresponding next frame, which effectively improves the accuracy of the neck wrinkle image setting in each remaining image. For example, for the image association relationship between the first frame image and the second frame image, the third frame image and the fourth frame image, and the fifth frame image and the sixth frame image, the neck area position displacement between the first frame image and the second frame image, the third frame image and the fourth frame image, and the fifth frame image and the sixth frame image are determined respectively to obtain the first position displacement, the second position displacement and the third position displacement. According to the first position displacement, the neck wrinkle image of the first frame image is overlaid on the second frame image, according to the second position displacement, the neck wrinkle image of the third frame image is overlaid on the fourth frame image, and according to the third position displacement, the neck wrinkle image of the fifth frame image is overlaid on the sixth frame image.

[0074] Furthermore, in this embodiment, the method further includes:

[0075] According to the image position of each neck wrinkle image, mask processing is performed on the corresponding sampled image and the remaining image in the video to be processed to obtain a neck wrinkle output video;

[0076] Among them, by performing mask processing on the corresponding sampling images and remaining images in the video to be processed, the neck wrinkle images in each sampling image and the remaining images can be effectively converted into mask images. Based on the image position of the converted mask image, the subsequent image processing operations on each neck wrinkle image are effectively facilitated.

[0077] In this embodiment, the student model is trained by adopting knowledge distillation technology, so that the converged student model reduces the computational amount of the model while ensuring the accuracy of neck wrinkle detection for each sampled image, thereby improving the efficiency of neck wrinkle detection for each sampled image. By determining the image association relationship between each sampled image and the remaining images in the video to be processed, and setting the neck wrinkle image of each sampled image to the neck wrinkle image of the corresponding remaining image according to the image association relationship, it is avoided to use the converged student model for neck wrinkle detection for each frame of image, thereby improving the efficiency of neck wrinkle detection for images in the video to be processed, and improving the coherence between the sampled image and the corresponding remaining image. By inputting each sampled image into the converged student model for neck wrinkle detection, and setting the neck wrinkle image of each sampled image to the neck wrinkle image of the corresponding remaining image according to the image association relationship, there is no need for the user to manually select the neck wrinkle image, thereby improving the accuracy of neck wrinkle detection.

[0078] Example 2

[0079] See also Figure 2 , is a flow chart of a neck wrinkle detection method provided by a second embodiment of the present invention, which further refines step S10 and includes the following steps:

[0080] Step S11, inputting the sample neck image into the teacher model for downsampling to obtain a first sampling result, performing attention fusion on the first sampling result to obtain a first fusion feature;

[0081] Optionally, in this step, inputting the sample neck image into the teacher model for downsampling to obtain a first sampling result includes:

[0082] Inputting the sample neck image into the teacher model for performing expanded channel downsampling and downsampling convolution respectively to obtain a first sampling feature and a second sampling feature;

[0083] Performing downsampling convolution on the first sampling feature to obtain a third sampling feature;

[0084] The first sampling result includes the first sampling feature, the second sampling feature and the third sampling feature. There are two downsampling branches in the teacher model. The first branch is called the detail branch. Figure 3 , which is a specific implementation flow chart of step S10 provided in this embodiment. First, the sample neck image is downsampled through the expanded channel to obtain a first sampling feature without losing information. The expanded channel downsampling obtains multiple grid-shaped sampling feature maps in the row and column directions of the sample neck image with a certain step, and each grid of the same size is spliced ​​in the channel dimension, so that the spatial size of the sampled image is reduced and the channel dimension is correspondingly enlarged. Then, the downsampling convolution is used to extract higher-dimensional features from the first sampling feature to obtain a third sampling feature.

[0085] The second branch is the semantic branch, which only uses downsampling convolution on the sample neck image to obtain the second sampling feature, and the downsampling multiple is greater than the downsampling multiple of the detail branch to obtain a smaller and higher-dimensional feature.

[0086] Furthermore, in this step, performing attention fusion on the first sampling result to obtain a first fusion feature includes:

[0087] Superimposing the third sampling feature and the second sampling feature to obtain a first superimposed feature;

[0088] Performing upsampling convolution on the second sampling feature to obtain a fourth sampling feature, and superimposing the fourth sampling feature with the third sampling feature to obtain a second superimposed feature;

[0089] Performing feature fusion on the first superimposed feature and the second superimposed feature to obtain a third fused feature, and performing convolution on the third fused feature to obtain the first fused feature;

[0090] The size of the detail branch is larger than that of the semantic branch, and the third sampling feature is an attention map with a value of each position in the range of 0 to 1 of the same size as the semantic branch. The third sampling feature is multiplied by the semantic branch result (the second sampling feature) to obtain a first superposition feature;

[0091] The second sampling feature in the semantic branch is upsampled and convolved to obtain a fourth sampling feature, which is an attention map with the same size as the detail branch and the value of each position in the range of 0 to 1. It is multiplied by the third sampling feature in the detail branch to obtain a second superimposed feature, and then the two branches multiplied by the attention are merged and convolved, that is, the first superimposed feature and the second superimposed feature are feature fused to obtain a third fused feature, and the third fused feature is convolved to obtain the first fused feature.

[0092] Step S12: performing upsampling convolution on the first fusion feature and the first sampling result to obtain the first segmented image;

[0093] Optionally, in this step, performing upsampling convolution on the first fusion feature and the first sampling result to obtain the first segmented image includes:

[0094] Upsampling the first fused feature, and fusing the upsampled first fused feature with the first sampled feature to obtain a fourth fused feature;

[0095] Performing upsampling convolution on the fourth fusion feature to obtain the first segmented image;

[0096] The fourth fusion feature is obtained by upsampling the first fusion feature and fusing the upsampled first fusion feature with the first sampling feature to achieve the effect of channel fusion upsampling convolution;

[0097] After the detail branch is merged with the semantic branch, the size is the same as the downsampling result of the detail branch. The upsampled image is fused with the feature map of the detail branch before downsampling (the first sampling feature) and convolved. Finally, the upsampled image is convolved to the input image size to obtain the first segmented image.

[0098] Step S13, inputting the sample neck image into the student model for downsampling to obtain a third sampling result, and downsampling the third sampling result to obtain a fourth sampling result;

[0099] Optionally, in this step, inputting the sample neck image into the student model for downsampling to obtain a third sampling result, and downsampling the third sampling result to obtain a fourth sampling result includes:

[0100] Inputting the sample neck image into the student model for expanded channel downsampling to obtain the third sampling result;

[0101] Performing depth-wise separable downsampling convolution on the third sampling result to obtain the fourth sampling result;

[0102] Among them, similar to the detail branch in the teacher network, the student model uses the expanded channel downsampling method to downsample the sample neck image to obtain the third sampling result, uses pixel-by-pixel convolution to amplify the output channel, and then uses depth-wise separable downsampling convolution to gradually downsample and convolve the feature map in the third sampling result to obtain the fourth sampling result. Each time the depth-wise separable convolution is performed, each output channel result is only obtained through convolution calculation from the corresponding channel, and non-corresponding channels do not participate in the calculation.

[0103] Step S14: performing feature fusion on the third sampling result and the fourth sampling result to obtain a second fused feature, and performing upsampling convolution on the second fused feature to obtain a second segmented image;

[0104] Optionally, in this step, performing feature fusion on the third sampling result and the fourth sampling result to obtain a second fused feature includes:

[0105] Upsampling the fourth sampling result, and performing feature fusion on the upsampled fourth sampling result and the third sampling result to obtain the second fused feature;

[0106] Among them, the depth-separable downsampling result is upsampled and fused with the expanded channel downsampling result on the channel, and then gradually upsampled and convolved back to the input image size to obtain the semantic segmentation result.

[0107] In this embodiment, by inputting the sample neck image into the teacher model for downsampling, a first sampling result is obtained, and attention fusion is performed on the first sampling result to obtain a first fusion feature, which can effectively perform feature extraction on the sample neck image based on the teacher model, and by upsampling and convolving the first fusion feature with the first sampling result, the neck wrinkle detection result of the sample neck image by the teacher model is obtained, by inputting the sample neck image into the student model for downsampling, a third sampling result is obtained, and the third sampling result is downsampled, which can effectively perform feature extraction on the sample neck image based on the student model, by feature fusing the third sampling result and the fourth sampling result to obtain a second fusion feature, and by upsampling and convolving the second fusion feature, the neck wrinkle detection result of the sample neck image by the student model is obtained.

[0108] Example 3

[0109] See also Figure 4, is a schematic structural diagram of a neck wrinkle detection system 100 provided in a third embodiment of the present invention, comprising: a semantic segmentation module 10, a student model training module 11, an association relationship determination module 12, and a neck wrinkle detection module 13, wherein:

[0110] The semantic segmentation module 10 is used to input the sample neck image into the teacher model and the student model respectively for semantic segmentation to obtain a first segmented image and a second segmented image.

[0111] The semantic segmentation module 10 is further configured to: input the sample neck image into the teacher model for downsampling to obtain a first sampling result, and perform attention fusion on the first sampling result to obtain a first fusion feature;

[0112] Perform upsampling convolution on the first fusion feature and the first sampling result to obtain the first segmented image;

[0113] Inputting the sample neck image into the student model for downsampling to obtain a third sampling result, and downsampling the third sampling result to obtain a fourth sampling result;

[0114] The third sampling result and the fourth sampling result are subjected to feature fusion to obtain a second fused feature, and the second fused feature is subjected to upsampling convolution to obtain the second segmented image.

[0115] Optionally, the semantic segmentation module 10 is further configured to: input the sample neck image into the teacher model to perform expansion channel downsampling and downsampling convolution respectively, to obtain a first sampling feature and a second sampling feature;

[0116] Down-sampling convolution is performed on the first sampling feature to obtain a third sampling feature, wherein the first sampling result includes the first sampling feature, the second sampling feature, and the third sampling feature.

[0117] Furthermore, the semantic segmentation module 10 is further configured to: superimpose the third sampling feature and the second sampling feature to obtain a first superimposed feature;

[0118] Performing upsampling convolution on the second sampling feature to obtain a fourth sampling feature, and superimposing the fourth sampling feature with the third sampling feature to obtain a second superimposed feature;

[0119] The first superimposed feature and the second superimposed feature are fused to obtain a third fused feature, and the third fused feature is convolved to obtain the first fused feature.

[0120] Furthermore, the semantic segmentation module 10 is further configured to: upsample the first fused feature, and perform feature fusion on the upsampled first fused feature and the first sampled feature to obtain a fourth fused feature;

[0121] Perform upsampling convolution on the fourth fusion feature to obtain the first segmented image.

[0122] Preferably, the semantic segmentation module 10 is further configured to: input the sample neck image into the student model for expanded channel downsampling to obtain the third sampling result;

[0123] Performing depth-wise separable downsampling convolution on the third sampling result to obtain the fourth sampling result.

[0124] Optionally, the semantic segmentation module 10 is further used to: upsample the fourth sampling result, and perform feature fusion on the upsampled fourth sampling result and the third sampling result to obtain the second fusion feature.

[0125] The student model training module 11 is used to calculate the loss of the student model based on the first segmented image and the second segmented image to obtain a model loss value, and update the parameters of the student model according to the model loss value until the student model converges.

[0126] The association relationship determination module 12 is configured to sample the video to be processed to obtain at least one sampled image, and determine an image association relationship between each sampled image and the remaining images in the video to be processed.

[0127] The association relationship determination module 12 is further configured to: perform frame-by-frame sampling on the video to be processed to obtain at least one sampled image;

[0128] The relationship between each sampled image and the remaining images of the corresponding next frame is determined as the image association relationship.

[0129] The neck wrinkle detection module 13 is used to input each sampling image into the converged student model to perform neck wrinkle detection, obtain the neck wrinkle image of each sampling image, and set the neck wrinkle image of each sampling image as the neck wrinkle image of the corresponding remaining image according to the image association relationship.

[0130] The neck wrinkle detection module 13 is further configured to: determine the neck region position displacement between the sampled image and the remaining image of the corresponding next frame in the image association relationship;

[0131] For each image association relationship, according to the position displacement of the neck region, the neck wrinkle image of the sampled image is overlaid onto the remaining image of the corresponding next frame to obtain the neck wrinkle image of each remaining image.

[0132] Furthermore, the neck wrinkle detection module 13 is further configured to perform mask processing on the corresponding sampled images and the remaining images in the video to be processed according to the image position of each neck wrinkle image, so as to obtain a neck wrinkle output video.

[0133] In this embodiment, the knowledge distillation technology is used to train the student model, so that the converged student model reduces the computational complexity of the model while ensuring the accuracy of neck wrinkle detection for each sampled image, thereby improving the efficiency of neck wrinkle detection for each sampled image. By determining the image association relationship between each sampled image and the remaining images in the video to be processed, and setting the neck wrinkle image of each sampled image to the neck wrinkle image of the corresponding remaining image according to the image association relationship, it is avoided that each frame of the image is subjected to neck wrinkle detection using the converged student model, thereby improving the efficiency of neck wrinkle detection for images in the video to be processed, and improving the coherence between the sampled image and the corresponding remaining image. By inputting each sampled image into the converged student model for neck wrinkle detection, and setting the neck wrinkle image of each sampled image to the neck wrinkle image of the corresponding remaining image according to the image association relationship, it is unnecessary for the user to manually select the neck wrinkle image, thereby improving the accuracy of neck wrinkle detection.

[0134] Example 4

[0135] Figure 5 This is a structural block diagram of a terminal device 2 provided in the fourth embodiment of the present application. Figure 5 As shown, the terminal device 2 of this embodiment includes: a processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the processor 20, such as a program for the neck wrinkle detection method. When the processor 20 executes the computer program 22, the steps in each embodiment of the above-mentioned neck wrinkle detection method are implemented, such as Figure 1 S10 to S40 as shown, or Figure 2 Alternatively, the processor 20 executes the computer program 22 to implement the above Figure 4 The functions of each unit in the corresponding embodiment are, for example, Figure 4 For details on the functions of units 10 to 13, please refer to Figure 4 The relevant descriptions in the corresponding embodiments are not repeated here.

[0136] Exemplarily, the computer program 22 can be divided into one or more units, which are stored in the memory 21 and executed by the processor 20 to complete the present application. The one or more units can be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 22 in the terminal device 2. For example, the computer program 22 can be divided into a semantic segmentation module 10, a student model training module 11, an association relationship determination module 12, and a neck wrinkle detection module 13, and the specific functions of each unit are as described above.

[0137] The terminal device may include, but is not limited to, a processor 20 and a memory 21. Those skilled in the art will understand that Figure 4 It is only an example of the terminal device 2 and does not constitute a limitation on the terminal device 2. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0138] The processor 20 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0139] The memory 21 may be an internal storage unit of the terminal device 2, such as a hard disk or memory of the terminal device 2. The memory 21 may also be an external storage device of the terminal device 2, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 2. Furthermore, the memory 21 may include both an internal storage unit of the terminal device 2 and an external storage device. The memory 21 is used to store the computer program and other programs and data required by the terminal device. The memory 21 may also be used to temporarily store data that has been output or is about to be output.

[0140] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0141] If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium can be non-volatile or volatile. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device that can carry computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in computer-readable storage media can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable storage media do not include electrical carrier signals and telecommunications signals.

[0142] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A neck wrinkle detection method, characterized in that: The method comprises: The sample neck image is input into the teacher model and the student model for semantic segmentation respectively, and the first segmentation image and the second segmentation image are obtained; Calculating the loss of the student model according to the first segmented image and the second segmented image to obtain a model loss value, and updating parameters of the student model according to the model loss value until the student model converges; Sampling the video to be processed to obtain at least one sampled image, and determining an image association relationship between each sampled image and the remaining images in the video to be processed; Inputting each sampling image into the converged student model to perform neck wrinkle detection to obtain a neck wrinkle image of each sampling image, and setting the neck wrinkle image of each sampling image as the neck wrinkle image of the corresponding remaining image according to the image association relationship; Determining an image association relationship between each sampled image and the remaining images in the video to be processed includes: Determine the relationship between each sampled image and the remaining image of the corresponding next frame as the image association relationship; The step of setting the neck wrinkle image of each sampled image as the neck wrinkle image of the corresponding remaining image according to the image association relationship includes: respectively determining, in each image association relationship, a position displacement of the neck region between the sampled image and a corresponding remaining image of the next frame; For each image association relationship, according to the position displacement of the neck area, the neck wrinkle image of the sampled image is overlaid onto the remaining image of the corresponding next frame to obtain the neck wrinkle image of each remaining image; According to the image position of each neck wrinkle image, mask processing is performed on the corresponding sampled image and the remaining images in the video to be processed to obtain a neck wrinkle output video.

2. The neck wrinkle detection method according to claim 1, wherein: The sample neck image is input into the teacher model and the student model respectively for semantic segmentation to obtain a first segmented image and a second segmented image, including: Inputting the sample neck image into the teacher model for downsampling to obtain a first sampling result, and performing attention fusion on the first sampling result to obtain a first fusion feature; Perform upsampling convolution on the first fusion feature and the first sampling result to obtain the first segmented image; Inputting the sample neck image into the student model for downsampling to obtain a third sampling result, and downsampling the third sampling result to obtain a fourth sampling result; The third sampling result and the fourth sampling result are subjected to feature fusion to obtain a second fused feature, and the second fused feature is subjected to upsampling convolution to obtain the second segmented image.

3. The neck wrinkle detection method according to claim 2, wherein: The step of inputting the sample neck image into the teacher model for downsampling to obtain a first sampling result includes: Inputting the sample neck image into the teacher model for performing expanded channel downsampling and downsampling convolution respectively to obtain a first sampling feature and a second sampling feature; Down-sampling convolution is performed on the first sampling feature to obtain a third sampling feature, wherein the first sampling result includes the first sampling feature, the second sampling feature, and the third sampling feature.

4. The neck wrinkle detection method according to claim 3, wherein: The performing attention fusion on the first sampling result to obtain a first fusion feature includes: Superimposing the third sampling feature and the second sampling feature to obtain a first superimposed feature; Performing upsampling convolution on the second sampling feature to obtain a fourth sampling feature, and superimposing the fourth sampling feature with the third sampling feature to obtain a second superimposed feature; The first superimposed feature and the second superimposed feature are fused to obtain a third fused feature, and the third fused feature is convolved to obtain the first fused feature.

5. The neck wrinkle detection method according to claim 3, wherein: The step of performing up-sampling convolution on the first fusion feature and the first sampling result to obtain the first segmented image includes: Upsampling the first fused feature, and fusing the upsampled first fused feature with the first sampled feature to obtain a fourth fused feature; Perform upsampling convolution on the fourth fusion feature to obtain the first segmented image.

6. The neck wrinkle detection method according to claim 2, wherein: The step of inputting the sample neck image into the student model for downsampling to obtain a third sampling result, and downsampling the third sampling result to obtain a fourth sampling result comprises: Inputting the sample neck image into the student model for expanded channel downsampling to obtain the third sampling result; Performing depth-wise separable downsampling convolution on the third sampling result to obtain the fourth sampling result.

7. The neck wrinkle detection method according to claim 6, wherein: The step of fusing the third sampling result and the fourth sampling result to obtain a second fused feature includes: The fourth sampling result is upsampled, and feature fusion is performed on the upsampled fourth sampling result and the third sampling result to obtain the second fused feature.

8. A neck wrinkle detection system, characterized in that: The system comprises: A semantic segmentation module is used to input the sample neck image into the teacher model and the student model respectively for semantic segmentation to obtain a first segmented image and a second segmented image; a student model training module, configured to calculate a loss of the student model according to the first segmented image and the second segmented image to obtain a model loss value, and update parameters of the student model according to the model loss value until the student model converges; an association relationship determination module, configured to sample the video to be processed to obtain at least one sampled image, and determine an image association relationship between each sampled image and the remaining images in the video to be processed; a neck wrinkle detection module, configured to input each sampled image into the converged student model to perform neck wrinkle detection, obtain a neck wrinkle image of each sampled image, and set the neck wrinkle image of each sampled image as the neck wrinkle image of the corresponding remaining image based on the image association relationship; The association relationship determination module is further configured to: determine the relationship between each sampled image and the remaining images of the corresponding next frame as the image association relationship; The neck wrinkle detection module is further configured to: determine, in each image association relationship, a position displacement of the neck region between the sampled image and the remaining image of the corresponding next frame; For each image association relationship, according to the position displacement of the neck area, the neck wrinkle image of the sampled image is overlaid onto the remaining image of the corresponding next frame to obtain the neck wrinkle image of each remaining image; According to the image position of each neck wrinkle image, mask processing is performed on the corresponding sampled image and the remaining images in the video to be processed to obtain a neck wrinkle output video.

9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images