Image processing method, image processing device, and electronic equipment

Abstract

The image processing method, image processing device, and electronic device provided by the present application relate to the field of image processing technology. In the present application, first, multiple frames of sample texture images are obtained, wherein the image label information corresponding to the sample texture images includes the feature point coordinates of the texture feature points in the sample texture images; secondly, for each frame of sample texture images, the image label information corresponding to the sample texture images is updated based on the image label information corresponding to the standard texture image to obtain image label update information, wherein the image label information corresponding to the standard texture image includes the feature point coordinates of the texture feature points in the standard texture image; then, based on the multiple frames of sample texture images and the corresponding image label update information, a neural network model is trained to obtain a texture feature point detection model. Based on the above method, the problem of poor detection accuracy of the trained model in the prior art can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, an image processing device, and an electronic device. Background Art

[0002] Image modeling (e.g., user image modeling) is a typical application in image processing technology. For example, a similar cartoon image can be constructed based on a user's image. A key foundation for user image modeling is processing the user's texture image, such as detecting feature points within the texture image. However, the inventors have discovered that the trained detection models in existing technologies suffer from poor accuracy in detecting texture feature points. Summary of the Invention

[0003] In view of this, the purpose of the present application is to provide an image processing method, an image processing device and an electronic device to improve the problem of poor detection accuracy of the texture feature point detection model trained in the prior art.

[0004] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0005] An image processing method, comprising:

[0006] Obtaining multiple frames of sample texture images, wherein the image label information corresponding to each frame of the sample texture image includes the feature point coordinates of each texture feature point in the sample texture image;

[0007] For each frame of the sample texture image, updating the image label information corresponding to the sample texture image based on predetermined image label information corresponding to a standard texture image to obtain image label update information corresponding to the sample texture image, wherein the image label information corresponding to the standard texture image includes feature point coordinates of each texture feature point in the standard texture image;

[0008] Based on the multiple frames of sample texture images and the image label update information corresponding to each frame of the sample texture image, a pre-constructed neural network model is trained to obtain a texture feature point detection model corresponding to the neural network model, wherein the texture feature point detection model is used to perform texture feature point detection on the target texture image to obtain the feature point coordinates of each texture feature point.

[0009] In a preferred embodiment of the present application, in the above-mentioned image processing method, the step of updating the image label information corresponding to the sample texture image based on the predetermined image label information corresponding to the standard texture image for each frame of the sample texture image to obtain the image label update information corresponding to the sample texture image includes:

[0010] Determining a standard texture image, wherein the feature point coordinates of each texture feature point in the standard texture image serve as image label information corresponding to the standard texture image;

[0011] For each texture feature point in the sample texture image of each frame, calculating the difference between the feature point coordinates of the texture feature point in the sample texture image and the feature point coordinates of the texture feature point in the standard texture image, to obtain the feature point difference coordinates of the texture feature point;

[0012] For each frame of the sample texture image, the feature point difference coordinates of each texture feature point in the sample texture image are determined as the image label update information corresponding to the sample texture image.

[0013] In a preferred embodiment of the present application, in the above-mentioned image processing method, the step of determining the standard texture image includes:

[0014] Determine the feature point coordinates of each texture feature point in each frame of the reference texture image obtained;

[0015] For each of the texture feature points, calculating an average value of feature point coordinates of the texture feature point in each frame of the reference texture image to obtain an average feature point coordinate of the texture feature point;

[0016] A standard texture image is obtained based on the average coordinates of the feature points of each texture feature point.

[0017] In a preferred embodiment of the present application, in the above-mentioned image processing method, the step of obtaining multiple frames of sample texture images includes:

[0018] For each frame of the obtained original texture image, perform texture feature point detection on the original texture image based on a pre-trained first texture feature point detection model to obtain image label information corresponding to the original texture image, wherein the detection accuracy of the first texture feature point detection model is lower than the detection accuracy of the texture feature point detection model;

[0019] For each frame of the original texture image, image label comparison processing is performed based on image label information corresponding to the original texture image and image label information corresponding to a predetermined standard texture image to determine whether to use the original texture image as a sample texture image.

[0020] In a preferred embodiment of the present application, in the above-mentioned image processing method, the step of performing image label comparison processing on each frame of the original texture image based on the image label information corresponding to the original texture image and the image label information corresponding to the predetermined standard texture image to determine whether to use the original texture image as a sample texture image includes:

[0021] For each texture feature point in the original texture image of each frame, calculating a difference between a feature point coordinate of the texture feature point in the original texture image and a feature point coordinate of the texture feature point in a predetermined standard texture image, to obtain the original difference coordinate of the texture feature point, wherein the image label information corresponding to the original texture image includes the feature point coordinates of each texture feature point in the original texture image;

[0022] For each frame of the original texture image, it is determined whether to use the original texture image as a sample texture image based on the original difference coordinates of each texture feature point in the original texture image.

[0023] In a preferred embodiment of the present application, in the above-mentioned image processing method, the step of training a pre-built neural network model based on the multiple frames of sample texture images and the image label update information corresponding to each frame of the sample texture image to obtain a texture feature point detection model corresponding to the neural network model includes:

[0024] For each frame of the multiple frames of sample texture images, perform texture feature point detection on the sample texture image based on a pre-built neural network model to obtain image label information to be determined corresponding to the sample texture image, and determine a loss value corresponding to the sample texture image based on the image label information to be determined and image label update information corresponding to the sample texture image;

[0025] For each frame of the multiple frames of sample texture images, based on the image label update information corresponding to the sample texture image, a weighting coefficient corresponding to the sample texture image is determined, and a loss value corresponding to the sample texture image is updated based on the weighting coefficient to obtain a target loss value corresponding to the sample texture image;

[0026] The network parameters of the neural network model are optimized based on the target loss value corresponding to each frame of the multiple frames of sample texture images, until the current target loss value meets the set training convergence condition, thereby obtaining a texture feature point detection model.

[0027] In a preferred embodiment of the present application, in the above-mentioned image processing method, the sample texture image includes a face sample texture image, and the texture feature points include face texture feature points.

[0028] In a preferred embodiment of the present application, in the above-mentioned image processing method, the image processing method further includes:

[0029] Based on the texture feature point detection model, texture feature point detection is performed on the obtained target texture image to obtain feature point difference coordinates of each texture feature point in the target texture image;

[0030] For each texture feature point in the target texture image, based on the feature point coordinates of the texture feature point in the predetermined standard texture image, the feature point difference coordinates of the texture feature point are updated to obtain the feature point coordinates of the texture feature point in the target texture image.

[0031] The present application also provides an image processing device, including:

[0032] A texture image acquisition module, configured to acquire multiple frames of sample texture images, wherein the image label information corresponding to each frame of the sample texture image includes the feature point coordinates of each texture feature point in the sample texture image;

[0033] an image label updating module, configured to update, for each frame of the sample texture image, the image label information corresponding to the sample texture image based on the predetermined image label information corresponding to the standard texture image, to obtain the image label update information corresponding to the sample texture image, wherein the image label information corresponding to the standard texture image includes the feature point coordinates of each texture feature point in the standard texture image;

[0034] The network model training module is used to train a pre-built neural network model based on the multiple frames of sample texture images and the image label update information corresponding to each frame of the sample texture image, and obtain a texture feature point detection model corresponding to the neural network model, wherein the texture feature point detection model is used to perform texture feature point detection on the target texture image to obtain the feature point coordinates of each texture feature point.

[0035] Based on the above, an embodiment of the present application further provides an electronic device, including:

[0036] memory for storing computer programs;

[0037] The processor connected to the memory is used to execute the computer program stored in the memory to implement the above-mentioned image processing method.

[0038] The image processing method, image processing device, and electronic device provided by the present application can first update the image label information corresponding to the sample texture image based on the image label information corresponding to the predetermined standard texture image to obtain the image label update information corresponding to the sample texture image, and then train the pre-built neural network model based on the image label update information to obtain the corresponding texture feature point detection model. Based on this, since the image label information corresponding to the sample texture image is first updated based on the image label information corresponding to the standard texture image before training the neural network model, the similarity between the image label information corresponding to the sample texture images is likely to be less than the similarity between the image label information corresponding to the sample texture images, thereby improving to a certain extent the problem of overfitting that is prone to occur during the training model due to the large similarity between the image label information corresponding to the sample texture images, and further improving the problem of poor detection accuracy of the texture feature point detection model obtained by training in the prior art due to overfitting.

[0039] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a structural block diagram of an electronic device provided in an embodiment of the present application.

[0041] Figure 2 A flowchart illustrating the steps included in the image processing method provided in an embodiment of the present application.

[0042] Figure 3 A regular face mesh.

[0043] Figure 4 It is a conventional face texture map.

[0044] Figure 5 for Figure 2 Schematic diagram of the flow of sub-steps included in step S110.

[0045] Figure 6 for Figure 5 Schematic diagram of the flow of sub-steps included in step S112.

[0046] Figure 7 for Figure 2 Schematic diagram of the flow of sub-steps included in step S120.

[0047] Figure 8 for Figure 7 Schematic diagram of the flow of sub-steps included in step S121.

[0048] Figure 9for Figure 2 Schematic diagram of the flow of sub-steps included in step S130.

[0049] Figure 10 This is a flowchart of other steps included in the image processing method provided in the embodiment of the present application.

[0050] Figure 11 A target face texture map provided in an embodiment of the present application.

[0051] Figure 12 A block diagram of an image processing device provided in an embodiment of the present application.

[0052] Icons: 10 - electronic device; 12 - memory; 14 - processor; 100 - image processing device; 110 - texture image acquisition module; 120 - image label update module; 130 - network model training module. DETAILED DESCRIPTION

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0054] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0055] like Figure 1 As shown, an embodiment of the present application provides an electronic device 10 , which may include a memory 12 , a processor 14 , and an image processing device 100 .

[0056] The memory 12 and the processor 14 are electrically connected directly or indirectly to achieve data transmission or interaction. For example, they can be electrically connected to each other through one or more communication buses or signal lines. The image processing device 100 includes at least one software function module that can be stored in the memory 12 in the form of software or firmware. The processor 14 is used to execute the executable computer program stored in the memory 12, for example, the software function module and computer program included in the image processing device 100, to implement the image processing method provided in the embodiment of the present application.

[0057] Optionally, the memory 12 can be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable read-only memory (EEPROM), etc.

[0058] Optionally, the processor 14 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0059] I understand. Figure 1 The structure shown is for illustration only. The electronic device 10 may also include Figure 1 More or fewer components than shown, or with Figure 1 The different configurations shown, for example, may also include a communication unit for exchanging information with other devices (such as mobile phones and other terminal devices).

[0060] Combine Figure 2 The embodiment of the present application also provides an image processing method applicable to the electronic device 10. The method steps defined in the process related to the image processing method can be implemented by the electronic device 10. Figure 2 The specific process shown is explained in detail.

[0061] Step S110: obtaining multiple frames of sample texture images.

[0062] In the embodiment of the present application, the electronic device 10 may first obtain multiple frames of sample texture images, wherein the image label information corresponding to each frame of the sample texture image includes the feature point coordinates of each texture feature point in the sample texture image.

[0063] Step S120 , for each frame of the sample texture image, updating the image label information corresponding to the sample texture image based on the predetermined image label information corresponding to the standard texture image, to obtain the image label update information corresponding to the sample texture image.

[0064] In an embodiment of the present application, after obtaining the multiple frames of sample texture images based on step S110, the electronic device 10 may update the image label information corresponding to each frame of the sample texture image based on the predetermined image label information corresponding to the standard texture image, thereby obtaining updated image label information corresponding to the sample texture image. The image label information corresponding to the standard texture image includes the feature point coordinates of each texture feature point in the standard texture image.

[0065] Step S130 , training a pre-built neural network model based on the multiple frames of sample texture images and image label update information corresponding to each frame of the sample texture image, to obtain a texture feature point detection model corresponding to the neural network model.

[0066] In an embodiment of the present application, after obtaining the image label update information based on step S120, the electronic device 10 can train a pre-built neural network model based on the multiple frames of sample texture images and the image label update information corresponding to each frame of the sample texture image to obtain a texture feature point detection model corresponding to the neural network model. The texture feature point detection model is used to detect texture feature points of the target texture image and obtain the feature point coordinates of each texture feature point (such as each texture feature point in the target texture image).

[0067] Based on this, since the image label information corresponding to the sample texture image is updated based on the image label information corresponding to the standard texture image before training the neural network model, the similarity between the image label information corresponding to the sample texture images is likely to be smaller than the similarity between the image label information corresponding to the sample texture images, thereby improving to a certain extent the problem of overfitting that is prone to occur in the training model process due to the large similarity between the image label information corresponding to the sample texture images, and further improving the problem of poor detection accuracy caused by overfitting of the texture feature point detection model obtained by training in the prior art.

[0068] First, it should be noted that the specific method of obtaining the multiple frames of sample texture images is not limited and can be selected according to actual application requirements.

[0069] For example, in an alternative example, each frame of the obtained multiple frames of original texture images can be directly used as a sample texture image to obtain multiple frames of sample texture images. The multiple frames of original texture images can be obtained by directly obtaining multiple frames of original texture images from other devices or databases, or by processing the obtained multiple frames of sample images to obtain corresponding multiple frames of original texture images. For example, for each frame of sample image, the sample image can be reconstructed, such as face reconstruction, to obtain a face mesh map (such as a face mesh map) corresponding to the sample image. Figure 3 Then, the texture of the face grid image is sampled to obtain the original texture image corresponding to the sample image (as shown in Figure 4 The facial texture image shown).

[0070] It is understood that in the above example, the sample texture image may include a human face sample texture image, and correspondingly, the texture feature points may include human face texture feature points. In other examples, the sample texture image may also be other types of texture images, such as texture images of animals such as dogs and cats, or texture images of non-animals.

[0071] For example, in another alternative example, in order to improve the accuracy of training the neural network model, the original texture image obtained can be screened and combined with Figure 5 , step S110 may further include step S111 and step S112, the specific contents of which are described as follows.

[0072] In step S111 , for each frame of the original texture image obtained, texture feature point detection is performed on the original texture image based on a pre-trained first texture feature point detection model to obtain image label information corresponding to the original texture image.

[0073] In an embodiment of the present application, after obtaining multiple frames of original texture images (the specific acquisition method can be referred to the relevant description above), texture feature point detection can be performed on each frame of the original texture image based on a pre-trained first texture feature point detection model to obtain image label information corresponding to the original texture image. The detection accuracy of the first texture feature point detection model is lower than the detection accuracy of the texture feature point detection model. For example, the first texture feature point detection model can be trained based on some existing conventional techniques, and the specific training method is not described in detail here.

[0074] Step S112 : for each frame of the original texture image, performing image label comparison processing based on image label information corresponding to the original texture image and image label information corresponding to a predetermined standard texture image, to determine whether to use the original texture image as a sample texture image.

[0075] In the embodiment of the present application, after obtaining the image label information corresponding to the original texture image in step S111, an image label comparison process can be performed for each frame of the original texture image based on the image label information corresponding to the original texture image and the image label information corresponding to a predetermined standard texture image to determine whether to use the original texture image as a sample texture image. The method for determining the standard texture image can be referred to in the relevant description below.

[0076] Optionally, the specific method of determining whether to use the original texture image as the sample texture image in step S112 is not limited and can be selected according to actual application requirements. For example, in an alternative example, Figure 6 , step S112 may further include step S112a and step S122b, the specific contents of which are described as follows.

[0077] Step S112a, for each texture feature point in the original texture image of each frame, calculate the difference between the feature point coordinates of the texture feature point in the original texture image and the feature point coordinates of the texture feature point in the predetermined standard texture image to obtain the original difference coordinates of the texture feature point.

[0078] In an embodiment of the present application, after obtaining the image label information corresponding to the original texture image in step S111, the difference between the feature point coordinates of each texture feature point in the original texture image and the feature point coordinates of the texture feature point in a predetermined standard texture image can be calculated for each texture feature point in each frame of the original texture image to obtain the original difference coordinates of the texture feature point. The image label information corresponding to the original texture image includes the feature point coordinates of each texture feature point in the original texture image.

[0079] Step S122b: for each frame of the original texture image, determine whether to use the original texture image as a sample texture image based on the original difference coordinates of each texture feature point in the original texture image.

[0080] In an embodiment of the present application, after obtaining the original differential coordinates of the texture feature points based on step S112a, a determination can be made for each frame of the original texture image whether to use the original texture image as a sample texture image based on the original differential coordinates of each texture feature point in the original texture image. For example, in order to use an original texture image that is significantly different from the standard texture image as a sample texture image, the original texture image that has a significantly different coordinate between the corresponding feature point and the feature point coordinates in the standard texture image can be determined as the sample texture image based on the original differential coordinates of each texture feature point.

[0081] It can be understood that in a specific application example, for each texture feature point, the vector distance of the original difference coordinates of the texture feature point can be calculated (that is, the coordinate distance between the feature point coordinates of the texture feature point in the original texture image and the feature point coordinates in the standard texture image), and then the vector distance corresponding to each texture feature point is fused, such as calculating the sum or average value of the vector distances corresponding to each texture feature point, and then determining the relative size relationship between the sum or average value and a pre-configured distance threshold (normalization can also be performed first, and then the relative size relationship is determined). In this way, when the sum or average value is greater than or equal to the distance threshold, the corresponding original texture image can be used as a sample texture image.

[0082] Secondly, it should be noted that for step S120 , the specific method of updating the image label information corresponding to the sample texture image based on the image label information corresponding to the standard texture image is not limited and can be selected according to actual application requirements.

[0083] For example, in an alternative example, in order to reduce the similarity between the updated image label information, the similar parts between the image label information corresponding to each sample texture image can be removed based on the image label information corresponding to the standard texture image, and combined with Figure 7 Step S120 may include step S121, step S122 and step S123, the specific contents of which are described as follows.

[0084] Step S121: Determine a standard texture image.

[0085] In the embodiment of the present application, a standard texture image may be determined first, wherein the feature point coordinates of each texture feature point in the standard texture image may be used as image label information corresponding to the standard texture image.

[0086] Step S122 , for each texture feature point in the sample texture image of each frame, calculate the difference between the feature point coordinates of the texture feature point in the sample texture image and the feature point coordinates of the texture feature point in the standard texture image to obtain the feature point difference coordinates of the texture feature point.

[0087] In an embodiment of the present application, after the standard texture image is determined based on step S121, the difference between the feature point coordinates of the texture feature point in the sample texture image and the feature point coordinates of the texture feature point in the standard texture image can be calculated for each texture feature point in each frame of the sample texture image to obtain the feature point difference coordinates of the texture feature point.

[0088] Step S123 : for each frame of the sample texture image, determining the feature point difference coordinates of each texture feature point in the sample texture image as image label update information corresponding to the sample texture image.

[0089] In an embodiment of the present application, after obtaining the feature point difference coordinates of each of the texture feature points based on step S122, the feature point difference coordinates of each texture feature point in the sample texture image can be determined as the image label update information corresponding to the sample texture image for each frame of the sample texture image.

[0090] Optionally, in step S121 , the specific method of determining the standard texture image is not limited and can be selected according to actual application requirements.

[0091] For example, in an alternative example, any frame of corresponding texture image obtained can be used as a standard texture image. For example, when the sample texture image is a face sample texture image, any frame of face texture image can be used as a standard texture image.

[0092] For another example, in order to make the image label information corresponding to the standard texture image be able to largely remove the similar parts between the image label information corresponding to each sample texture image, so that the similarity between the updated image label information is small, thereby avoiding the problem of overfitting in the process of training the model, Figure 8 Step S121 may include step S121a, step S121b and step S121c, the specific contents of which are described as follows.

[0093] Step S121a: determining the feature point coordinates of each texture feature point in each frame of the reference texture image obtained.

[0094] In an embodiment of the present application, after obtaining multiple frames of reference texture images, the feature point coordinates of each texture feature point in each frame of the reference texture image can be determined. The multiple frames of reference texture images can be all or part of the sample texture images in the multiple frames of sample texture images, or other texture images, such as corresponding multiple frames of reference texture images obtained by texture sampling based on any multiple frames of images.

[0095] Step S121b: for each of the texture feature points, calculate the average value of the feature point coordinates of the texture feature point in each frame of the reference texture image to obtain the feature point average coordinates of the texture feature point.

[0096] In an embodiment of the present application, after obtaining the feature point coordinates of each texture feature point in each frame of the reference texture image based on step S121a, the average value of the feature point coordinates of the texture feature point in each frame of the reference texture image can be calculated for each of the texture feature points to obtain the average feature point coordinates of the texture feature point.

[0097] Step S121c: obtaining a standard texture image based on the average coordinates of each texture feature point.

[0098] In the embodiment of the present application, after obtaining the feature point average coordinates of each texture feature point based on step S121, a standard texture image can be obtained based on the feature point average coordinates of each texture feature point. The feature point average coordinates of each texture feature point can be used as image tag information of the standard texture image. Thus, when the standard texture image is a facial texture image, the standard texture image can be understood as a facial texture image with an average face. Thus, during the update process, similar portions between the sample texture images can be removed by calculating the coordinate difference between the facial texture image and the average face.

[0099] Thirdly, it should be noted that for step S130, the specific method of training the neural network model is not limited and can be selected according to actual application requirements.

[0100] For example, in an alternative example, texture feature point detection can be performed on a sample texture image based on a pre-built neural network model to obtain the to-be-determined image label information corresponding to the sample texture image. Based on the to-be-determined image label information and the image label update information corresponding to the sample texture image, a loss value corresponding to the sample texture image can be determined. Then, the sum of the loss values ​​corresponding to each sample texture image can be calculated to obtain a corresponding total loss value. The network parameters of the neural network model can then be optimized based on this total loss value until the current target loss value meets the set training convergence condition, thereby obtaining a texture feature point detection model.

[0101] For example, in another alternative example, considering that the similarity between the obtained sample texture images is high, after the update process, the obtained image label update information still has a high reliability of similarity. Based on this, in order to further improve the problem of overfitting in the model training process caused by the high similarity between the image label update information, combined with Figure 9 , step S130 may further include the following steps S131, S132 and S133, the specific contents of which are described as follows.

[0102] Step S131: For each frame of the multiple frames of sample texture images, texture feature point detection is performed on the sample texture image based on a pre-built neural network model to obtain the image label information to be determined corresponding to the sample texture image, and based on the image label information to be determined and the image label update information corresponding to the sample texture image, the loss value corresponding to the sample texture image is determined.

[0103] In an embodiment of the present application, after obtaining the image label update information corresponding to the sample texture image based on step S120, texture feature point detection can be performed on each frame of the multiple frames of sample texture images based on a pre-built neural network model to obtain the image label information to be determined corresponding to the sample texture image, and based on the image label information to be determined and the image label update information corresponding to the sample texture image, the loss value corresponding to the sample texture image can be determined.

[0104] In step S132, for each frame of the sample texture image in the multiple frames of the sample texture image, based on the image label update information corresponding to the sample texture image, a weighting coefficient corresponding to the sample texture image is determined, and based on the weighting coefficient, a loss value corresponding to the sample texture image is updated to obtain a target loss value corresponding to the sample texture image.

[0105] In an embodiment of the present application, after obtaining the loss value corresponding to the sample texture image based on step S131, the weighting coefficient corresponding to each frame of the sample texture image in the multiple frames of the sample texture image can be determined based on the image label update information corresponding to the sample texture image, and the loss value corresponding to the sample texture image can be updated based on the weighting coefficient to obtain the target loss value corresponding to the sample texture image.

[0106] Step S133, optimizing the network parameters of the neural network model based on the target loss value corresponding to each frame of the multiple frames of sample texture images until the current target loss value meets the set training convergence condition, thereby obtaining a texture feature point detection model.

[0107] In an embodiment of the present application, after obtaining the target loss value corresponding to the sample texture image based on step S132, the network parameters of the neural network model can be optimized based on the target loss value corresponding to each frame of the sample texture image in the multiple frames of sample texture images until the current target loss value meets the set training convergence conditions, thereby obtaining a texture feature point detection model.

[0108] Optionally, in step S131, the specific method for calculating the loss value corresponding to the sample texture image is not limited and can be selected according to actual application requirements. For example, in an alternative example, the loss function for calculating the loss value can be an L2loss function. In an alternative example, the neural network model can be shufflenetV2.

[0109] Optionally, in step S132, the specific method of determining the weighting coefficient corresponding to the sample texture image is not limited and can be selected according to actual application requirements. For example, in an alternative example, for each texture feature point, the vector distance of the feature point difference coordinates of the texture feature point can be calculated (that is, the coordinate distance between the feature point coordinates of the texture feature point in the sample texture image and the feature point coordinates in the standard texture image), and then the vector distance corresponding to each texture feature point is fused, such as calculating the sum or average of the vector distances corresponding to each texture feature point, and then normalizing the sum or average to obtain the weighting coefficient corresponding to the sample texture image. Finally, the product of the weighting coefficient and the loss value corresponding to the sample texture image can be calculated to obtain the target loss value corresponding to the sample texture image.

[0110] On the basis of the above embodiment, in an alternative example, combined with Figure 10 The image processing method may further include step S140 and step S150, the specific contents of which are described below.

[0111] Step S140 : performing texture feature point detection on the obtained target texture image based on the texture feature point detection model to obtain feature point difference coordinates of each texture feature point in the target texture image.

[0112] In the embodiment of the present application, after the texture feature point detection model is obtained through training based on step S130, the electronic device 10 can obtain the target texture image (such as Figure 11 As shown, the target face texture image) is subjected to texture feature point detection to obtain the feature point difference coordinates of each texture feature point in the target texture image.

[0113] Step S150 , for each texture feature point in the target texture image, based on the feature point coordinates of the texture feature point in the predetermined standard texture image, the feature point difference coordinates of the texture feature point are updated to obtain the feature point coordinates of the texture feature point in the target texture image.

[0114] In an embodiment of the present application, after obtaining the feature point difference coordinates of each texture feature point in the target texture image based on step S140, the feature point difference coordinates of each texture feature point in the target texture image can be updated based on the feature point coordinates of the texture feature point in the predetermined standard texture image to obtain the feature point coordinates of the texture feature point in the target texture image.

[0115] Optionally, based on the different updating methods performed in step S120, the updating methods performed in step S150 may also be different. For example, in the above embodiment, when the updating method in step S120 is to calculate coordinate differences, the updating method in step S150 may be to calculate coordinate sums. For example, for each texture feature point in the target texture image, the sum of the feature point coordinates of the texture feature point in the standard texture image and the feature point difference coordinates of the texture feature point is calculated to obtain the feature point coordinates of the texture feature point in the target texture image. In this way, step S150 can be understood as the reverse calculation of step S120.

[0116] Combine Figure 12 The embodiment of the present application further provides an image processing device 100 applicable to the electronic device 10 . The image processing device 100 may include a texture image acquisition module 110 , an image label update module 120 , and a network model training module 130 .

[0117] The texture image acquisition module 110 is used to obtain multiple frames of sample texture images, wherein the image label information corresponding to each frame of the sample texture image includes the feature point coordinates of each texture feature point in the sample texture image. In this embodiment, the texture image acquisition module 110 can be used to perform Figure 2 As shown in step S110 , for the relevant content of the texture image obtaining module 110 , reference may be made to the above description of step S110 .

[0118] The image label update module 120 is used to update the image label information corresponding to the sample texture image for each frame based on the image label information corresponding to the predetermined standard texture image, and obtain the image label update information corresponding to the sample texture image, wherein the image label information corresponding to the standard texture image includes the feature point coordinates of each texture feature point in the standard texture image. In this embodiment, the image label update module 120 can be used to perform Figure 2 Regarding step S120 shown, the relevant contents of the image tag updating module 120 may refer to the above description of step S120 .

[0119] The network model training module 130 is used to train the pre-built neural network model based on the multiple frames of sample texture images and the image label update information corresponding to each frame of the sample texture image, and obtain a texture feature point detection model corresponding to the neural network model, wherein the texture feature point detection model is used to detect texture feature points of the target texture image and obtain the feature point coordinates of each texture feature point. In this embodiment, the network model training module 130 can be used to perform Figure 2 As shown in step S130, for the relevant content of the network model training module 130, please refer to the description of step S130 above.

[0120] In an embodiment of the present application, corresponding to the above-mentioned image processing method applied to the electronic device 10, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is run, each step of the above-mentioned image processing method is executed.

[0121] The steps executed when the aforementioned computer program is running will not be described in detail here, and reference may be made to the above explanation of the image processing method.

[0122] In summary, the present application provides an image processing method, an image processing device, and an electronic device, which can first update the image label information corresponding to the sample texture image based on the image label information corresponding to the standard texture image to obtain the image label update information corresponding to the sample texture image, and then train the pre-built neural network model based on the image label update information to obtain the corresponding texture feature point detection model. Based on this, since the image label information corresponding to the sample texture image is first updated based on the image label information corresponding to the standard texture image before training the neural network model, the similarity between the image label information corresponding to the sample texture images is likely to be less than the similarity between the image label information corresponding to the sample texture images, thereby improving to a certain extent the problem of overfitting that is prone to occur in the process of training the model due to the large similarity between the image label information corresponding to the sample texture images, and further improving the problem of poor detection accuracy of the texture feature point detection model obtained by training in the prior art due to overfitting.

[0123] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0125] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or partly contributed to the prior art or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, electronic device, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. It should be noted that, in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without further constraints, an element defined by the phrase "comprises a..." does not preclude the existence of additional identical elements in the process, method, article or apparatus that includes the element.

[0126] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. An image processing method, characterized in that: include: Obtaining multiple frames of sample texture images, wherein the image label information corresponding to each frame of the sample texture image includes the feature point coordinates of each texture feature point in the sample texture image; Determine a standard texture image, wherein the feature point coordinates of each texture feature point in the standard texture image are used as image label information corresponding to the standard texture image; for each texture feature point in each frame of the sample texture image, calculate the difference between the feature point coordinates of the texture feature point in the sample texture image and the feature point coordinates of the texture feature point in the standard texture image to obtain feature point difference coordinates of the texture feature point; for each frame of the sample texture image, determine the feature point difference coordinates of each texture feature point in the sample texture image as image label update information corresponding to the sample texture image; Based on the multiple frames of sample texture images and the image label update information corresponding to each frame of the sample texture image, a pre-constructed neural network model is trained to obtain a texture feature point detection model corresponding to the neural network model, wherein the texture feature point detection model is used to perform texture feature point detection on the target texture image to obtain the feature point coordinates of each texture feature point.

2. The image processing method according to claim 1, wherein: The step of determining the standard texture image includes: Determine the feature point coordinates of each texture feature point in each frame of the reference texture image obtained; For each of the texture feature points, calculating an average value of feature point coordinates of the texture feature point in each frame of the reference texture image to obtain an average feature point coordinate of the texture feature point; A standard texture image is obtained based on the average coordinates of the feature points of each texture feature point.

3. The image processing method according to claim 1, wherein: The step of obtaining multiple frames of sample texture images includes: For each frame of the obtained original texture image, perform texture feature point detection on the original texture image based on a pre-trained first texture feature point detection model to obtain image label information corresponding to the original texture image, wherein the detection accuracy of the first texture feature point detection model is lower than the detection accuracy of the texture feature point detection model; For each frame of the original texture image, image label comparison processing is performed based on image label information corresponding to the original texture image and image label information corresponding to a predetermined standard texture image to determine whether to use the original texture image as a sample texture image.

4. The image processing method according to claim 3, wherein: The step of performing image label comparison processing on each frame of the original texture image based on image label information corresponding to the original texture image and image label information corresponding to a predetermined standard texture image to determine whether to use the original texture image as a sample texture image includes: For each texture feature point in the original texture image of each frame, calculating a difference between a feature point coordinate of the texture feature point in the original texture image and a feature point coordinate of the texture feature point in a predetermined standard texture image, to obtain the original difference coordinate of the texture feature point, wherein the image label information corresponding to the original texture image includes the feature point coordinates of each texture feature point in the original texture image; For each frame of the original texture image, it is determined whether to use the original texture image as a sample texture image based on the original difference coordinates of each texture feature point in the original texture image.

5. The image processing method according to claim 1, wherein: The step of training a pre-built neural network model based on the multiple frames of sample texture images and the image label update information corresponding to each frame of the sample texture image to obtain a texture feature point detection model corresponding to the neural network model includes: For each frame of the multiple frames of sample texture images, perform texture feature point detection on the sample texture image based on a pre-built neural network model to obtain image label information to be determined corresponding to the sample texture image, and determine a loss value corresponding to the sample texture image based on the image label information to be determined and image label update information corresponding to the sample texture image; For each frame of the multiple frames of sample texture images, based on the image label update information corresponding to the sample texture image, a weighting coefficient corresponding to the sample texture image is determined, and a loss value corresponding to the sample texture image is updated based on the weighting coefficient to obtain a target loss value corresponding to the sample texture image; The network parameters of the neural network model are optimized based on the target loss value corresponding to each frame of the multiple frames of sample texture images, until the current target loss value meets the set training convergence condition, thereby obtaining a texture feature point detection model.

6. The image processing method according to any one of claims 1 to 5, characterized in that: The sample texture image includes a human face sample texture image, and the texture feature points include human face texture feature points.

7. The image processing method according to any one of claims 1 to 5, characterized in that: The image processing method further includes: Based on the texture feature point detection model, texture feature point detection is performed on the obtained target texture image to obtain feature point difference coordinates of each texture feature point in the target texture image; For each texture feature point in the target texture image, based on the feature point coordinates of the texture feature point in the predetermined standard texture image, the feature point difference coordinates of the texture feature point are updated to obtain the feature point coordinates of the texture feature point in the target texture image.

8. An image processing device, characterized in that: include: A texture image acquisition module, configured to acquire multiple frames of sample texture images, wherein the image label information corresponding to each frame of the sample texture image includes the feature point coordinates of each texture feature point in the sample texture image; An image label update module is configured to determine a standard texture image, wherein the feature point coordinates of each texture feature point in the standard texture image serve as image label information corresponding to the standard texture image; for each texture feature point in each frame of the sample texture image, calculate the difference between the feature point coordinates of the texture feature point in the sample texture image and the feature point coordinates of the texture feature point in the standard texture image to obtain feature point difference coordinates of the texture feature point; for each frame of the sample texture image, determine the feature point difference coordinates of each texture feature point in the sample texture image as image label update information corresponding to the sample texture image; The network model training module is used to train a pre-built neural network model based on the multiple frames of sample texture images and the image label update information corresponding to each frame of the sample texture image, and obtain a texture feature point detection model corresponding to the neural network model, wherein the texture feature point detection model is used to perform texture feature point detection on the target texture image to obtain the feature point coordinates of each texture feature point.

9. An electronic device, characterized in that: include: memory for storing computer programs; A processor connected to the memory, configured to execute a computer program stored in the memory to implement the image processing method according to any one of claims 1 to 7.

Images