Image Processing Method, Model Training Method, Device, Medium and Equipment

Abstract

The present disclosure relates to an image processing method, a model training method, an apparatus, a medium, and a device. The method includes: obtaining image information to be processed; inputting the image information to be processed into a trained image processing model to obtain output key point information, where the key point information is used to determine the position of a target to be measured, and the image processing model includes a Bayesian convolutional layer, and the convolutional kernel in the Bayesian convolutional layer is in a distribution form. Through the above technical solution, the obtained image information to be processed is input into the trained image processing model to obtain the key point information output by the image processing model for determining the position of the target to be measured, where the image processing model includes a Bayesian convolutional layer with a convolutional kernel in a distribution form. Thus, regardless of whether the obtained image information to be processed is a clear image or a blurred image, the position of the target to be measured can be accurately determined based on the image information to be processed and the image processing model.

Description

Technical Field

[0001] The present disclosure relates to the field of machine learning, and in particular, to an image processing method, a model training method, an apparatus, a medium, and a device. Background Art

[0002] Augmented reality is generally based on object localization within an image. For a camera with a low frame rate, if the camera moves or shakes during the process of capturing an image, it will cause the captured image to be relatively blurred. In the prior art, it is not easy to localize an object in a relatively blurred image. In the scenario of using augmented reality, the user holds a device, and the camera is very likely to move or shake, resulting in a blurred image captured by the camera. In this way, it is not easy to localize the object in the image, and thus the augmented reality content cannot be smoothly presented. Summary of the Invention

[0003] The purpose of the present disclosure is to provide an image processing method, a model training method, an apparatus, a medium, and a device, which can accurately determine the position of a target to be measured in a blurred image.

[0004] To achieve the above purpose, the present disclosure provides an image processing method, including:

[0005] Obtain image information to be processed;

[0006] Input the image information to be processed into a trained image processing model to obtain output key point information, where the key point information is used to determine the position of a target to be measured, and the image processing model includes a Bayesian convolutional layer, and the convolutional kernel in the Bayesian convolutional layer is in a distribution form.

[0007] Optionally, the distribution form is a normal distribution form.

[0008] Optionally, in the Bayesian convolutional layer, the input of the Bayesian convolutional layer is subjected to a convolutional operation to obtain a convolutional operation result, and the convolutional operation result is subjected to a Gaussian mixture model algorithm operation to obtain the output of the Bayesian convolutional layer, where the convolutional operation result is in a distribution form, and the output is the mean of the Gaussian distribution with the largest corresponding probability among one or more Gaussian distributions obtained by subjecting the convolutional operation result to the Gaussian mixture model algorithm operation.

[0009] The present disclosure further provides a method for training an image processing model, the method including:

[0010] Obtain training samples;

[0011] Input the training samples into a motion blur model to obtain blurred training samples;

[0012] Input the blurred training samples into the initial image processing model to obtain training key point output information;

[0013] Determine the loss value of the initial image processing model according to the training key point output information and the true value of the key points;

[0014] Adjust the parameters of the initial image processing model according to the loss value to train the initial image processing model and obtain the image processing model;

[0015] Wherein, the initial image processing model and the image processing model include Bayesian convolutional layers, and the convolutional kernels in the Bayesian convolutional layers are in a distributed form.

[0016] Optionally, the method further includes:

[0017] Input the training samples into a control image processing model to obtain control key point output information, wherein the control image processing model includes non-Bayesian convolutional layers, and the convolutional kernels in the non-Bayesian convolutional layers are in a fixed form;

[0018] The determining the loss value of the initial image processing model according to the training key point output information and the true value of the key points includes:

[0019] Determine the loss value of the initial image processing model according to the training key point output information, the true value of the key points, and the control key point output information.

[0020] Optionally, the image processing model further includes a residual network.

[0021] The present disclosure also provides an image processing device, including:

[0022] A first acquisition module configured to acquire image information to be processed;

[0023] A first determination module configured to input the image information to be processed into the trained image processing model to obtain output key point information, wherein the key point information is used to determine the position of the target to be measured, and the image processing model includes Bayesian convolutional layers, and the convolutional kernels in the Bayesian convolutional layers are in a distributed form.

[0024] The present disclosure also provides a training device for an image processing model, including:

[0025] A second acquisition module configured to acquire training samples;

[0026] A second determination module configured to input the training samples into a motion blur model to obtain blurred training samples;

[0027] A third determination module, configured to input the fuzzy training samples into an initial image processing model to obtain training key point output information;

[0028] A fourth determination module, configured to determine a loss value of the initial image processing model according to the training key point output information and the true value of the key points;

[0029] A fifth determination module, configured to adjust parameters of the initial image processing model according to the loss value to train the initial image processing model to obtain the image processing model;

[0030] Wherein, the initial image processing model and the image processing model include a Bayesian convolutional layer, and the convolutional kernel in the Bayesian convolutional layer is in a distribution form.

[0031] The present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the above image processing method are implemented, or when the program instructions are executed by a processor, the steps of the above training method of the image processing model are implemented.

[0032] The present disclosure also provides an electronic device, including:

[0033] A memory, on which a computer program is stored;

[0034] A processor, configured to execute the computer program in the memory to implement the steps of the above image processing method or the training method of the image processing model.

[0035] Through the above technical solution, the obtained image information to be processed is input into the trained image processing model, and key point information for determining the position of the target to be measured output by the image processing model is obtained. Among them, the image processing model includes a Bayesian convolutional layer with a convolutional kernel in a distribution form. Thus, regardless of whether the obtained image information to be processed is a clear image or a blurred image, the position of the target to be measured can be accurately determined according to the image information to be processed and the image processing model.

[0036] Other features and advantages of the present disclosure will be described in detail in the subsequent specific implementation part. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings are used to provide a further understanding of the present disclosure, and constitute a part of the specification. Together with the following specific implementation, they are used to explain the present disclosure, but do not constitute a limitation to the present disclosure. In the drawings:

[0038] Figure 1 is a flowchart of an image processing method shown according to an exemplary embodiment of the present disclosure.

[0039] Figure 2 is a flowchart of a method for training an image processing model shown according to an exemplary embodiment of the present disclosure.

[0040] Figure 3 is a flowchart of a method for training an image processing model shown according to another exemplary embodiment of the present disclosure.

[0041] Figure 4 is a schematic structural diagram of an image processing model shown according to an exemplary embodiment of the present disclosure.

[0042] Figure 5 is a schematic structural diagram of a control image processing model shown according to an exemplary embodiment of the present disclosure.

[0043] Figure 6 is a block diagram of an image processing apparatus shown according to an exemplary embodiment of the present disclosure.

[0044] Figure 7 is a block diagram of a training apparatus for an image processing model shown according to an exemplary embodiment of the present disclosure.

[0045] Figure 8 is a block diagram of an electronic device shown according to an exemplary embodiment of the present disclosure.

[0046] Figure 9 is a block diagram of an electronic device shown according to an exemplary embodiment of the present disclosure. Detailed Embodiments

[0047] The following will describe in detail the specific embodiments of the present disclosure with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining and illustrating the present disclosure, and are not used to limit the present disclosure.

[0048] It should be noted that all actions of obtaining signals, information or data in the present disclosure are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where it is located and obtaining authorization from the owner of the corresponding device.

[0049] Figure 1 is a flowchart of an image processing method shown according to an exemplary embodiment of the present disclosure. As Figure 1 shown, the image processing method includes steps S101 to S102.

[0050] In step S101, the image information to be processed is obtained. The image information to be processed is the image that needs to be processed. The image information to be processed can be collected by a camera. The image information to be processed collected by the camera may be clear or blurred. For example, the camera is located on a learning machine, and the learning machine is fixed by a bracket. Therefore, during the process of collecting the image to be processed, if the learning machine does not shake, the image information to be processed collected by the camera may be clear; for another example, when the user holds the learning machine and the camera is collecting the image to be processed, due to the shaking of the learning machine, the image information to be processed collected by the camera may be blurred.

[0051] In step S102, the image information to be processed is input into the trained image processing model to obtain the output key point information. The key point information is used to determine the position of the target to be measured. The image processing model includes a Bayesian convolutional layer, and the convolutional kernel in the Bayesian convolutional layer is in a distribution form.

[0052] The image processing model can be a model pre-trained by machine learning. In one implementation, the image processing model is a model used to process the image information to be processed and obtain the position of the target to be measured in the image information to be processed.

[0053] For example, the image processing method provided by the present disclosure can be applied to an augmented reality scenario. In an augmented reality scenario, the content that is hoped to be displayed through augmented reality technology is usually displayed based on the position of the target to be measured in the image to be processed. Therefore, it is necessary to determine the position of the target to be measured in the image to be processed so that the content that is hoped to be displayed through augmented reality technology can be displayed. The key point information is the output of the image processing model and can be used to determine the position of the target to be measured (or it can indicate the position of the target to be measured in the image information to be processed).

[0054] In the image processing model, a Bayesian convolutional layer can be included, where the Bayesian convolutional kernel is in a distribution form. In the related art, when processing an image, the convolutional kernel used for convolutional operation is in a fixed form. For example, when processing an image, the convolutional kernel used for convolutional operation is a preset matrix, and each element value in the preset matrix is a fixed value.

[0055] For example, in the related art, the result of the convolutional operation can be expressed by the following formula:

[0056] y=∑ω·x+b (1)

[0057] Where y is the result of the convolutional operation, x is the parameter that needs to be subjected to the convolutional operation (for example, the input of the convolutional layer), b is the bias. ω is a convolutional kernel in a fixed form.

[0058] In the image processing method provided by the present disclosure, the image processing model used includes a Bayesian convolutional layer, and the convolutional kernel in the Bayesian convolutional layer is in a distribution form. For example, the convolutional kernel used in the Bayesian convolutional layer can be the product of a fixed-form convolutional kernel and a certain probability density function. In one implementation, if the Bayesian convolutional kernel is a matrix, each element in the matrix can be the product of a fixed value and a certain probability density function.

[0059] In one implementation, the convolution operation result of the Bayesian convolutional layer can be expressed by the following formula:

[0060] y = ∑f B (ω)·x + b (2)

[0061] where y is the result of the convolution operation, x is the parameter to be convolved (e.g., the input of the convolutional layer), and b is the bias. f B (ω) is a convolutional kernel in a distribution form and can be a function with a fixed-form convolutional kernel as the independent variable.

[0062] A trained image processing model can be obtained based on machine learning techniques. That is, the expression of the convolutional kernel in the distribution form in the Bayesian convolutional layer can be obtained through training. Before training the image processing model, the parameters included in the expression of the convolutional kernel in the distribution form can be preset. During the training of the image processing model, the values of the various parameters included in the expression of the convolutional kernel in the distribution form are adjusted to obtain a trained image processing model.

[0063] During the training of the image processing model, a blurred image can be used as a sample to train the image processing model so that the trained image processing model can process the blurred image. Among them, the reason for the blurring of the blurred image used for training can include that the image captured is blurred due to camera shake or movement during shooting.

[0064] Thus, when the image information to be processed obtained in step S101 is a blurred image, during the execution of step S102, the image processing model can be used to process the blurred image (image information to be processed) to obtain the output key point information, and determine the position of the target to be measured in the image information to be processed. Since the image processing model includes a Bayesian convolutional layer, the convolutional kernel in the Bayesian convolutional layer is in a distribution form, and during the training of the image processing model, blurred images are used as samples to train the image processing model, and the parameters in the expression of the convolutional kernel with a distribution form are adjusted. Therefore, in step S102, when the image information to be processed is a blurred image, the image processing model can analyze the blurring trend (for example, the reason for the image blurring is the relative movement between the camera and the object being photographed, and the trend of the relative movement between the camera and the object being photographed is the blurring trend). Furthermore, when the image information to be processed obtained in step S101 is a blurred image, the accurate position of the target to be measured can be determined based on the image information to be processed and the image processing model.

[0065] If the image information to be processed obtained in step S101 is a clear image, since the convolutional kernel of the Bayesian convolutional layer is in a distribution form, it can be considered that the image processing model can still analyze the blurring trend of the image information to be processed (that is, there is no blurring trend). Thus, if the image information to be processed is a clear image, the accurate position of the target to be measured can also be determined based on the image information to be processed and the image processing model.

[0066] Through the above technical solution, the obtained image information to be processed is input into the trained image processing model to obtain the key point information output by the image processing model for determining the position of the target to be measured, where the image processing model includes a Bayesian convolutional layer with a convolutional kernel in a distribution form. Thus, regardless of whether the obtained image information to be processed is a clear image or a blurred image, the position of the target to be measured can be accurately determined based on the image information to be processed and the image processing model.

[0067] It should be noted that for devices with a low camera frame rate, when the device shakes rapidly, the obtained images will be relatively blurred, making it difficult to determine the position of the target to be measured, and it will be difficult to display the content that is hoped to be displayed through augmented reality technology. Using the image processing method provided by the present disclosure can well solve the above problems. That is, when the image information to be processed obtained is a blurred image, the position of the target to be measured can also be accurately determined according to the image processing method provided by the present disclosure, and then the content that is hoped to be displayed through augmented reality technology can be displayed based on the determined position of the target to be measured.

[0068] In another embodiment, the distribution form is a normal distribution form.

[0069] The convolutional kernel in the form of a distribution can be represented by the following formula:

[0070]

[0071] Among them, σ is the standard deviation parameter of the normal distribution, and μ is the mean parameter of the normal distribution. The standard deviation parameter and the mean parameter in the expression of the convolutional kernel in the form of a normal distribution can be adjusted during the training of the image processing model. That is to say, adjusting the standard deviation parameter and the mean parameter in the expression of the convolutional kernel in the form of a normal distribution is also the training content for training the image processing model. In this embodiment, the convolutional kernel in the Bayesian convolutional layer is a convolutional kernel in the form of a normal distribution. In this way, when the image processing model is trained and the standard deviation parameter and the mean parameter in the expression of the convolutional kernel in the form of a normal distribution are adjusted to appropriate values, during the process of executing the image processing method provided by the present disclosure, the position of the target to be measured can be determined more accurately, and the implementation effect is good.

[0072] In another embodiment, in the Bayesian convolutional layer, the input of the Bayesian convolutional layer undergoes a convolutional operation to obtain a convolutional operation result, and the convolutional operation result undergoes a Gaussian Mixture Model (GMM) algorithm operation to obtain the output of the Bayesian convolutional layer, where the convolutional operation result is in the form of a distribution, and the output is the mean of the Gaussian distribution with the largest corresponding probability among one or more Gaussian distributions obtained by operating the convolutional operation result with the Gaussian mixture model algorithm.

[0073] The convolutional operation result is the result obtained by the input of the Bayesian convolutional layer undergoing a convolutional operation. Since the convolutional kernel used in the convolutional operation process in the Bayesian convolutional layer is in the form of a distribution, the convolutional operation result is also in the form of a distribution.

[0074] For example, if the convolutional kernel in the fixed form is a matrix with a dimension of 3*3, the convolutional kernel in the form of a distribution can be a matrix with a dimension of 3*3*3. For example, if the convolutional kernel in the Bayesian convolutional layer is a convolutional kernel in the form of a normal distribution, the two additional channels of the convolutional kernel in the form of a normal distribution compared to the convolutional kernel in the fixed form are respectively the standard deviation parameter corresponding to each element in the fixed form convolutional kernel (for example, the standard deviation parameter of the normal distribution corresponding to a certain element) and the mean parameter corresponding to each element in the fixed form convolutional kernel (for example, the mean parameter of the normal distribution corresponding to a certain element).

[0075] After performing a convolution operation on the input of the Bayesian convolutional layer to obtain a convolution operation result, the convolution operation result can be input into a Gaussian mixture model algorithm for operation to obtain the output of the Bayesian convolutional layer. Since the convolution operation result is in the form of a distribution, the Gaussian mixture model algorithm can be used to process the convolution operation result. The Gaussian mixture model algorithm can be preset, and the use of the Gaussian mixture model algorithm is well-known to those skilled in the art and will not be described in detail here.

[0076] Parameter estimation can be performed based on the Gaussian mixture model algorithm (for example, using the likelihood function method to estimate the parameters of the Gaussian mixture model) to determine a Gaussian distribution that fits the convolution operation result, and the mean of the Gaussian distribution that fits the convolution operation result is determined as the output of the Bayesian convolutional layer.

[0077] In one implementation, after the convolution operation result is processed by the Gaussian mixture model algorithm, it is determined that the distribution that fits the convolution operation result may be a distribution formed by the linear superposition of one or more Gaussian distributions. The mean of the Gaussian distribution with the highest corresponding probability in this Gaussian distribution (when the obtained distribution is a distribution composed of one Gaussian distribution) or multiple Gaussian distributions (when the obtained distribution is a distribution formed by the superposition of multiple Gaussian distributions) can be determined as the output of the Bayesian convolutional layer. Thus, the value output by the Bayesian convolutional layer can be a determined value (compared to the value in the form of a distribution) and is no longer in the form of a distribution.

[0078] In this embodiment, the Gaussian mixture model algorithm is used to process the convolution operation result, so that the output of the Bayesian convolutional layer is a determined value. In this way, the output of the Bayesian convolutional layer can be used as an input and input into other layers in the image processing model, maintaining the isolation between levels in the image processing model and facilitating data processing.

[0079] Figure 2 It is a flowchart of a training method for an image processing model shown according to an exemplary embodiment of the present disclosure. The image processing model trained by applying this training method for the image processing model can be applied to the image processing method provided by the present disclosure. As Figure 2 shown, this training method for the image processing model includes steps S201 to S205.

[0080] In step S201, training samples are obtained.

[0081] In step S202, the training samples are input into the motion blur model to obtain blurred training samples.

[0082] In step S203, the blurred training samples are input into the initial image processing model to obtain training key point output information, where the initial image processing model includes a Bayesian convolutional layer, and the convolutional kernel in the Bayesian convolutional layer is in the form of a distribution.

[0083] The training samples can be samples used to train the initial image processing model. By training the initial image processing model, the image processing model used in step S102 can be obtained. The training samples can be clear images.

[0084] The motion blur model can be preset based on artificial motion blur technology. After inputting a clear image, it blurs the clear image to obtain a blurred image. Artificial motion blur technology is well-known to those skilled in the art and will not be elaborated here. After the training samples are input into the motion blur model, the obtained blurred image is the blurred training sample.

[0085] The initial image processing model is the model to be trained. The R & D personnel can preset the initial image processing model, for example, they can preset the number of layers included in the initial image processing model, the number of parameters included in each layer, and the values of each parameter. In step S203, the blurred training samples can be input into the initial image processing model to obtain training key point output information.

[0086] The training key point output information is the output of the model being trained (the initial image processing model). The training key point output information is the information output by the model being trained for determining the position of the target to be measured in the training samples. Or rather, the training key point output information is the information output by the model being trained indicating the position of the target to be measured in the training samples.

[0087] In step S204, the loss value of the initial image processing model is determined according to the training key point output information and the true value of the key points.

[0088] The true value of the key points is the true position of the target to be measured in the training samples. The loss value of the initial image processing model can be determined according to the training key point output information and the true value of the key points. For example, the distance between the position of the target to be measured in the training samples represented by the training key point output information and the true value of the key points (i.e., the true position of the target to be measured in the training samples) can be determined as the loss value of the initial image processing model.

[0089] In step S205, the parameters of the initial image processing model are adjusted according to the loss value to train the initial image processing model to obtain an image processing model, where the image processing model includes a Bayesian convolutional layer, and the convolutional kernel in the Bayesian convolutional layer is in a distribution form.

[0090] Adjusting the parameters of the model to be trained according to the loss value to train the model to be trained is well-known to those skilled in the art and will not be elaborated here. That is to say, the initial image processing model can be trained and its parameters adjusted so that the loss value of the initial image processing model reaches a preset condition, and the image processing model used in step S102 can be obtained.

[0091] In one embodiment, steps S201 to S205 can be executed multiple times so that the loss value of the obtained image processing model reaches a preset condition.

[0092] Through the above technical solution, the initial image processing model can be trained to obtain the image processing model used in step S102, and the method is simple and has good implementation effects.

[0093] Figure 3 It is a flowchart of a method for training an image processing model shown in another exemplary embodiment of the present disclosure. As Figure 3 shown, on the basis of Figure 2 this, the method further includes step S206.

[0094] In step S206, the training sample is input into the control image processing model to obtain control key point output information, where the control image processing model includes a non-Bayesian convolutional layer, and the convolutional kernel of the non-Bayesian convolutional layer is in a fixed form.

[0095] The control image processing model can be a pre-trained model. That is to say, it can be considered that after the training sample is input into the control image processing model, the control key point output information output by the control image processing model is the position of the target to be measured in the training sample that is determined to be relatively accurate (for example, relatively accurate means that the distance from the true position of the target to be measured in the training sample is less than a predetermined distance). The control key point output information is the output of the control image processing model.

[0096] In this embodiment, determining the loss value of the initial image processing model according to the training key point output information and the true value of the key point in step S204 can include step S2041.

[0097] In step S2041, the loss value of the initial image processing model is determined according to the training key point output information, the true value of the key point, and the control key point output information.

[0098] In one embodiment, the loss value of the initial image processing model can be determined according to the following formula:

[0099]

[0100] Where loss is the loss value of the initial image processing model, is the position for training the representation of key point output information, is the position for comparing the representation of key point output information, y gt is the true value of the key point (the true position of the target to be measured), represents the Euclidean distance between the position for training the representation of key point output information and the true position of the target to be measured, represents the Euclidean distance between the position for training the representation of key point output information and the position for comparing the representation of key point output information.

[0101] As Figure 3 shown, in this embodiment, a clear image is used as the training sample. The training sample is input into the comparison image processing model to obtain the comparison key point output information; the training sample is input into the motion blur model to obtain a blurred training sample, and the blurred training sample is input into the initial image processing model to obtain the training key point output information; the loss value of the initial image processing model is determined according to the training key point output information, the true value of the key point, and the comparison key point output information, and the initial image processing model is parameter - adjusted using the determined loss value to train the initial image processing model to obtain an image processing model. In this way, the position of the target to be measured determined by the key point information output after the trained image processing model inputs a blurred image can be more accurate.

[0102] In another embodiment, the image processing model further includes a residual network. Figure 4 is a schematic structural diagram of an image processing model shown according to an exemplary embodiment of the present disclosure. As Figure 4 shown, the image processing model is composed of 5 - layer Bayesian convolutional layers, a residual network, and a key point output layer. Among them, the residual network can be Resnet18. Figure 5 is a schematic structural diagram of a comparison image processing model shown according to an exemplary embodiment of the present disclosure. As Figure 5 shown, the comparison image processing model can be composed of 5 - layer non - Bayesian convolutional layers, a residual network, and a key point output layer.

[0103] Figure 6 is a block diagram of an image processing device shown according to an exemplary embodiment of the present disclosure. As Figure 6 shown, the image processing device 300 includes a first acquisition module 301 and a first determination module 302.

[0104] The first acquisition module 301 is configured to acquire the image information to be processed.

[0105] The first determination module 302 is configured to input the image information to be processed into the trained image processing model to obtain the output key point information, where the key point information is used to determine the position of the target to be measured, and the image processing model includes a Bayesian convolutional layer, and the convolutional kernel in the Bayesian convolutional layer is in a distribution form.

[0106] In another embodiment, the distribution form is a normal distribution form.

[0107] In another embodiment, in the Bayesian convolutional layer, the input of the Bayesian convolutional layer is subjected to a convolutional operation to obtain a convolutional operation result, and the convolutional operation result is subjected to a Gaussian mixture model algorithm operation to obtain the output of the Bayesian convolutional layer, where the convolutional operation result is in a distribution form, and the output is the mean value of the Gaussian distribution with the highest corresponding probability among one or more Gaussian distributions obtained by subjecting the convolutional operation result to the Gaussian mixture model algorithm operation.

[0108] Figure 7 It is a block diagram of a training device for an image processing model shown according to an exemplary embodiment of the present disclosure. As Figure 7 shown, the training device 400 for the image processing model includes a second acquisition module 401, a second determination module 402, a third determination module 403, a fourth determination module 404, and a fifth determination module 405.

[0109] The second acquisition module 401 is configured to acquire training samples.

[0110] The second determination module 402 is configured to input the training samples into the motion blur model to obtain blurred training samples.

[0111] The third determination module 403 is configured to input the blurred training samples into the initial image processing model to obtain training key point output information.

[0112] The fourth determination module 404 is configured to determine the loss value of the initial image processing model according to the training key point output information and the true value of the key points.

[0113] The fifth determination module 405 is configured to adjust the parameters of the initial image processing model according to the loss value to train the initial image processing model to obtain an image processing model.

[0114] Wherein, the initial image processing model and the image processing model include a Bayesian convolutional layer, and the convolutional kernel in the Bayesian convolutional layer is in a distribution form.

[0115] In another embodiment, the training device 400 for the image processing model further includes a sixth determination module.

[0116] The sixth determination module is configured to input training samples into a control image processing model to obtain control key point output information, where the control image processing model includes a non-Bayesian convolutional layer, and the convolutional kernel of the non-Bayesian convolutional layer has a fixed form.

[0117] The fourth determination module 404 is further configured to determine the loss value of the initial image processing model according to the training key point output information, the true value of the key points, and the control key point output information.

[0118] In another embodiment, the image processing model further includes a residual network.

[0119] The present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the above image processing method are implemented, or when the program instructions are executed by a processor, the steps of the above image processing model training method are implemented.

[0120] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0121] Figure 8 is a block diagram of an electronic device 700 shown according to an exemplary embodiment of the present disclosure. As Figure 8 shown, the electronic device 700 may include: a processor 701, a memory 702. The electronic device 700 may further include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0122] Among them, the processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned image processing method or the training method of the image processing model. The memory 702 is used to store various types of data to support the operation of the electronic device 700. These data may include, for example, instructions for any application or method operating on the electronic device 700, as well as application-related data, such as contact data, sent and received messages, pictures, audio, video, and so on. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc. The multimedia component 703 may include a screen and an audio component. The screen can be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal can be further stored in the memory 702 or sent through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, and the above-mentioned other interface modules can be a keyboard, a mouse, buttons, etc. These buttons can be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 705 may include: a Wi-Fi module, a Bluetooth module, an NFC module, and so on.

[0123] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components, and is used to execute the above-mentioned image processing method or the training method of the image processing model.

[0124] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When the program instructions are executed by a processor, the steps of the above-mentioned image processing method or the training method of the image processing model are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions can be executed by the processor 701 of the electronic device 700 to complete the above-mentioned image processing method or the training method of the image processing model.

[0125] Figure 9 FIG. 1900 is a block diagram of an electronic device 1900 according to an exemplary embodiment of the present disclosure. For example, the electronic device 1900 can be provided as a server. Referring to Figure 9 , the electronic device 1900 includes a processor 1922, the number of which can be one or more, and a memory 1932 for storing computer programs executable by the processor 1922. The computer programs stored in the memory 1932 can include one or more modules each corresponding to a set of instructions. In addition, the processor 1922 can be configured to execute the computer program to execute the above-mentioned image processing method or the training method of the image processing model.

[0126] In addition, the electronic device 1900 can further include a power supply component 1926 and a communication component 1950. The power supply component 1926 can be configured to perform power management of the electronic device 1900, and the communication component 1950 can be configured to implement communication of the electronic device 1900, for example, wired or wireless communication. In addition, the electronic device 1900 can further include an input / output (I / O) interface 1958. The electronic device 1900 can operate based on an operating system stored in the memory 1932.

[0127] In another exemplary embodiment, there is also provided a computer-readable storage medium including program instructions, which, when executed by a processor, implement the steps of the above-described image processing method or the training method of the image processing model. For example, the non-transitory computer-readable storage medium may be the above-described memory 1932 including program instructions, and the above program instructions may be executed by the processor 1922 of the electronic device 1900 to complete the above-described image processing method or the training method of the image processing model.

[0128] In another exemplary embodiment, there is also provided a computer program product, which includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above-described image processing method or the training method of the image processing model when executed by the programmable device.

[0129] The preferred embodiments of the present disclosure have been described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0130] In addition, it should be noted that, in the above specific embodiments, the various specific technical features described can be combined in any suitable manner without conflict. To avoid unnecessary repetition, the present disclosure does not separately describe various possible combination methods.

[0131] Furthermore, any combination can be made between various different embodiments of the present disclosure as long as it does not violate the idea of the present disclosure, and it should also be regarded as the content disclosed by the present disclosure.

Claims

1. An image processing method, characterized in that, Including: Obtain the image information to be processed; Input the image information to be processed into the trained image processing model to obtain the output key point information. Among them, the key point information is used to determine the position of the target to be measured. The image processing model includes a Bayesian convolutional layer, and the convolutional kernel in the Bayesian convolutional layer is in a distribution form. In the Bayesian convolutional layer, the input of the Bayesian convolutional layer undergoes a convolutional operation to obtain a convolutional operation result, and the convolutional operation result undergoes a Gaussian mixture model algorithm operation to obtain the output of the Bayesian convolutional layer. Among them, the convolutional operation result is in a distribution form, and the output is the mean of the Gaussian distribution with the largest corresponding probability among one or more Gaussian distributions obtained by the convolutional operation result through the Gaussian mixture model algorithm operation.

2. The method according to claim 1, wherein The distribution form is a normal distribution form.

3. A training method for an image processing model, characterized in that The method includes: Obtain training samples; Input the training samples into the motion blur model to obtain blurred training samples; Input the blurred training samples into the initial image processing model to obtain the training key point output information; Determine the loss value of the initial image processing model according to the training key point output information and the true value of the key points; Adjust the parameters of the initial image processing model according to the loss value to train the initial image processing model to obtain the image processing model; Among them, the initial image processing model and the image processing model include a Bayesian convolutional layer, and the convolutional kernel in the Bayesian convolutional layer is in a distribution form. In the Bayesian convolutional layer, the input of the Bayesian convolutional layer undergoes a convolutional operation to obtain a convolutional operation result, and the convolutional operation result undergoes a Gaussian mixture model algorithm operation to obtain the output of the Bayesian convolutional layer. Among them, the convolutional operation result is in a distribution form, and the output is the mean of the Gaussian distribution with the largest corresponding probability among one or more Gaussian distributions obtained by the convolutional operation result through the Gaussian mixture model algorithm operation.

4. The method according to claim 3, characterized in that, The method further includes: Input the training samples into the control image processing model to obtain the control key point output information. Among them, the control image processing model includes a non-Bayesian convolutional layer, and the convolutional kernel of the non-Bayesian convolutional layer is in a fixed form; The determining the loss value of the initial image processing model according to the training key point output information and the true value of the key points includes: Determine the loss value of the initial image processing model according to the training key point output information, the true value of the key points, and the control key point output information.

5. The method according to any one of claims 1-4, characterized in that, The image processing model further includes a residual network.

6. An image processing apparatus, characterized in that, Including: The first acquisition module is configured to obtain the image information to be processed; A first determination module, configured to input the to-be-processed image information into a trained image processing model to obtain output key point information, where the key point information is used to determine the position of a to-be-detected target, the image processing model includes a Bayesian convolutional layer, the convolutional kernel in the Bayesian convolutional layer is in a distribution form, in the Bayesian convolutional layer, the input of the Bayesian convolutional layer undergoes a convolutional operation to obtain a convolutional operation result, and the convolutional operation result undergoes a Gaussian mixture model algorithm operation to obtain the output of the Bayesian convolutional layer, where the convolutional operation result is in a distribution form, and the output is the mean of the Gaussian distribution with the highest corresponding probability among one or more Gaussian distributions obtained by the convolutional operation result through the Gaussian mixture model algorithm operation.

7. A training device for an image processing model, characterized in that, Comprising: A second acquisition module, configured to acquire training samples; A second determination module, configured to input the training samples into a motion blur model to obtain blurred training samples; A third determination module, configured to input the blurred training samples into an initial image processing model to obtain training key point output information; A fourth determination module, configured to determine the loss value of the initial image processing model according to the training key point output information and the true value of the key points; A fifth determination module, configured to adjust the parameters of the initial image processing model according to the loss value to train the initial image processing model to obtain the image processing model; Wherein, the initial image processing model and the image processing model include a Bayesian convolutional layer, the convolutional kernel in the Bayesian convolutional layer is in a distribution form, in the Bayesian convolutional layer, the input of the Bayesian convolutional layer undergoes a convolutional operation to obtain a convolutional operation result, and the convolutional operation result undergoes a Gaussian mixture model algorithm operation to obtain the output of the Bayesian convolutional layer, where the convolutional operation result is in a distribution form, and the output is the mean of the Gaussian distribution with the highest corresponding probability among one or more Gaussian distributions obtained by the convolutional operation result through the Gaussian mixture model algorithm operation.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the program instruction is executed by a processor, it implements the steps of the method according to any one of claims 1-2, or when the program instruction is executed by a processor, it implements the steps of the method according to any one of claims 3 or 4.

9. An electronic device, characterized in that, Comprising: A memory, on which a computer program is stored; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1-4.

Images