Image processing method and device and electronic equipment

The image processing method enhances key point detection efficiency and accuracy by using feature extraction and position classification vectors, eliminating the need for high-resolution thermal maps and reducing computational burden.

CN119090953APending Publication Date: 2024-12-06BEIJING ZITIAO NETWORK TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310659453.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-05
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Existing image processing systems generate Gaussian heat maps at a slow speed, leading to low efficiency and accuracy in determining key points in images due to the need for high-resolution thermal maps, which consume significant computing resources.

Method used

An image processing method that involves feature extraction to obtain M feature images, using fully connected layers to determine position classification vectors, allowing accurate key point positioning without generating high-resolution thermal maps, thus reducing computational load.

Benefits of technology

Improves the efficiency and accuracy of key point detection by calculating key point positions using position classification vectors, reducing the need for high-resolution thermal maps and conserving computing resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090953A_ABST
    Figure CN119090953A_ABST
Patent Text Reader

Abstract

The invention provides an image processing method and apparatus, and an electronic device. The method comprises the steps of obtaining a target image; feature extraction processing is carried out on the target image to obtain M feature images, the M feature images are feature images corresponding to M key points in the target image, and M is an integer greater than 0; according to the M feature images, determining M first position classification vectors of the M key points in the abscissa of the target image and M second position classification vectors of the M key points in the ordinate of the target image; and determining the positions of the M key points in the target image according to the M first position classification vectors and the M second position classification vectors. And the accuracy and efficiency of determining the key point position are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of image processing technology, and in particular to an image processing method, device and electronic device. Background Art

[0002] Image key point detection technology is particularly important in the field of image recognition technology. Objects in an image can be identified based on the key points in the image.

[0003] At present, electronic devices can process images based on key point recognition models. The key point recognition model can output the Gaussian heat map corresponding to the image. The electronic device can determine multiple points with large heat values ​​in the Gaussian heat map as key points in the image. However, the key point recognition model generates a Gaussian heat map with high clarity at a slow speed, resulting in low efficiency in determining the key points in the image. Summary of the invention

[0004] The present disclosure provides an image processing method, device and electronic device, which are used to solve one or more technical problems in the prior art.

[0005] In a first aspect, the present disclosure provides an image processing method, the method comprising:

[0006] Get the target image;

[0007] Performing feature extraction processing on the target image to obtain M feature images, where the M feature images are feature images corresponding to M key points in the target image, and M is an integer greater than 0;

[0008] Determine, according to the M feature images, M first position classification vectors of the M key points in the abscissa of the target image and M second position classification vectors in the ordinate;

[0009] The positions of the M key points are determined in the target image according to the M first position classification vectors and the M second position classification vectors.

[0010] In a second aspect, the present disclosure provides an image processing device, the image processing device comprising an acquisition module, a processing module, a first determination module and a second determination module, wherein:

[0011] The acquisition module is used to acquire a target image;

[0012] The processing module is used to perform feature extraction processing on the target image to obtain M feature images, where the M feature images are feature images corresponding to M key points in the target image, and M is an integer greater than 0;

[0013] The first determination module is used to determine, according to the M feature images, M first position classification vectors of the M key points in the horizontal coordinates of the target image and M second position classification vectors in the vertical coordinates;

[0014] The second determination module is used to determine the positions of the M key points in the target image according to the M first position classification vectors and the M second position classification vectors.

[0015] In a third aspect, an embodiment of the present disclosure provides an electronic device including: a processor and a memory;

[0016] The memory stores computer-executable instructions;

[0017] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the image processing method as described in the first aspect and various possible aspects of the first aspect.

[0018] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the image processing method as described in the first aspect and various possible aspects of the first aspect are implemented.

[0019] The present disclosure provides an image processing method, device and electronic device. The electronic device can obtain a target image and perform feature extraction processing on the target image to obtain M feature images, wherein the M feature images are feature images corresponding to M key points in the target image, and M is an integer greater than 0. According to the M feature images, M first position classification vectors in the horizontal coordinates of the M key points in the target image and M second position classification vectors in the vertical coordinates are determined. The electronic device can determine the positions of the M key points in the target image according to the M first position classification vectors and the M second position classification vectors. In the above method, since the electronic device can obtain the position classification vector of the horizontal coordinate of the key point in the target image and the position classification vector of the vertical coordinate of the key point in the target image based on the feature images corresponding to the key point, the electronic device can accurately determine the position of the key point in the target image, improve the accuracy of determining the position of the key point, and the electronic device does not need to generate a high-definition thermal map, reduce the calculation amount of the electronic device, and thus improve the efficiency of determining the key points in the target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 A schematic diagram of determining key points in the prior art provided in an embodiment of the present disclosure;

[0022] Figure 2 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0023] Figure 3 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0024] Figure 4 A schematic diagram of a process for determining M feature images provided by an embodiment of the present disclosure;

[0025] Figure 5 A schematic diagram of determining a position classification vector provided by an embodiment of the present disclosure;

[0026] Figure 6 A schematic diagram of determining a location distribution probability provided by an embodiment of the present disclosure;

[0027] Figure 7 A schematic diagram of a method for determining the positions of M key points provided in an embodiment of the present disclosure;

[0028] Figure 8 A schematic diagram of a process for determining a scale conversion vector provided by an embodiment of the present disclosure;

[0029] Fig. 9 A schematic diagram of an image processing method provided by an embodiment of the present disclosure;

[0030] Fig.10 A schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure;

[0031] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0033] To facilitate understanding, the concepts involved in the embodiments of the present disclosure are explained below.

[0034] Electronic device: is a device with wireless transceiver function. Electronic devices can be deployed on land, including indoors or outdoors, handheld, wearable or vehicle-mounted. The electronic device can be a mobile phone, a tablet computer, a computer with wireless transceiver function, a virtual reality (VR) electronic device, an augmented reality (AR) electronic device, a wireless terminal in industrial control, a vehicle-mounted electronic device, a wireless terminal in self-driving, a wireless electronic device in remote medical care, a wireless electronic device in smart grid, a wireless electronic device in transportation safety, a wireless electronic device in smart city, a wireless electronic device in smart home, a wearable electronic device, etc. The electronic device involved in the embodiments of the present disclosure can also be called a terminal, a user equipment (UE), an access electronic device, a vehicle-mounted terminal, an industrial control terminal, a UE unit, a UE station, a mobile station, a mobile station, a remote station, a remote electronic device, a mobile device, a UE electronic device, a wireless communication device, a UE agent or a UE device, etc. The electronic device can also be fixed or mobile.

[0035] Next, combine Figure 1 , the method of determining key points in the prior art is explained.

[0036] Figure 1 A schematic diagram of determining key points in the prior art provided by the embodiment of the present disclosure. Figure 1, including an electronic device. The display page of the electronic device may include an image. If the electronic device performs key point extraction processing on the image, the electronic device may generate a thermal map corresponding to the image. The thermal map may include multiple points with high thermal values, and the electronic device may determine the multiple points with high thermal values ​​as key points corresponding to the image. In this way, the key points corresponding to the image can be intuitively obtained by means of a thermal map.

[0037] However, in Figure 1 In the illustrated embodiment, the electronic device can only locate the key point at a maximum of one pixel based on the thermal map, resulting in low accuracy of the key points determined by the electronic device. Furthermore, since the electronic device needs to determine the key points based on the thermal values ​​on the thermal map, if the resolution of the thermal map is low, the accuracy of multiple points with higher thermal values ​​in the thermal map is poor, resulting in the electronic device being unable to accurately locate the key points based on the thermal values ​​in the thermal map. If the resolution of the thermal map is high, the electronic device generates the thermal map slowly, which not only occupies more computing resources but also results in low efficiency in determining the key points.

[0038] In order to solve the above technical problems, an embodiment of the present disclosure provides an image processing method, an electronic device obtains a target image, and performs feature extraction processing on the target image to obtain M feature images, wherein the M feature images are feature images corresponding to M key points in the target image, the electronic device can process the M feature images according to a first fully connected layer to obtain M first position classification vectors of the M key points in the horizontal coordinate of the target image, and process the M feature images according to a second fully connected layer to obtain M second position classification vectors of the M key points in the vertical coordinate of the target image, and determine the M position distribution probabilities of the M key points in the target image according to the M first position classification vectors and the M second position classification vectors, and the electronic device can determine the positions of the M key points in the target image according to the M position distribution probabilities. In the above method, since the electronic device can obtain the position of the key point in the target image based on the position classification vector of the key point, the electronic device does not need to render a high-resolution thermal map, saving the computing resources of the electronic device and improving the efficiency of determining the key points. In addition, the electronic device can accurately calculate the specific position of the key point in the target image based on the position classification vector of the horizontal coordinate and the position classification vector of the vertical coordinate of the key point, thereby improving the accuracy of determining the position of the key point and improving the precision of the key point.

[0039] Next, combine Figure 2 , the application scenarios of the embodiments of the present disclosure are described.

[0040] Figure 2 A schematic diagram of an application scenario provided by an embodiment of the present disclosure. Figure 2, including: an electronic device. The display screen of the electronic device may include image A, and image A may include an object. If the electronic device performs key point extraction processing on image A, the electronic device may generate coordinates of 14 key points corresponding to image A, and the electronic device may mark each key point in image A according to the coordinates of the 14 key points, thereby obtaining image B. Image B may include the object in image A, and image B also includes 14 key points. In this way, the electronic device can accurately determine information such as the object and the outline of the object in image B through the 14 key points.

[0041] It should be noted that Figure 2 The application scenarios of the embodiments of the present disclosure are merely illustrative and are not intended to limit the application scenarios of the embodiments of the present disclosure.

[0042] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0043] Figure 3 A flowchart of an image processing method provided by an embodiment of the present disclosure. Figure 3 , the method may include:

[0044] S301, acquiring a target image.

[0045] The execution subject of the embodiments of the present disclosure may be an electronic device, or an image processing device provided in the electronic device. The image processing device may be implemented based on software, or may be implemented based on a combination of software and hardware, which is not limited in the embodiments of the present disclosure. Optionally, the electronic device may be any device with terminal computing capability, which is not limited in the embodiments of the present disclosure.

[0046] The target image may include an object. For example, the target image may include any object such as a limb, a face, a landscape, or a fruit, which is not limited in the embodiments of the present disclosure. The target image may include a plurality of key points to be detected. For example, the key points may be feature points, corner points, or other points in the target image that can describe image features, which is not limited in the embodiments of the present disclosure.

[0047] Optionally, when the image content of the target image is different, the key points to be detected included in the target image are also different. For example, if the target image includes a face image, the electronic device can detect 30 key points related to the face image in the target image, and if the target image includes a human body image, the electronic device can detect 63 key points related to the human body image in the target image.

[0048] It should be noted that the key points to be detected included in each type of target image can be pre-determined key points. For example, when the target image includes a face image, the electronic device can pre-determine that the face image can include 30 key points, and the electronic device can pre-set the face position indicated by each key point (such as, the first key point can be located at the top of the head, the 30th key point can be located at the chin, etc.), so that when the electronic device detects the key points in the target image including the face image, the electronic device can obtain the 30 key points corresponding to the target image. For example, when the target image includes a human body image, the electronic device can pre-determine that the human body image can include 63 key points, and pre-set the human body position indicated by each key point (such as, the 20th key point can be located at the left hand, the 40th key point can be located at the right hand, etc.), so that when the electronic device detects the key points in the target image including the human body image, the electronic device can obtain the 63 key points corresponding to the target image, and the electronic device can obtain the outline of the human body in the target image based on the 63 key points.

[0049] Optionally, the electronic device may receive a target image sent by other devices. For example, the electronic device may receive a key point detection request sent by a server, and may receive an image of at least one key point to be detected associated with the key point detection request sent by the server, and the electronic device may determine the image of at least one key point to be detected as the target image.

[0050] Optionally, the electronic device may obtain the target image in a database. For example, a plurality of images of key points to be detected may be pre-stored in the database, and the electronic device may obtain the plurality of images of key points to be detected (i.e., target images) in the database. For example, a plurality of images of key points to be detected may be stored in the database, and each image of the key points to be detected may have a unique identifier, and the electronic device may receive a key point detection request, wherein the key point detection request may include the identifiers of the plurality of images of the key points to be detected, and the electronic device may obtain the image of the key points to be detected (i.e., target image) in the database based on the identifiers of the plurality of images of the key points to be detected.

[0051] Optionally, the electronic device may also acquire the target image based on any feasible implementation method, which is not limited in the embodiments of the present disclosure.

[0052] S302: Perform feature extraction processing on the target image to obtain M feature images.

[0053] Among them, the M feature images are feature images corresponding to the M key points in the target image, and M is an integer greater than 0. For example, if 30 key points can be detected in the target image, 30 feature images can be obtained by convolution processing the target image, and if 63 key points can be detected in the target image, 63 feature images can be obtained by convolution processing the target image.

[0054] Among them, the M feature images correspond to the M key points one by one. For example, the first key point to be detected in the target image can correspond to the first feature image, the second key point to be detected in the target image can correspond to the second feature image, etc. For example, each key point to be detected in the target image can be associated with a feature image, and the electronic device can determine the position of the key point to be detected in the target image based on the feature image.

[0055] It should be noted that since the objects included in the target image are different, the number of key points to be detected in the target image is also different. Therefore, when the electronic device performs convolution processing on different types of target images (such as a target image including a face image and a target image including a limb image), the number of feature images obtained is also different.

[0056] Optionally, the electronic device may process the target image based on a convolutional neural network to obtain M feature images. For example, the electronic device may perform convolution processing on the target image based on a convolutional neural network to obtain M feature images corresponding to the target image.

[0057] Optionally, the electronic device may also obtain M feature images corresponding to the target image according to any feasible implementation method, which is not limited in the embodiments of the present disclosure.

[0058] Next, combine Figure 4 , the process of determining M feature images is explained.

[0059] Figure 4 A schematic diagram of a process for determining M feature images provided by an embodiment of the present disclosure. Figure 4 , including: a target image and a convolutional neural network CNN. The target image may include a human body image. After receiving the target image, the convolutional neural network can extract features in the target image, and then obtain feature image 1, feature image 2, ..., feature image 63. In this way, the electronic device can obtain 63 feature images corresponding to 63 key points in the target image based on the convolutional neural network.

[0060] It should be noted that in Figure 4 In the illustrated embodiment, the convolutional neural network may be a pre-trained neural network, and when the convolutional neural network is trained, the image types in the training samples are the same. For example, the training samples of the convolutional neural network may be sample images including human images, and the labels may be 63 key points of the human image, so that at the end of the convolutional neural network training, if the image received by the convolutional neural network is an image including a human body, the convolutional neural network may output 63 feature images, wherein the 63 feature images correspond one-to-one to the 63 key points of the human image, and the electronic device may determine the 63 key points of the human image based on the 63 feature images.

[0061] S303 . Determine, according to the M feature images, M first position classification vectors of the M key points in the horizontal coordinate and M second position classification vectors in the vertical coordinate of the target image.

[0062] The first position classification vector may be a classification vector corresponding to the position of the horizontal coordinate of the key point on the width of the target image, and the second position classification vector may be a classification vector corresponding to the position of the vertical coordinate of the key point on the height of the target image. For example, the first position classification vector and the second position classification vector may be one-dimensional vectors, and the electronic device may classify the positions of the key points on the X-axis based on the one-dimensional first position classification vector, and the electronic device may classify the positions of the key points on the Y-axis based on the one-dimensional second position classification vector.

[0063] Among them, the electronic device can determine the M first position classification vectors of the M key points in the horizontal coordinates of the target image and the M second position classification vectors in the vertical coordinates according to the following feasible implementation method: the M feature images are processed according to the first fully connected layer to obtain the M first position classification vectors, and the M feature images are processed according to the second fully connected layer to obtain the M second position classification vectors. In this way, the image information included in the feature image can be integrated through the fully connected layer, thereby improving the accuracy of the first position classification vector and the second position classification vector.

[0064] The first fully connected layer is a fully connected layer for determining the first position classification vector of the key point in the horizontal and vertical coordinates of the target image, and the second fully connected layer is a fully connected layer for determining the second position classification vector of the key point in the vertical coordinate of the target image. It should be noted that the first fully connected layer and the second fully connected layer are pre-trained fully connected layers, which is not limited in the embodiments of the present disclosure.

[0065] Next, combine Figure 5 , the process of determining the position classification vector of the key point is explained.

[0066] Figure 5A schematic diagram of a vector for determining a location classification provided by an embodiment of the present disclosure. Figure 5 , including the feature image corresponding to the key point, the first fully connected layer and the second fully connected layer. Among them, the electronic device ( Figure 5 (not shown) the feature image corresponding to the key point can be input into the first fully connected layer, and the first fully connected layer can output a first position classification vector of the key point in the horizontal coordinate of the target image. The electronic device can input the feature image corresponding to the key point into the second fully connected layer, and the second fully connected layer can output a second position classification vector of the key point in the vertical coordinate of the target image.

[0067] exist Figure 5 In the illustrated embodiment, the first position classification vector and the second position classification vector corresponding to the key point are one-dimensional vectors, and the length of the vector is related to the size of the target image. For example, if the width of the target image is W and the height is H, the length of the first position classification vector of the key point is W, and the length of the second position classification vector of the key point is H.

[0068] Optionally, after the electronic device obtains the first position classification vector and the second position classification vector corresponding to the key point, it can also scale the first position classification vector and the second position classification vector based on a preset scaling factor, thereby improving the accuracy of position classification. For example, if the first position classification vector corresponding to the key point is W and the second position classification vector is H, the electronic device can multiply W by the scaling factor to obtain a new first position classification vector, and multiply H by the scaling factor to obtain a new second position classification vector. If the scaling factor amplifies W and H, the accuracy of position classification of the key point is higher.

[0069] S304. Determine the positions of M key points in the target image according to the M first position classification vectors and the M second position classification vectors.

[0070] Among them, the electronic device can determine the positions of M key points in the target image according to the following feasible implementation method: determine the M position distribution probabilities of the M key points in the target image according to M first position classification vectors and M second position classification vectors, and determine the positions of the M key points in the target image according to the M position distribution probabilities.

[0071] The position distribution probability may be the distribution probability of the position of the key point in the target image. For example, the position distribution probability may include the distribution probability of the key point in the horizontal coordinate and the distribution probability of the key point in the vertical coordinate.

[0072] Among them, the electronic device can determine the M position distribution probabilities of the M key points in the target image according to the following feasible implementation method: normalize the M first position classification vectors to obtain the M first distribution probabilities of the M key points in the horizontal coordinate of the target image, and normalize the M second position classification vectors to obtain the M second distribution probabilities of the M key points in the vertical coordinate of the target image.

[0073] Among them, the position distribution probability corresponding to the key point may include a first distribution probability and a second distribution probability of the key point. The first distribution probability may be the distribution probability of the position of the horizontal coordinate of the key point in the target image, and the second distribution probability may be the distribution probability of the position of the vertical coordinate of the key point in the target image. For example, the first distribution probability may include the probability of the key point appearing at each position in the horizontal axis of the target image, and the second distribution probability may include the probability of the key point appearing at each position in the vertical axis of the target image. For example, if the width of the target image is 200 pixels, the first distribution probability corresponding to the key point may include the probability of the key point appearing at the position of 200 pixels, and if the width of the target image is 400 pixels, the second distribution probability corresponding to the key point may include the probability of the key point appearing at the position of 400 pixels.

[0074] For example, for any key point, the electronic device can normalize the first position classification vector corresponding to the key point based on the softmax method, and then obtain the first distribution probability corresponding to the key point. The electronic device can normalize the second position classification vector corresponding to the key point based on the softmax method, and then obtain the second distribution probability corresponding to the key point.

[0075] Next, combine Figure 6 , the process of determining the position distribution probability corresponding to the key point is explained.

[0076] Figure 6 A schematic diagram of determining the probability of position distribution provided by an embodiment of the present disclosure. Figure 6 , including: feature images corresponding to key points, a first fully connected layer and a second fully connected layer. Electronic device ( Figure 6 (not shown) can input feature images corresponding to key points into the first fully connected layer and the second fully connected layer, the first fully connected layer can output a first position classification vector corresponding to the key point, and the second fully connected layer can output a second position classification vector corresponding to the key point. The electronic device normalizes the first position classification vector to obtain a first distribution probability of the horizontal coordinate of the key point in the target image, and normalizes the first position classification vector to obtain a second distribution probability of the vertical coordinate of the key point in the target image.

[0077] See also Figure 6 , if the width of the target image is 4 pixels and the height is 4 pixels, then after the first position classification vector is normalized, the first distribution probability obtained can be (0, 0.2, 0.6, 0.2), that is, the probability of the horizontal coordinate of the key point being at the position of the first pixel of the horizontal axis is 0, the probability of being at the position of the second pixel of the horizontal axis is 0.2, the probability of being at the position of the third pixel of the horizontal axis is 0.6, and the probability of being at the position of the fourth pixel of the horizontal axis is 0.2. The second distribution probability obtained after the second position classification vector is normalized is similar to the first distribution probability, and the embodiments of the present disclosure will not be described in detail here. In this way, the electronic device can convert the vectors into distribution probabilities by normalizing the first position classification vector and the second position classification vector, and determine all possible positions of the key points through the distribution probabilities, thereby improving the accuracy of determining the positions of the key points.

[0078] It should be noted that Figure 6 The embodiment shown is only an exemplary illustration of the size of the target image, and is not intended to limit the target image. Figure 6 In the illustrated embodiment, after normalizing the first position classification vector, a one-dimensional vector is obtained, and the sum of the values ​​of the one-dimensional vector is 1. Similarly, after normalizing the second position classification vector, a one-dimensional vector is also obtained, and the sum of the values ​​of the one-dimensional vector is 1.

[0079] Optionally, after the electronic device determines the M position distribution probabilities corresponding to the M key points, it can determine the M coordinates of the M key points in the target image based on the M position distribution probabilities, and then determine the positions of the M key points in the target image through the M coordinates. For example, optionally, the electronic device can perform numerical coordinate regression processing on the positions of the M key points according to the M position distribution probabilities, and then obtain the positions of the M key points in the target image.

[0080] The disclosed embodiment provides an image processing method, in which an electronic device acquires a target image and performs feature extraction processing on the target image to obtain M feature images. The electronic device can process the M feature images according to the first fully connected layer to obtain M first position classification vectors of the M key points in the horizontal coordinate of the target image, and process the M feature images according to the second fully connected layer to obtain M second position classification vectors of the M key points in the vertical coordinate of the target image. According to the M first position classification vectors and the M second position classification vectors, M position distribution probabilities are determined. The electronic device can determine the positions of the M key points in the target image according to the M position distribution probabilities. In the above method, since the electronic device can calculate the position coordinates of the key points in the target image based on the position distribution vectors of the key points in the target image, the accuracy of determining the positions of the key points can be improved, and the electronic device does not need to generate a high-resolution thermal map, which reduces the amount of calculation of the electronic device, thereby improving the efficiency of determining the key points in the target image.

[0081] exist Figure 2 Based on the embodiment shown below, combined with Figure 7 , a method for determining the positions of M key points in a target image according to M position distribution probabilities in the above-mentioned image processing method is described.

[0082] Figure 7 A schematic diagram of a method for determining the positions of M key points provided by an embodiment of the present disclosure. Figure 7 The method flow includes:

[0083] S701 , performing a scale conversion on the horizontal coordinate of each pixel in the target image to obtain a first scale conversion vector.

[0084] Optionally, the value of the horizontal coordinate of each pixel after the scale conversion is within a preset range. For example, the preset range may be between (-1, 1), the horizontal coordinate of each pixel after the scale conversion may be between (-1, 1), and the vertical coordinate of each pixel may also be between (-1, 1).

[0085] The electronic device may perform scale conversion on the horizontal coordinate of each pixel in the target image according to the following formula:

[0086]

[0087] Among them, i is the horizontal coordinate of the pixel of the target image, n is the length of the first position classification vector, X i is the first scale conversion vector, and the range of i is 1, 2, ..., n.

[0088] Thus, if the length of the first position classification vector is 5 (n is 5), the horizontal coordinates of the pixels of the target image are scaled based on the above formula, and the obtained 5 scale-converted horizontal coordinates (i.e., the first scale conversion vector) can be -0.8, -0.4, 0, 0.4, and 0.8. This can reduce the computational complexity of determining the key point position and improve the efficiency of determining the key point position.

[0089] It should be noted that the electronic device may also perform scale conversion on the horizontal coordinate of the target image based on any other feasible implementation manner, and the embodiments of the present disclosure are not limited to this.

[0090] S702: Perform a scale conversion on the ordinate of each pixel in the target image to obtain a second scale conversion vector.

[0091] The electronic device may perform scale conversion on the ordinate of each pixel in the target image according to the following formula:

[0092]

[0093] Where j is the ordinate of the pixel of the target image, m is the length of the second position classification vector, and Y j is the second scale conversion vector, and the range of j is 1, 2, …, m.

[0094] Thus, if the length of the second position classification vector is 5 (n is 5), the ordinates of the pixels of the target image are scaled based on the above formula, and the five scale-converted ordinates (i.e., the second scale-converted vectors) obtained can be -0.8, -0.4, 0, 0.4, and 0.8. This can reduce the computational complexity of determining the key point positions and improve the efficiency of determining the key point positions.

[0095] It should be noted that the electronic device may also perform scale conversion processing on the vertical coordinate of the target image based on any feasible implementation method, and the embodiments of the present disclosure are not limited to this.

[0096] Next, combine Figure 8 , a process of obtaining a first scale conversion vector and a second scale conversion vector by an electronic device is described.

[0097] Figure 8 A schematic diagram of a process for determining a scale conversion vector provided by an embodiment of the present disclosure. Figure 8 , including: a target image. Wherein, the width of the target image is 5 pixels, and the height of the target image is 5 pixels. Electronic device ( Figure 8The target image may be scaled (not shown) to obtain a first scale conversion vector and a second scale conversion vector. The first scale conversion vector has a range of [-0.8, 0.8] (i.e., the value range of the X coordinate after scale conversion), and the second scale conversion vector has a range of [-0.8, 0.8] (i.e., the value range of the Y coordinate after scale conversion).

[0098] S703 : Determine positions of M key points in the target image according to the M position distribution probabilities, the first scale conversion vector, and the second scale conversion vector.

[0099] Among them, the electronic device can determine the positions of M key points in the target image according to the following feasible implementation method: multiply M first distribution probabilities among the M position distribution probabilities by the first scale conversion vector to obtain M groups of first values, multiply M second distribution probabilities among the M position distribution probabilities by the second scale conversion vector to obtain M groups of second values, and determine the positions of the M key points in the target image according to the M groups of first values ​​and the M groups of second values.

[0100] The first numerical value may be a plurality of numerical values ​​obtained by multiplying the first distribution probability by the first scale conversion vector. For example, since the first distribution probability and the first scale conversion vector have the same length (for example, both are 5 parameters), the number of numerical values ​​included in a set of first numerical values ​​corresponding to the key point may be the same as the length. For example, if the first distribution probability corresponding to the key point is 0, 0, 0.1, 0.7, 0.2, and the first scale conversion vector is -0.8, -0.4, 0, 0.4, 0.8, then the set of first numerical values ​​may be 0 (0×-0.8), 0 (0.×-0.4), 0 (0.1×0), 0.28 (0.7×0.4), 0.16 (0.2×0.8).

[0101] The second value may be a plurality of values ​​obtained by multiplying the second distribution probability by the second scale conversion vector. For example, since the second distribution probability and the second scale conversion vector have the same length (for example, both are 5 parameters), the number of values ​​included in a set of second values ​​corresponding to the key point may be the same as the length. For example, if the second distribution probability corresponding to the key point is 0, 0, 0.2, 0.6, 0.2, and the second scale conversion vector is -0.8, -0.4, 0, 0.4, 0.8, then the set of first values ​​may be 0 (0×-0.8), 0 (0.×-0.4), 0 (0.2×0), 0.24 (0.6×0.4), 0.16 (0.2×0.8).

[0102] In this way, the electronic device can determine the position of the key point in the target image not only based on the position with the highest probability in the position distribution probability, but also based on other positions where the key point may appear in the position distribution probability, thereby improving the accuracy and precision of determining the position of the key point.

[0103] Optionally, the electronic device determines the positions of M key points in the target image based on M groups of first values ​​and M groups of second values. Specifically, for any key point, obtain the sum of a group of first values ​​corresponding to the key point and the sum of a group of second values ​​corresponding to the key point, determine the horizontal coordinate of the key point in the target image based on the sum of the group of first values, and determine the vertical coordinate of the key point in the target image based on the sum of the group of second values.

[0104] After the electronic device obtains a set of first values ​​and a set of second values ​​corresponding to the key point, it can add multiple values ​​in the set of first values ​​and add multiple values ​​in the set of second values, and then determine the position of the key point in the target image based on the sum of the set of first values ​​and the sum of the set of second values. For example, if multiple values ​​in the set of first values ​​corresponding to the key point can be 0, 0, 0, 0.28, 0.16, and multiple values ​​in the set of second values ​​can be 0, 0, 0, 0.24, 0.16, then the sum of the set of first values ​​is 0.44, and the sum of the set of second values ​​is 0.4, that is, the electronic device can determine that the horizontal coordinate corresponding to the key point is 0.44 and the vertical coordinate is 0.4, and then the position of the key point in the target image can be obtained based on the horizontal coordinate and the vertical coordinate.

[0105] Optionally, in the above embodiment, the method can be implemented based on a key point detection model. For example, the electronic device can input a target image into a trained key point detection model, and the key point detection model can output the coordinates of the key points corresponding to the target image.

[0106] Next, the training process of the key point detection model is explained.

[0107] The electronic device may obtain a training data set of a key point detection model, wherein the training data set may include multiple sample images and M real position coordinates corresponding to the M key points included in each sample image.

[0108] The electronic device may process the sample image based on the key point detection model to obtain M sample feature images corresponding to the sample image, and determine, based on the M sample feature images, M sample first position classification vectors in the horizontal coordinate of the M key points in the sample image and M sample second position classification vectors in the vertical coordinate of the target image;

[0109] The electronic device may determine the KL divergence loss based on the M sample first position classification vectors, the M sample second position classification vectors, and the M real position coordinates corresponding to the M key points of the sample image;

[0110] The electronic device may determine M predicted position coordinates corresponding to the M key points of the sample image based on the M sample first position classification vectors and the M sample second position classification vectors;

[0111] The electronic device can determine the numerical regression loss based on the M predicted position coordinates and the M real position coordinates corresponding to the M key points of the sample image (i.e., the loss is determined based on the difference between the predicted position coordinates and the real position coordinates, such as using the square of the difference as the loss, etc., which is not limited to this in the embodiments of the present disclosure).

[0112] The electronic device may iteratively update the key point detection model based on the KL divergence loss and the numerical regression loss. For example, the electronic device may construct a loss function based on the weighted sum of the KL divergence loss and the numerical regression loss, and update the model parameters in the key point detection model based on the loss function.

[0113] It should be noted that the above embodiment is only an illustrative example of the loss function used in the training of the key point detection model, and is not a limitation on the type of loss function. In actual application, the electronic device can train the key point detection model based on any type of loss function, and the embodiments of the present disclosure do not limit this.

[0114] The disclosed embodiment provides a method for determining the positions of M key points. The electronic device can scale the horizontal coordinate of each pixel in the target image to obtain a first scale conversion vector, scale the vertical coordinate of each pixel in the target image to obtain a second scale conversion vector, and determine the positions of the M key points in the target image based on the M position distribution probabilities, the first scale conversion vector and the second scale conversion vector. In this way, the electronic device can not only combine the position probability distribution of the key point, but also determine the position of the key point in the target image based on the method of numerical coordinate regression and all possible positions of the key point in the target image, thereby improving the accuracy of determining the position of the key point, and the electronic device does not need to render a high-definition thermal map, saving computing resources and improving the efficiency of determining the position of the key point.

[0115] Based on any of the above embodiments, Fig. 9 , the process of the above-mentioned image processing method is explained.

[0116] Fig. 9 A schematic diagram of an image processing method provided by an embodiment of the present disclosure. Fig. 9 , including: target image, convolutional neural network, first fully connected layer and second fully connected layer. Electronic device ( Fig. 9 The target image (not shown) can be input into the convolutional neural network, and the target image can include a human body image. The convolutional neural network can extract the features of the target image to obtain feature image 1, feature image 2, ..., feature image 63. Among them, the 63 feature images correspond one by one to the 63 key points to be detected in the target image.

[0117] See also Fig. 9 , the electronic device can process the 63 feature images based on the first fully connected layer, and can process the 63 feature images based on the second fully connected layer, to obtain the first position classification vector and the second position classification vector corresponding to each feature image. The electronic device can normalize the first position classification vector and the second position classification vector corresponding to each feature image, to obtain the position distribution probability corresponding to each feature image, that is, the electronic device can obtain position distribution probability 1, position distribution probability 2, ..., position distribution probability 63 based on feature image 1, feature image 2, ..., feature image 63.

[0118] See also Fig. 9 , the electronic device can perform numerical coordinate regression processing on the position of the key point based on the 63 position distribution probabilities, wherein in the process of data coordinate regression processing, the electronic device can scale the target image to obtain a first scale transformation vector and a second scale transformation vector, and multiply the first distribution probability in the position distribution probability with the first scale transformation vector, and multiply the second distribution probability in the position distribution probability with the second scale transformation vector, thereby obtaining the positions corresponding to the 63 key points. In this way, the electronic device can accurately obtain the positions of the 63 key points in the target image, and there is no need to render a high-definition thermal map, thereby improving the efficiency of determining the positions of the key points, and improving the accuracy and precision of determining the positions of the key points.

[0119] Fig.10 This is a schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure. Fig.10 , the image processing device 100 may include an acquisition module 101, a processing module 102, a first determination module 103 and a second determination module 104, wherein:

[0120] The acquisition module 101 is used to acquire a target image;

[0121] The processing module 102 is used to perform feature extraction processing on the target image to obtain M feature images, where the M feature images are feature images corresponding to M key points in the target image, and M is an integer greater than 0;

[0122] The first determination module 103 is used to determine, according to the M feature images, M first position classification vectors of the M key points in the horizontal coordinates of the target image and M second position classification vectors in the vertical coordinates;

[0123] The second determination module 104 is used to determine the positions of the M key points in the target image according to the M first position classification vectors and the M second position classification vectors.

[0124] According to one or more embodiments of the present disclosure, the second determining module 104 is specifically configured to:

[0125] Determining, according to the M first position classification vectors and the M second position classification vectors, M position distribution probabilities of the M key points in the target image;

[0126] According to the M position distribution probabilities, positions of the M key points are determined in the target image.

[0127] According to one or more embodiments of the present disclosure, the second determining module 104 is specifically configured to:

[0128] Normalizing the M first position classification vectors to obtain M first distribution probabilities of the M key points in the horizontal coordinates of the target image;

[0129] The M second position classification vectors are normalized to obtain M second distribution probabilities of the M key points in the vertical coordinates of the target image; the position distribution probabilities corresponding to the key points include the first distribution probabilities and the second distribution probabilities of the key points.

[0130] According to one or more embodiments of the present disclosure, the second determining module 104 is specifically configured to:

[0131] Performing a scale conversion on the horizontal coordinate of each pixel in the target image to obtain a first scale conversion vector;

[0132] Performing a scale conversion on the ordinate of each pixel in the target image to obtain a second scale conversion vector;

[0133] The positions of the M key points in the target image are determined according to the M position distribution probabilities, the first scale conversion vector, and the second scale conversion vector.

[0134] According to one or more embodiments of the present disclosure, the second determining module 104 is specifically configured to:

[0135] Multiplying the M first distribution probabilities among the M position distribution probabilities by the first scale conversion vector to obtain M groups of first values;

[0136] Multiplying the M second distribution probabilities among the M position distribution probabilities by the second scale conversion vector to obtain M groups of second values;

[0137] The positions of the M key points in the target image are determined according to the M groups of first values ​​and the M groups of second values.

[0138] According to one or more embodiments of the present disclosure, the second determining module 104 is specifically configured to:

[0139] For any key point;

[0140] Obtaining a sum of a group of first values ​​corresponding to the key point and a sum of a group of second values ​​corresponding to the key point;

[0141] Determining the horizontal coordinate of the key point in the target image according to the sum of the set of first values;

[0142] The vertical coordinate of the key point in the target image is determined according to the sum of the group of second values.

[0143] According to one or more embodiments of the present disclosure, the first determining module 103 is specifically configured to:

[0144] Processing the M feature images according to the first fully connected layer to obtain the M first position classified vectors;

[0145] The M feature images are processed according to the second fully connected layer to obtain the M second position classification vectors.

[0146] The image processing device provided in the embodiment of the present disclosure may be used to execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0147] Fig.11 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Fig.11, which shows a schematic diagram of the structure of an electronic device 1100 suitable for implementing the embodiment of the present disclosure, and the electronic device 1100 may be an electronic device or a server. The electronic device may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.11 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0148] like Fig.11 As shown, the electronic device 1100 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage device 1108 to a random access memory (RAM) 1103. Various programs and data required for the operation of the electronic device 1100 are also stored in the RAM 1103. The processing device 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0149] Typically, the following devices may be connected to the I / O interface 1105: input devices 1106 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 1107 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1108 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1109. The communication devices 1109 may allow the electronic device 1100 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.11 The electronic device 1100 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0150] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 1109, or installed from a storage device 1108, or installed from a ROM 1102. When the computer program is executed by the processing device 1101, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0151] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0152] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0153] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0154] An embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the image processing methods that may be involved in the above embodiments are implemented.

[0155] An embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the image processing methods that may be involved in various embodiments described above.

[0156] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0157] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0158] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0159] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0160] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0161] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0162] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0163] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0164] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message. As an optional but non-limiting implementation method, in response to receiving an active request from the user, the method of sending a prompt message to the user can be, for example, a pop-up window, in which the prompt message can be presented in text. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device. It can be understood that the above notification and the process of obtaining user authorization are only illustrative and do not constitute a limitation on the implementation method of the present disclosure. Other methods that meet relevant laws and regulations can also be applied to the implementation method of the present disclosure.

[0165] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of the relevant laws and regulations. The data may include information, parameters and messages, such as flow switching indication information.

[0166] In a first aspect, according to one or more embodiments of the present disclosure, there is provided an image processing method, the method comprising:

[0167] Get the target image;

[0168] Performing feature extraction processing on the target image to obtain M feature images, where the M feature images are feature images corresponding to M key points in the target image, and M is an integer greater than 0;

[0169] Determine, according to the M feature images, M first position classification vectors of the M key points in the abscissa of the target image and M second position classification vectors in the ordinate;

[0170] The positions of the M key points are determined in the target image according to the M first position classification vectors and the M second position classification vectors.

[0171] According to one or more embodiments of the present disclosure, determining the positions of the M key points in the target image according to the M first position classification vectors and the M second position classification vectors includes:

[0172] Determining, according to the M first position classification vectors and the M second position classification vectors, M position distribution probabilities of the M key points in the target image;

[0173] According to the M position distribution probabilities, positions of the M key points in the target image are determined.

[0174] According to one or more embodiments of the present disclosure, determining the positions of the M key points in the target image according to the M first position classification vectors and the M second position classification vectors includes:

[0175] Normalizing the M first position classification vectors to obtain M first distribution probabilities of the M key points in the horizontal coordinates of the target image;

[0176] The M second position classification vectors are normalized to obtain M second distribution probabilities of the M key points in the vertical coordinates of the target image; the position distribution probabilities corresponding to the key points include the first distribution probabilities and the second distribution probabilities of the key points.

[0177] According to one or more embodiments of the present disclosure, determining the positions of the M key points in the target image according to the M position distribution probabilities includes:

[0178] Performing a scale conversion on the horizontal coordinate of each pixel in the target image to obtain a first scale conversion vector;

[0179] Performing a scale conversion on the ordinate of each pixel in the target image to obtain a second scale conversion vector;

[0180] The positions of the M key points in the target image are determined according to the M position distribution probabilities, the first scale conversion vector, and the second scale conversion vector.

[0181] According to one or more embodiments of the present disclosure, determining the positions of the M key points in the target image according to the M position distribution probabilities, the first scale conversion vector, and the second scale conversion vector includes:

[0182] Multiplying the M first distribution probabilities among the M position distribution probabilities by the first scale conversion vector to obtain M groups of first values;

[0183] Multiplying the M second distribution probabilities among the M position distribution probabilities by the second scale conversion vector to obtain M groups of second values;

[0184] The positions of the M key points in the target image are determined according to the M groups of first values ​​and the M groups of second values.

[0185] According to one or more embodiments of the present disclosure, determining the positions of the M key points in the target image according to the M groups of first values ​​and the M groups of second values ​​includes:

[0186] For any key point;

[0187] Obtaining a sum of a group of first values ​​corresponding to the key point and a sum of a group of second values ​​corresponding to the key point;

[0188] Determining the horizontal coordinate of the key point in the target image according to the sum of the set of first values;

[0189] The vertical coordinate of the key point in the target image is determined according to the sum of the group of second values.

[0190] According to one or more embodiments of the present disclosure, determining, based on the M feature images, M first position classification vectors of the M key points in the abscissa of the target image and M second position classification vectors in the ordinate, includes:

[0191] Processing the M feature images according to the first fully connected layer to obtain the M first position classified vectors;

[0192] The M feature images are processed according to the second fully connected layer to obtain the M second position classification vectors.

[0193] In a second aspect, an embodiment of the present disclosure provides an image processing device, which may include an acquisition module, a processing module, a first determination module, and a second determination module, wherein:

[0194] The acquisition module is used to acquire a target image;

[0195] The processing module is used to perform feature extraction processing on the target image to obtain M feature images, where the M feature images are feature images corresponding to M key points in the target image, and M is an integer greater than 0;

[0196] The first determination module is used to determine, according to the M feature images, M first position classification vectors of the M key points in the horizontal coordinates of the target image and M second position classification vectors in the vertical coordinates;

[0197] The second determination module is used to determine the positions of the M key points in the target image according to the M first position classification vectors and the M second position classification vectors.

[0198] According to one or more embodiments of the present disclosure, the second determining module is specifically configured to:

[0199] Determining, according to the M first position classification vectors and the M second position classification vectors, M position distribution probabilities of the M key points in the target image;

[0200] According to the M position distribution probabilities, positions of the M key points are determined in the target image.

[0201] According to one or more embodiments of the present disclosure, the second determining module is specifically configured to:

[0202] Normalizing the M first position classification vectors to obtain M first distribution probabilities of the M key points in the horizontal coordinates of the target image;

[0203] The M second position classification vectors are normalized to obtain M second distribution probabilities of the M key points in the vertical coordinates of the target image; the position distribution probabilities corresponding to the key points include the first distribution probabilities and the second distribution probabilities of the key points.

[0204] According to one or more embodiments of the present disclosure, the second determining module is specifically configured to:

[0205] Performing a scale conversion on the horizontal coordinate of each pixel in the target image to obtain a first scale conversion vector;

[0206] Performing a scale conversion on the ordinate of each pixel in the target image to obtain a second scale conversion vector;

[0207] The positions of the M key points in the target image are determined according to the M position distribution probabilities, the first scale conversion vector, and the second scale conversion vector.

[0208] According to one or more embodiments of the present disclosure, the second determining module is specifically configured to:

[0209] Multiplying the M first distribution probabilities among the M position distribution probabilities by the first scale conversion vector to obtain M groups of first values;

[0210] Multiplying the M second distribution probabilities among the M position distribution probabilities by the second scale conversion vector to obtain M groups of second values;

[0211] The positions of the M key points in the target image are determined according to the M groups of first values ​​and the M groups of second values.

[0212] According to one or more embodiments of the present disclosure, the second determining module is specifically configured to:

[0213] For any key point;

[0214] Obtaining a sum of a group of first values ​​corresponding to the key point and a sum of a group of second values ​​corresponding to the key point;

[0215] Determining the horizontal coordinate of the key point in the target image according to the sum of the set of first values;

[0216] The vertical coordinate of the key point in the target image is determined according to the sum of the group of second values.

[0217] According to one or more embodiments of the present disclosure, the first determining module is specifically configured to:

[0218] Processing the M feature images according to the first fully connected layer to obtain the M first position classified vectors;

[0219] The M feature images are processed according to the second fully connected layer to obtain the M second position classification vectors.

[0220] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor, and a memory and a transceiver communicatively connected to the processor;

[0221] The memory stores computer-executable instructions; the transceiver is used to send and receive data;

[0222] The processor executes the computer-executable instructions stored in the memory to implement the image processing method as described in the first aspect above.

[0223] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the image processing method described in the first aspect above.

[0224] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the image processing method described in the first aspect above.

[0225] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0226] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0227] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An image processing method, characterized in that: include: Get the target image; Performing feature extraction processing on the target image to obtain M feature images, where the M feature images are feature images corresponding to M key points in the target image, and M is an integer greater than 0; Determine, according to the M feature images, M first position classification vectors of the M key points in the abscissa of the target image and M second position classification vectors in the ordinate; The positions of the M key points are determined in the target image according to the M first position classification vectors and the M second position classification vectors.

2. The method according to claim 1, characterized in that Determining the positions of the M key points in the target image according to the M first position classification vectors and the M second position classification vectors includes: Determining, according to the M first position classification vectors and the M second position classification vectors, M position distribution probabilities of the M key points in the target image; According to the M position distribution probabilities, positions of the M key points are determined in the target image.

3. The method according to claim 2, characterized in that The determining, according to the M first position classification vectors and the M second position classification vectors, the M position distribution probabilities of the M key points in the target image comprises: Normalizing the M first position classification vectors to obtain M first distribution probabilities of the M key points in the horizontal coordinates of the target image; The M second position classification vectors are normalized to obtain M second distribution probabilities of the M key points in the vertical coordinates of the target image; the position distribution probabilities corresponding to the key points include the first distribution probabilities and the second distribution probabilities of the key points.

4. The method according to claim 2 or 3, characterized in that: Determining the positions of the M key points in the target image according to the M position distribution probabilities includes: Performing a scale conversion on the horizontal coordinate of each pixel in the target image to obtain a first scale conversion vector; Performing a scale conversion on the ordinate of each pixel in the target image to obtain a second scale conversion vector; The positions of the M key points in the target image are determined according to the M position distribution probabilities, the first scale conversion vector, and the second scale conversion vector.

5. The method according to claim 4, characterized in that The determining, according to the M position distribution probabilities, the first scale conversion vector, and the second scale conversion vector, positions of the M key points in the target image includes: Multiplying the first scale conversion vector by M first distribution probabilities among the M position distribution probabilities to obtain M groups of first values; Multiplying the M second distribution probabilities among the M position distribution probabilities by the second scale conversion vector to obtain M groups of second values; The positions of the M key points in the target image are determined according to the M groups of first values ​​and the M groups of second values.

6. The method according to claim 5, characterized in that Determining positions of the M key points in the target image according to the M groups of first values ​​and the M groups of second values ​​includes: For any key point; Obtaining a sum of a group of first values ​​corresponding to the key point and a sum of a group of second values ​​corresponding to the key point; Determining the horizontal coordinate of the key point in the target image according to the sum of the set of first values; The vertical coordinate of the key point in the target image is determined according to the sum of the group of second values.

7. The method according to any one of claims 1 to 3, characterized in that: The step of determining, according to the M feature images, M first position classification vectors of the M key points in the abscissa of the target image and M second position classification vectors in the ordinate, comprises: Processing the M feature images according to the first fully connected layer to obtain the M first position classified vectors; The M feature images are processed according to the second fully connected layer to obtain the M second position classification vectors.

8. An image processing device, characterized in that: It includes an acquisition module, a processing module, a first determination module and a second determination module, wherein: The acquisition module is used to acquire a target image; The processing module is used to perform feature extraction processing on the target image to obtain M feature images, where the M feature images are feature images corresponding to M key points in the target image, and M is an integer greater than 0; The first determination module is used to determine, according to the M feature images, M first position classification vectors of the M key points in the horizontal coordinates of the target image and M second position classification vectors in the vertical coordinates; The second determination module is used to determine the positions of the M key points in the target image according to the M first position classification vectors and the M second position classification vectors.

9. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the image processing method according to any one of claims 1 to 7 is implemented.