Key point detection method

Abstract

The embodiment of the invention provides a key point detection method, and the method comprises the steps: obtaining a to-be-detected image, and enabling the to-be-detected image to comprise a to-be-detected object; the to-be-detected image is input into a key point detection model, image key points of the to-be-detected object are obtained, the key point detection model predicts a distance matrix between reference key points and the image key points, and the image key points are determined according to the distance matrix. The to-be-detected image is input into the key point detection model, and the key point detection model predicts the distance matrix between the reference key point and the image key point, so that the image key point is determined based on the distance matrix, loss caused by decoding when the image key point is determined is avoided, and the accuracy of the image key point is improved.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of computer technology, and in particular to a key point detection method. Background Art

[0002] At present, in the field of computer vision, the demand for key point detection in images is increasing. In order to improve the efficiency of key point detection, the key point detection model that has been trained or other methods can be used to detect key points. However, the current key point detection scheme still has the problem that the accuracy does not meet the demand. In addition, if the current scheme is to improve the detection accuracy, a larger amount of calculation is required, resulting in a larger memory usage.

[0003] Therefore, a more reliable solution is needed to detect key points in images. Summary of the invention

[0004] In view of this, an embodiment of the present specification provides a key point detection method. One or more embodiments of the present specification also relate to a key point detection device, a computing device, a computer-readable storage medium and a computer program to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a key point detection method is provided, including:

[0006] Acquire an image to be detected, wherein the image to be detected contains an object to be detected;

[0007] The image to be detected is input into a key point detection model to obtain the image key points of the object to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines the image key points according to the distance matrix.

[0008] According to a second aspect of an embodiment of this specification, a key point detection device is provided, including:

[0009] An acquisition module is configured to acquire an image to be detected, wherein the image to be detected contains an object to be detected;

[0010] The input module is configured to input the image to be detected into a key point detection model to obtain the image key points of the object to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines the image key points based on the distance matrix.

[0011] According to a third aspect of an embodiment of this specification, a training method for a distance prediction module is provided, which is applied to a cloud-side device, and includes:

[0012] Determine a target sample pair in the image sample set, wherein the target sample pair is composed of a target sample image and a sample distance matrix of the target sample image;

[0013] Inputting the target sample image into a distance prediction module to obtain a predicted distance matrix output by the distance prediction module;

[0014] Calculate the prediction loss value according to the prediction distance matrix and the sample distance matrix;

[0015] Adjusting the model parameters of the distance prediction module based on the predicted loss value until a model training stop condition is reached to obtain the model parameters of the distance prediction module;

[0016] The model parameters of the distance prediction module are sent to the end-side device.

[0017] According to a fourth aspect of an embodiment of this specification, another key point detection method is provided, which is applied to a cloud-side device, including:

[0018] Receiving an image to be detected sent by a client, wherein the image to be detected contains an object to be detected;

[0019] Inputting the image to be detected into a key point detection model to obtain the image key points of the object to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines the image key points according to the distance matrix;

[0020] The image key points are returned to the client.

[0021] According to a fifth aspect of the embodiments of this specification, a method for detecting key points of a face image is provided, comprising:

[0022] Acquire a face image to be detected, wherein the face to be detected is contained in the image to be detected;

[0023] The face image to be detected is input into a key point detection model to obtain the image key points of the face to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines the image key points according to the distance matrix.

[0024] According to a sixth aspect of an embodiment of this specification, a computing device is provided, including:

[0025] Memory and processor;

[0026] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the key point detection method are implemented.

[0027] According to a seventh aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned key point detection method are implemented.

[0028] According to an eighth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned key point detection method.

[0029] A key point detection method provided by an embodiment of the present specification obtains an image to be detected, wherein the image to be detected contains an object to be detected; the image to be detected is input into a key point detection model to obtain image key points of the object to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines image key points based on the distance matrix. By inputting the image to be detected into a key point detection model, the key point detection model predicts a distance matrix between reference key points and image key points, thereby determining image key points based on the distance matrix, avoiding losses caused by decoding when determining image key points, and improving the accuracy of image key points. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is an architecture diagram of a key point detection system provided by an embodiment of this specification;

[0031] Figure 2 is a flow chart of a key point detection method provided by an embodiment of this specification;

[0032] Figure 3 is a schematic diagram of a distance matrix provided by an embodiment of this specification;

[0033] Figure 4 is a schematic diagram of a submatrix to be processed provided by an embodiment of this specification;

[0034] Figure 5 is a flowchart of a distance prediction module training method provided by an embodiment of this specification;

[0035] Figure 6 is a schematic diagram of a training distance prediction module provided by an embodiment of this specification;

[0036] Figure 7 is a flow chart of another key point detection method provided by an embodiment of this specification;

[0037] Figure 8 is a flowchart of a processing process of a method for detecting key points of a face image provided by an embodiment of this specification;

[0038] Fig. 9 It is a structural schematic diagram of a key point detection device provided by an embodiment of this specification;

[0039] Fig.10 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0040] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0041] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0042] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0043] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0044] First, the terms involved in one or more embodiments of this specification are explained.

[0045] Key points (key points or landmarks) are points used to identify the location of objects in an image, such as facial key points used to identify the location of facial features, human key points used to identify the location of body joints, and gesture key points used to identify the location of finger joints.

[0046] Key points detection (or landmark detection) refers to the visual task of extracting key points from videos or images through deep learning algorithms.

[0047] Trilateration: Also known as three-point positioning, it uses three determined points and their corresponding distances (or "ranges") to determine the unknown position coordinates of a point of interest.

[0048] Multilateration: Also known as multi-point positioning, when more than three points are involved it is called multilateration.

[0049] Key point detection is a hot topic in the field of computer vision. After years of development, there are relatively mature solutions. For example, the heatmap model for heatmap prediction is widely used due to its higher stability and accuracy. However, the key point detection model based on heatmap has the following disadvantages: 1) Loss of accuracy caused by quantization error 2) Large amount of calculation and memory overhead. The model accuracy is proportional to the heatmap resolution. The larger the heatmap resolution, the larger the number of model parameters, the greater the amount of calculation and memory usage, and the model is difficult to apply to mobile devices with limited computing power.

[0050] The solution of this specification combines three-point or multi-point positioning technology to propose a key point detection method, which can reduce the number of model parameters and calculations. And this solution can be applied to various key point detection scenarios, such as face key point detection, human body key point detection, gesture key point detection, and whole body key point detection.

[0051] In this specification, a key point detection method is provided. This specification also relates to a key point detection device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0052] See also Figure 1 , Figure 1 The architecture diagram of a key point detection system provided by an embodiment of the present specification is shown. The key point detection system may include a client 100 and a server 200;

[0053] The client 100 is used to send a key point detection request to the server 200;

[0054] The server 200 is used to receive a key point detection request, obtain an image to be detected based on the key point detection request, wherein the image to be detected contains an object to be detected; input the image to be detected into a key point detection model to obtain image key points of the object to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines the image key points based on the distance matrix; and send the image key points to the client 100.

[0055] The scheme of the embodiment of the present specification is applied, and a face image with an image resolution of 256*256*3 is input into the distance prediction module (CNN module) of the key point detection model, and the feature extraction submodule in the distance prediction module analyzes the image to be detected to obtain a reference point set, and the reference point set and the preset number of key points are input into the distance prediction submodule in the distance prediction module, and the distance between the reference key points and the image key points is predicted, and the distance matrix (distance map) output by the distance prediction module is obtained as 2*2*k, wherein k represents the number of image key points, 2*2 represents the number of reference key points is 4, and dis0, dis2, dis3, and dis4 respectively represent the distances from each reference key point predicted by the module to the image key point; the distance matrix is ​​input into the key point calculation module (decode module), and the key point coordinates (x, y) output by the key point calculation module are obtained, that is, the image key point of the image is determined to be the point with the coordinates (x, y), thereby realizing the key point detection of the image.

[0056] In order to enable the distance prediction module to have the function of outputting the distance matrix, the distance prediction module needs to be trained in advance, specifically: determining a target sample pair in the image sample set, wherein the target sample pair consists of a target sample image and a sample distance matrix of the target sample image, and the sample distance is calculated by the actual key point coordinates of the sample image and the reference key point coordinates; inputting the target sample image into the distance prediction module to obtain the predicted distance matrix output by the distance prediction module; calculating the predicted loss value according to the predicted distance matrix and the sample distance matrix; adjusting the model parameters of the distance prediction module based on the predicted loss value until the model training stop condition is reached, and obtaining a trained distance prediction module.

[0057] The key point detection system may include multiple clients 100 and a server 200, wherein the client 100 may be referred to as a terminal device and the server 200 may be referred to as a cloud device. Multiple clients 100 may establish a communication connection through the server 200. In the key point detection scenario, the server 200 is used to provide key point detection services between multiple clients 100. Multiple clients 100 may serve as a sender or a receiver, respectively, and realize communication through the server 200.

[0058] The user can interact with the server 200 through the client 100 to receive data sent by other clients 100, or send data to other clients 100, etc. In the key point detection scenario, the user can publish a data stream to the server 200 through the client 100, and the server 200 generates image key points according to the data stream and pushes the image key points to other clients that have established communication.

[0059] The client 100 and the server 200 are connected via a network. The network provides a medium for a communication link between the client 100 and the server 200. The network may include various connection types, such as wired or wireless communication links or optical fiber cables, etc. The data transmitted by the client 100 may need to be encoded, transcoded, compressed, etc. before being released to the server 200.

[0060] The client 100 can be a browser, an APP (Application), or a web application such as an H5 (HyperText Markup Language 5, Hypertext Markup Language Version 5) application, or a light application (also known as a mini-program, a lightweight application) or a cloud application, etc. The client 100 can be based on the software development kit (SDK, Software Development Kit) of the corresponding service provided by the server 200, such as based on the real-time communication (RTC, Real Time Communication) SDK development and acquisition. The client 100 can be deployed in an electronic device, and needs to rely on the device to run or some APPs in the device to run. For example, the electronic device can have a display screen and support information browsing, such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. Various other types of applications can also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0061] The server 200 may include servers that provide various services, such as servers that provide communication services to multiple clients, servers for background training that provide support for models used on clients, and servers that process data sent by clients. It should be noted that the server 200 can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The server can also be a server of a distributed system, or a server combined with a blockchain. The server can also be a cloud server for basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0062] It is worth noting that the key point detection method provided in the embodiments of this specification is generally executed by the server, but in other embodiments of this specification, the client may also have similar functions to the server, thereby executing the key point detection method provided in the embodiments of this specification. In other embodiments, the key point detection method provided in the embodiments of this specification may also be jointly executed by the client and the server.

[0063] The key point detection method of this specification obtains an image to be detected, wherein the image to be detected contains an object to be detected; the image to be detected is input into a key point detection model to obtain the image key points of the object to be detected, wherein the key point detection model predicts the distance matrix between the reference key points and the image key points, and determines the image key points based on the distance matrix. By inputting the image to be detected into the key point detection model, the key point detection model predicts the distance matrix between the reference key points and the image key points, thereby determining the image key points based on the distance matrix, avoiding the loss caused by decoding when determining the image key points, and improving the accuracy of the image key points.

[0064] See also Figure 2 , Figure 2 A flowchart of a key point detection method provided by an embodiment of the present specification is shown, which specifically includes the following steps:

[0065] Step 202: Acquire an image to be detected, wherein the image to be detected contains an object to be detected.

[0066] In practical applications, in order to locate objects in an image and then make further adjustments to the objects in the image, key point detection can be performed on the image.

[0067] Among them, the image to be detected refers to an image containing an object to be detected; the object to be detected refers to an object or a part of an object contained in the image to be detected; for example, the image to be detected may be a face image, and the object to be detected in the face image may be eyes, nose and other face parts; the image to be detected may be a landscape image, and the object to be detected in the landscape image may be a mountain, road, river, etc. in the landscape image.

[0068] Specifically, the image to be detected is obtained based on project requirements. For example, the image to be detected is determined according to the image identifier in the key point detection request, and the image to be detected is obtained according to the image analysis task trigger, etc.; the object to be detected is included in the image to be detected, so that the image key points of the object to be detected can be detected in the image to be detected later.

[0069] In a specific implementation of the present specification, a terminal receives a key point detection request; in response to the key point detection request, determines a face image in an image to be detected based on an image identifier contained in the key point detection request; and includes a face object in the face image so that key points of the face object can be subsequently detected.

[0070] By acquiring the image to be detected containing the object to be detected, the object to be detected can be further processed later.

[0071] Step 204: input the image to be detected into a key point detection model to obtain the image key points of the object to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines the image key points based on the distance matrix.

[0072] Among them, the key point detection model refers to a model that can output corresponding image key points based on the input image to be detected; image key points refer to key points in the image to be detected that can identify the target object to be detected, for example, key points that identify eyes and noses in face images, etc.; reference key points refer to key points used to determine image key points; distance matrix refers to a matrix composed of the distances between each reference key point and the image key point.

[0073] Specifically, the determined image to be detected is input into the trained key point detection model; the key point detection model analyzes the image to be detected, determines the reference key points in the image to be detected, or predicts the reference key points in the image to be detected based on needs; predicts the distance between the reference key points and the image key points based on the reference key points, and generates a distance matrix based on each distance; determines the image key points in the image to be detected according to the distance matrix.

[0074] In a specific implementation of the present specification, a trained key point detection model is determined; a face image to be detected is input into the key point detection model; reference key points in the face image to be detected are determined, and the distances between the reference key points and the image key points are predicted based on the reference key points to obtain a predicted distance matrix; and the coordinates of the image key points of the face image are calculated using the distance matrix and the coordinates of the reference key points.

[0075] At present, the image that needs to be detected by key points can be encoded into a heat map; then the key point coordinates are calculated based on the heat map; however, when using the heat map to calculate the key point coordinates, there will be a problem of precision loss caused by quantization error, as well as a large amount of calculation and memory overhead caused by a large heat map; for example, a 256*256*3 image is input to the encoding module, and K 64*64 resolution heat map images are output after network convolution processing, where K represents the number of key points; then the (x, y) coordinates of K key points are output after processing by the decoding module; where the heat map size is 64*64, when the heat map size takes a smaller value, the accuracy of the output key point coordinates decreases significantly, and when the heat map size takes a smaller value, the accuracy of the output key point coordinates decreases significantly. When the map takes a larger value, the accuracy is higher, but the calculation amount of the accompanying model also increases dramatically; specifically, taking the image containing S0 as an example, the coordinates of S0 in the image are determined to be (147.32, 131.8), and the actual coordinates under downsampling to 64*64 resolution are (36.83, 32.95), but after being processed by the decoding module, that is, the coordinates obtained after argmax of the heatmap are (36, 32), and the precision after the decimal point is lost. Then the x and y coordinates are upsampled 4 times to the original image resolution to obtain the final coordinate S1 is (144, 28). It can be found that there is a deviation of (3.32, 3.8) between the input coordinate S0 and the output coordinate S1. Even if the encoding module can predict an accurate heatmap, the coordinate values ​​of the key points will also have precision loss due to quantization error when processed by the decoding module.

[0076] Therefore, in order to avoid the above problems, the method of the present specification directly predicts the distance between the key point and the reference key point to obtain a distance matrix, and then determines the key point based on the distance matrix, thereby eliminating the quantization error; and the distance matrix generated by encoding occupies less space than the heatmap, which can reduce the amount of calculation and memory overhead.

[0077] In practical applications, since the encoded heatmap also contains the distance information between the key points and the reference key points, the heatmap can also be used as a distance matrix, and the key point detection model can decode the distance information in the heatmap to determine the key points and obtain more accurate key point coordinates; that is, in the current key point detection method, the decoding process of the heatmap can be adjusted to improve the accuracy of key point detection.

[0078] By inputting the image to be detected into the key point detection model, the key point detection model predicts the distance matrix, and then determining the image key points based on the distance matrix, the errors in determining the image key points can be avoided and the accuracy of the image key points can be improved.

[0079] Furthermore, in practical applications, the key point detection model includes a distance prediction module and a key point calculation module;

[0080] The method of inputting the image to be detected into a key point detection model to obtain the image key points of the object to be detected may include:

[0081] Inputting the image to be detected into the distance prediction module to obtain a distance matrix output by the distance prediction module;

[0082] The distance matrix is ​​input into the key point calculation module to obtain the image key points output by the key point calculation module.

[0083] The distance prediction module refers to a module that can predict a distance matrix based on an input image to be detected; the key point calculation module refers to a module that can calculate the coordinates corresponding to the key points of the image based on the distance matrix.

[0084] Specifically, the key point detection model can be a model composed of a distance prediction module and a key point calculation module; the distance prediction module can be trained through samples so that it has a distance matrix that can be predicted based on the input image output; the key point calculation module can calculate the coordinates of the image key points in the image to be detected based on the input distance matrix.

[0085] In a specific implementation of the present specification, a determined face image is input into a distance prediction module of a key point detection model to obtain a distance matrix corresponding to the face image; the distance matrix is ​​input into a key point calculation module to obtain the coordinates of the key point where the nose object is located in the face image.

[0086] In practical applications, the distance prediction module can be pre-trained so that it can output a distance matrix based on an image, or it can determine the distance matrix corresponding to the image to be detected based on other methods. This manual takes the example of a distance prediction module including a feature extraction submodule and a distance prediction submodule to describe the process of generating the distance matrix.

[0087] Specifically, the distance prediction module includes a feature extraction submodule and a distance prediction submodule; the method of inputting the image to be detected into the distance prediction module and obtaining the distance matrix output by the distance prediction module may include:

[0088] Inputting the image to be detected into the feature extraction submodule, and obtaining a reference point set corresponding to the image to be detected based on a preset number of reference points;

[0089] The preset number of key points and the reference point set are input into the distance prediction submodule to obtain a distance matrix.

[0090] Among them, the feature extraction submodule refers to a module that can determine a reference point set based on the input image to be detected; the distance prediction submodule refers to a module that can determine a distance matrix based on a preset number of key points and a reference point set; the preset number of reference points refers to the number of reference points pre-set for determining image key points in the image to be detected; the preset number of key points refers to the preset number of image key points that need to be determined in the image to be detected; the reference point set refers to a set composed of reference points in the image to be detected. In practical applications, the number of reference points in the reference point set needs to be greater than or equal to three, and the reference points are not collinear, so that the image key points in the image to be detected can be predicted subsequently.

[0091] Specifically, the image to be detected is first input into the feature extraction submodule of the distance prediction module, and the feature extraction submodule extracts the image features of the image to be detected based on a preset number of reference points, thereby determining the reference key points in the image to be detected and generating a key point set; in practical applications, the reference key points can be determined in the image to be detected based on dimensions such as image resolution and image size, which are not specifically limited in this specification; further, the preset number of key points is obtained, and the preset number of key points and the reference point set are input into the distance prediction submodule of the distance prediction module, and the distance prediction submodule predicts the image key points of the image to be detected based on the preset number of key points and the reference point set.

[0092] In a specific implementation of this specification, a face image is input into a feature extraction submodule, and the feature extraction submodule extracts the face image according to the resolution of the face image; Figure 3 , Figure 3 is a schematic diagram of a distance matrix provided by an embodiment of the present specification, wherein a reference point set is determined in a face image, and the reference point set includes: Figure 3There are four reference key points a1, a2, a3, and a4 shown in the figure; the preset key point number 1 and the reference point set are input into the distance prediction submodule, and the distance prediction submodule predicts the distances d1, d2, d3, and d4 between each reference key point pair and the image key point s0 according to the preset key point number and the reference point set, and generates a distance matrix corresponding to the face image based on each distance.

[0093] By determining the reference point set of the image to be detected, and then predicting the distance matrix of the image to be detected based on the reference point set and the preset number of key points, the coordinates of the image key points can be directly calculated based on the distance matrix, thereby avoiding the loss caused by calculating the image key points.

[0094] In practical applications, after determining the distance matrix of the image to be detected, the predicted distance between the key point and the reference key point can be obtained by decoding the distance matrix. Based on the predicted distance and the coordinates of the reference key point in the image to be detected, the coordinates of the image key point can be directly calculated, thereby avoiding the loss caused by determining the image key point through decoding and improving the accuracy of the image key point.

[0095] Specifically, the distance prediction module predicts the distance between the key point and the reference key point to generate a distance matrix; the generated distance matrix is ​​input to the key point calculation module, and the key point calculation module calculates the coordinates of the key point according to the distance matrix.

[0096] Furthermore, a distance sub-matrix can be determined in the distance matrix, so as to calculate the coordinates of the key points of the image based on the distance sub-matrix, thereby reducing the amount of calculation and improving the accuracy of the coordinates of the key points of the image. The specific processing process is as follows:

[0097] Specifically, the method of inputting the distance matrix into the key point calculation module to obtain the image key points output by the key point calculation module may include:

[0098] determining a target distance in the distance matrix;

[0099] Filtering a distance sub-matrix in the distance matrix based on the target distance;

[0100] The image key points are determined according to the distance sub-matrix and the reference key points corresponding to the distance sub-matrix.

[0101] Among them, the target distance refers to the distance selected based on the needs in the distance matrix; the distance submatrix refers to the part of the distance matrix determined in the distance matrix according to the target distance. For example, if the distance matrix is ​​an 8*8 matrix, the distance submatrix can be a 3*3 matrix determined in the distance matrix. It should be noted that the size of the distance submatrix is ​​smaller than the distance matrix. For example, if the distance matrix is ​​a 10*10 matrix, the distance submatrix can be a matrix of 5*5, 8*8, etc.; the reference key points corresponding to the distance submatrix refer to the reference key points corresponding to each distance in the distance submatrix.

[0102] Specifically, the target distance is filtered in the distance matrix based on the distance value corresponding to the distance matrix. For example, the distance whose distance value is greater than a preset distance threshold can be filtered as the target distance, and it can be a target distance randomly selected in the distance matrix, etc.; according to the target distance, a distance sub-matrix containing the target distance is determined in the distance matrix; and then based on the distance sub-matrix and the reference key points corresponding to each distance in the distance sub-matrix, the image key points of the image to be detected are determined.

[0103] In a specific implementation of the present specification, in a distance matrix of a landscape image, distance d1 is selected as a target distance based on the distance value; based on the target distance, a distance sub-matrix composed of d1, d2, d3 and d4 is selected in the distance matrix; based on the distance sub-matrix and reference key points a1, a2, a3 and a4 corresponding to d1, d2, d3 and d4 in the distance sub-matrix, the image key point c of the river object in the landscape image is determined.

[0104] By determining the target distance in the distance matrix and then filtering the distance sub-matrix based on the target distance, the image key points can be determined based on the distance sub-matrix, which reduces the amount of calculation to determine the image key points, thereby reducing the error accumulation caused by the large amount of calculation, thereby improving the accuracy of the image key points.

[0105] Further, in a preferred embodiment of the present specification, the method for determining the target distance in the distance matrix may include:

[0106] Sorting according to the distance value of each distance in the distance matrix;

[0107] A target distance is determined among the distances based on the sorting result.

[0108] The sorting result refers to the sorting result of the distances in the distance matrix.

[0109] Specifically, since the distance represents the distance between the image key point and the reference key point, when the distance prediction module outputs the encoded distance matrix, the larger the distance value in the distance matrix, the closer the distance between the image key point and the reference key point, and the smaller the distance value, the farther the distance between the image key point and the reference key point; therefore, based on the distance value in the distance matrix, the distances can be sorted from large to small according to the distance value, and the distance at the head of the queue can be selected as the target distance, that is, a larger distance value is selected for subsequent determination of the distance submatrix.

[0110] In a specific implementation of the present specification, the distance values ​​of each distance in the distance matrix are determined: distance d1 = 0.8, d2 = 0.4, d3 = 0.7; the distances are sorted based on the distance values, so that the distance d1 with the larger distance value is determined as the target distance.

[0111] Further, in a preferred embodiment of the present specification, the method of screening a distance sub-matrix in the distance matrix based on the target distance may include:

[0112] Determining at least two sub-matrices to be processed in the distance matrix according to a matrix distance threshold and the target distance;

[0113] Sort the sub-matrices to be processed based on the sum of the distance values ​​in the sub-matrices to be processed;

[0114] The distance sub-matrix is ​​selected from each sub-matrix to be processed according to the sorting result.

[0115] Among them, the matrix distance threshold refers to the specification threshold of the distance submatrix. For example, the matrix distance threshold can be 4*4, that is, a distance submatrix with a specification of 4*4 is selected in the distance matrix; the submatrix to be processed refers to the submatrix selected in the distance matrix according to the matrix distance threshold and the target distance.

[0116] Specifically, determine the matrix distance threshold and the target distance, wherein the matrix distance threshold can be set based on demand and is not specifically limited in this specification; select multiple sub-matrices to be processed according to the matrix distance threshold and the target distance; calculate the sum of the distance values ​​according to the distance values ​​corresponding to the distances in each sub-matrix to be processed; and then sort each sub-matrix to be processed based on the sum of the distance values ​​corresponding to each sub-matrix to be processed; since the larger the distance value, the closer the distance between the reference key point and the image key point, the sub-matrix to be processed with a larger sum of distance values ​​can be selected as the distance sub-matrix.

[0117] In a specific embodiment of this specification, see Figure 4 , Figure 4It is a schematic diagram of a submatrix to be processed provided by an embodiment of the present specification. According to the matrix distance threshold 2*2 and the target distance d10, submatrix 1 to be processed, submatrix 2 to be processed, submatrix 3 to be processed and submatrix 4 to be processed are determined in distance matrix A; the sum of the distance values ​​corresponding to the submatrix to be processed is calculated based on the distance values ​​in each submatrix to be processed, which are respectively: the sum of the distance values ​​of submatrix 1 to be processed is 0.8+0.8+0.8+0.9=3.3, the sum of the distance values ​​of submatrix 2 to be processed is 0.8+0.7+0.9+0.7=3.1, the sum of the distance values ​​of submatrix 3 to be processed is 0.8+0.9+0.6+0.6=2.9, and the sum of the distance values ​​of submatrix 4 to be processed is 0.9+0.7+0.6+0.7=2.9; if it is determined that the sum of the distance values ​​of submatrix 1 to be processed is larger, it is used as the distance submatrix of distance matrix A, thereby completing the screening of the distance submatrix.

[0118] By determining the sum of the distance values ​​corresponding to the submatrix to be processed, and then screening the distance submatrix in the submatrix to be processed based on the sum of the distance values, the accuracy of the distance submatrix is ​​ensured, and then the subsequent image key point calculation can be performed based on the distance submatrix with higher accuracy, thereby improving the accuracy of the image key points.

[0119] Further, determining the image key points according to the distance sub-matrix and the reference key points corresponding to the distance sub-matrix includes:

[0120] Determine the reference key point coordinates of the reference key point corresponding to the distance sub-matrix;

[0121] The coordinates of the image key points are calculated based on the reference key point coordinates and the distances in the distance sub-matrix.

[0122] The reference key point coordinates refer to the coordinates of the reference key points in the image to be detected.

[0123] Specifically, the coordinates of the reference key points and the distance values ​​of each distance in the distance submatrix are determined, and the coordinates of the image key points in the image to be detected are calculated by using the least square method, the coordinates of each reference key point and each distance value.

[0124] In a specific implementation of this specification, the number of reference key points is determined to be 3, that is, the reference key points and coordinates corresponding to the distance submatrix are a0(x0, y0), a1(x1, y1) and a2(x2, y2), respectively; the distance values ​​in the distance submatrix are determined to be d0, d1 and d2, respectively; the coordinates of the image key point s0 can be accurately calculated as (u, v) by combining the coordinates of the reference key points, the distance values ​​and the following formula 1:

[0125]

[0126] In another specific implementation of the present specification, the number of reference key points is determined to be n, that is, the reference key points and coordinates corresponding to the distance sub-matrix are a0(x0, y0), a1(x1, y1) ... an(xn, yn), respectively; the distance values ​​in the distance sub-matrix are determined to be d0, d1 ... dn, respectively; the coordinates of the reference key points, the distance values ​​and the following formulas 2 to 7 can be accurately calculated to be β′:

[0127]

[0128] Y=βX Formula 3

[0129]

[0130]

[0131]

[0132] β′=(X T X) -1 X T Y formula 7

[0133] The coordinates of the key points of the image are calculated by referring to the coordinates of the key points and the distances in the distance submatrix, thereby avoiding the calculation loss caused by calculating the coordinates of the key points of the image through decoding, thereby improving the accuracy of the coordinates of the key points of the image.

[0134] In practical applications, in order to enable the distance prediction module to have the function of predicting the distance matrix, the distance prediction module needs to be trained. Specifically, the distance prediction module is trained based on the following steps:

[0135] Determine a target sample pair in the image sample set, wherein the target sample pair is composed of a target sample image and a sample distance matrix of the target sample image;

[0136] Inputting the target sample image into a distance prediction module to obtain a predicted distance matrix output by the distance prediction module;

[0137] Calculate the model loss value according to the prediction distance matrix and the sample distance matrix;

[0138] The model parameters of the distance prediction module are adjusted based on the model loss value until the model training stop condition is reached.

[0139] Among them, the target sample pair refers to a sample pair consisting of a target sample image and its corresponding sample distance; the target sample pair is any sample pair in the image sample set; the target sample image refers to a sample image used to train the distance prediction module; the sample distance matrix is ​​a matrix generated based on the distance between the sample key points and the reference key points of the target sample image; the predicted distance matrix refers to the distance matrix of the target sample image predicted by the distance prediction module; the model loss value is a loss value used to adjust the parameters of the distance prediction module.

[0140] Specifically, a sample pair is arbitrarily selected from an image sample set as a target sample pair, wherein the target sample pair includes a target sample image and a sample distance matrix corresponding to the target sample image; the target sample image is input into a distance prediction module to obtain a predicted distance matrix predicted by the distance prediction module; based on the predicted distance matrix and the sample distance matrix in the target sample pair, a model loss value is calculated; and according to the model loss value, a model parameter of the distance prediction module is adjusted until a trained distance prediction module is obtained.

[0141] In a specific implementation of the present specification, a sample pair a is determined in an image sample set, wherein the sample pair a includes a sample image and a corresponding sample distance matrix; the sample image is input into a feature extraction submodule of a distance prediction module to obtain a set of predicted reference points; the set of predicted reference points and a preset number of key points are input into a distance prediction submodule of the distance prediction module to obtain a predicted distance matrix; a model loss value is calculated based on the predicted distance matrix and the sample distance matrix in the sample pair a; model parameters of the distance prediction module are adjusted based on the model loss value; and the above steps are repeated until a trained distance prediction module is obtained.

[0142] The distance prediction module is trained by presetting the sample pairs of the sample distance matrix, so that the distance prediction module has the function of outputting the corresponding distance matrix based on the image.

[0143] In practical applications, it is necessary to determine the sample distance matrix of the target sample image in advance and then generate sample pairs. That is, before determining the target sample pairs in the image sample set, it also includes:

[0144] Determine a target sample image and sample key points corresponding to the target sample image;

[0145] Generate a sample distance matrix based on the sample key points and the reference key points of the target sample image;

[0146] The sample distance matrix is ​​added to the target sample image to obtain a target sample pair.

[0147] The sample key points refer to the real key points of the target sample image, for example, the image key points of the nose object in the face image are annotated by manual annotation.

[0148] Specifically, the sample key points and reference key points of the target sample image are determined; the distance between the sample key points and the corresponding reference key points is calculated based on the sample key point coordinates and the reference key point coordinates to obtain a sample distance matrix; in practical applications, the true distance between the sample key points and the corresponding reference key points can be calculated based on the sample key point coordinates and the reference key point coordinates, and the sample distance matrix is ​​obtained by Gaussian encoding the calculated true distance; a target sample pair is generated based on the sample distance matrix and the target sample image, and the target sample pair is added to the image sample set.

[0149] The sample distance matrix corresponding to the target sample image is generated through the sample key points and reference key points of the target sample image, thereby enriching the sample labels of the target sample image, so that the distance prediction module can be trained based on the target sample image carrying the sample distance matrix.

[0150] The key point detection method of the present specification obtains an image to be detected, wherein the image to be detected contains an object to be detected; the image to be detected is input into a key point detection model to obtain image key points of the object to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines the image key points based on the distance matrix.

[0151] By inputting the image to be detected into the key point detection model, the key point detection model predicts the distance matrix between the reference key points and the image key points, and then determines the image key points based on the distance matrix, avoiding the loss caused by decoding when determining the image key points, and improving the accuracy of the image key points.

[0152] See also Figure 5 , Figure 5 A flowchart of a distance prediction module training method provided by an embodiment of the present specification is shown, which is applied to a cloud-side device and specifically includes the following steps:

[0153] Step 502: Determine a target sample pair in an image sample set, wherein the target sample pair is composed of a target sample image and a sample distance matrix of the target sample image.

[0154] Specifically, before determining the target sample pair in the image sample set, it also includes: determining the target sample image and the sample key points corresponding to the target sample image; generating a sample distance matrix based on the sample key points and the reference key points of the target sample image; adding the sample distance matrix to the target sample image to obtain a target sample pair; and generating an image sample set based on the target sample pair.

[0155] Step 504: input the target sample image into a distance prediction module to obtain a predicted distance matrix output by the distance prediction module.

[0156] Step 506: Calculate the prediction loss value according to the prediction distance matrix and the sample distance matrix.

[0157] Step 508: Adjust the model parameters of the distance prediction module based on the predicted loss value until the model training stop condition is reached, thereby obtaining the model parameters of the distance prediction module.

[0158] Step 510: Send the model parameters of the distance prediction module to the terminal side device.

[0159] Specifically, Figure 6 As shown, Figure 6 It is a schematic diagram of a training distance prediction module provided by an embodiment of the present specification, wherein a sample image is input into the distance prediction module, and a predicted distance matrix of the sample image output by the distance prediction module is obtained, wherein the predicted distance matrix includes the distances dis0, dis1, dis2, and dis3 between the predicted key points of the sample image and the reference key points; the real sample key point coordinates of the sample image and the coordinates of each reference key point are determined, and the distance between the key point and the reference key point is calculated based on the sample key point coordinates and the corresponding reference key point coordinates, and the sample distance matrix is ​​obtained by Gaussian encoding the calculated distance between the sample key point and the reference key point, wherein the sample distance matrix includes the actual distances dis4, dis5, dis6, and dis7 between the key point and each reference key point; the model loss value is calculated based on the predicted distance matrix and the sample distance matrix of the sample image, and the model parameters of the distance prediction module are adjusted based on the model loss value until a trained distance prediction module is obtained.

[0160] The training method of the distance prediction module of the present specification determines a target sample pair in an image sample set, wherein the target sample pair is composed of a target sample image and a sample distance matrix of the target sample image; inputs the target sample image into the distance prediction module to obtain a predicted distance matrix output by the distance prediction module; calculates a predicted loss value based on the predicted distance matrix and the sample distance matrix; adjusts the model parameters of the distance prediction module based on the predicted loss value until a model training stop condition is reached to obtain the model parameters of the distance prediction module; and sends the model parameters of the distance prediction module to an end-side device.

[0161] The distance prediction module is trained by using a target sample image that carries a sample distance matrix, so that the distance prediction module has the function of outputting the distance matrix corresponding to the image; the cloud-side device completes the training of the distance prediction module, and returns the model parameters of the trained distance prediction module to the client for the client to call the trained distance prediction module, thereby saving the client's computing resources and occupied space and improving the client's processing efficiency.

[0162] See also Figure 7 , Figure 7 Another key point detection method provided by an embodiment of the present specification is shown, which is applied to a cloud side device and specifically includes the following steps:

[0163] Step 702: receiving an image to be detected sent by a client, wherein the image to be detected includes an object to be detected.

[0164] Step 704: Input the image to be detected into a key point detection model to obtain the image key points of the object to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines the image key points based on the distance matrix.

[0165] Step 706: Return the image key points to the client.

[0166] The key point detection method applied to a cloud-side device of the present specification receives an image to be detected sent by a client, wherein the image to be detected contains an object to be detected; inputs the image to be detected into a key point detection model to obtain image key points of the object to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines image key points based on the distance matrix; and returns the image key points to the client.

[0167] After the key points of the image are determined by the cloud-side device, the key points of the image are returned to the client, thereby realizing the calculation of the key points of the image on the cloud-side device, saving the computing resources of the client and thus improving the processing efficiency of the client.

[0168] The following combination Figure 8 , taking the application of the key point detection method provided in this specification to a face image as an example, the key point detection method is further described. Figure 8 A flowchart of a method for detecting key points in a face image provided by an embodiment of the present specification is shown, which specifically includes the following steps:

[0169] Step 802: Acquire a face image to be detected, wherein the face to be detected is included in the image to be detected.

[0170] Step 804: Input the face image to be detected into a key point detection model to obtain the image key points of the face to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines the image key points based on the distance matrix.

[0171] The method for detecting key points of a face image of the present specification obtains a face image to be detected, wherein the face to be detected is contained in the face to be detected; the face image to be detected is input into a key point detection model to obtain image key points of the face to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines image key points based on the distance matrix. By inputting a face image into a key point detection model, the key point detection model predicts a distance matrix between reference key points and image key points, thereby determining image key points based on the distance matrix, avoiding losses caused by decoding when determining image key points, and improving the accuracy of image key points.

[0172] Corresponding to the above method embodiment, this specification also provides a key point detection device embodiment, Figure 8 FIG. 2 shows a schematic diagram of a key point detection device provided by an embodiment of the present specification. Figure 8 As shown, the device comprises:

[0173] The acquisition module 902 is configured to acquire an image to be detected, wherein the image to be detected contains an object to be detected;

[0174] The input module 904 is configured to input the image to be detected into the key point detection model to obtain the image key points of the object to be detected, wherein the key point detection model predicts the distance matrix between the reference key points and the image key points, and determines the image key points based on the distance matrix.

[0175] Optionally, the key point detection model includes a distance prediction module and a key point calculation module;

[0176] The input module 904 is further configured to:

[0177] Inputting the image to be detected into the distance prediction module to obtain a distance matrix output by the distance prediction module;

[0178] The distance matrix is ​​input into the key point calculation module to obtain the image key points output by the key point calculation module.

[0179] Optionally, the distance prediction module includes a feature extraction submodule and a distance prediction submodule;

[0180] The input module 904 is further configured to:

[0181] Inputting the image to be detected into the feature extraction submodule, and obtaining a reference point set corresponding to the image to be detected based on a preset number of reference points;

[0182] The preset number of key points and the reference point set are input into the distance prediction submodule to obtain a distance matrix.

[0183] Optionally, the input module 904 is further configured to:

[0184] determining a target distance in the distance matrix;

[0185] Filtering a distance sub-matrix in the distance matrix based on the target distance;

[0186] The image key points are determined according to the distance sub-matrix and the reference key points corresponding to the distance sub-matrix.

[0187] Optionally, the input module 904 is further configured to:

[0188] Sorting according to the distance value of each distance in the distance matrix;

[0189] A target distance is determined among the distances based on the sorting result.

[0190] Optionally, the input module 904 is further configured to:

[0191] Determining at least two sub-matrices to be processed in the distance matrix according to a matrix distance threshold and the target distance;

[0192] Sort the sub-matrices to be processed based on the sum of the distance values ​​in the sub-matrices to be processed;

[0193] The distance sub-matrix is ​​selected from each sub-matrix to be processed according to the sorting result.

[0194] Optionally, the input module 904 is further configured to:

[0195] Determine the reference key point coordinates of the reference key point corresponding to the distance sub-matrix;

[0196] The coordinates of the image key points are calculated based on the reference key point coordinates and the distances in the distance sub-matrix.

[0197] Optionally, the device further comprises a training module, wherein the training module is configured to:

[0198] Determine a target sample pair in the image sample set, wherein the target sample pair is composed of a target sample image and a sample distance matrix of the target sample image;

[0199] Inputting the target sample image into a distance prediction module to obtain a predicted distance matrix output by the distance prediction module;

[0200] Calculate the model loss value according to the prediction distance matrix and the sample distance matrix;

[0201] The model parameters of the distance prediction module are adjusted based on the model loss value until the model training stop condition is reached.

[0202] Optionally, the device further comprises a generating module, and the training module is configured to:

[0203] Determine a target sample image and sample key points corresponding to the target sample image;

[0204] Generate a sample distance matrix based on the sample key points and the reference key points of the target sample image;

[0205] The sample distance matrix is ​​added to the target sample image to obtain a target sample pair.

[0206] The key point detection device of this specification, the acquisition module, is configured to acquire the image to be detected, wherein the image to be detected contains the object to be detected; the input module is configured to input the image to be detected into the key point detection model, obtain the image key points of the object to be detected, wherein the key point detection model predicts the distance matrix between the reference key points and the image key points, and determines the image key points according to the distance matrix. By inputting the image to be detected into the key point detection model, the key point detection model predicts the distance matrix between the reference key points and the image key points, thereby determining the image key points based on the distance matrix, avoiding the loss caused by decoding when determining the image key points, and improving the accuracy of the image key points.

[0207] The above is a schematic scheme of a key point detection device of this embodiment. It should be noted that the technical scheme of the key point detection device and the technical scheme of the key point detection method described above belong to the same concept, and the details not described in detail in the technical scheme of the key point detection device can be referred to the description of the technical scheme of the key point detection method described above.

[0208] Fig.10 The block diagram of a computing device 1000 according to an embodiment of the present specification is shown. The components of the computing device 1000 include but are not limited to a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and the database 1050 is used to store data.

[0209] The computing device 1000 also includes an access device 1040 that enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1040 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0210] In one embodiment of the present specification, the above components of the computing device 1000 and Fig.10 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.10 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0211] The computing device 1000 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 1000 may also be a mobile or stationary server.

[0212] The processor 1020 is used to execute the following computer executable instructions, which, when executed by the processor, implement the steps of the above-mentioned key point detection method.

[0213] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above key point detection method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above key point detection method.

[0214] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned key point detection method.

[0215] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the key point detection method described above are of the same concept, and the details not described in detail in the technical scheme of the storage medium can be found in the description of the technical scheme of the key point detection method described above.

[0216] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned key point detection method.

[0217] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the key point detection method described above are of the same concept, and the details not described in detail in the technical scheme of the computer program can be found in the description of the technical scheme of the key point detection method described above.

[0218] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0219] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0220] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0221] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0222] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A key point detection method, comprising: Acquire an image to be detected, wherein the image to be detected contains an object to be detected; The image to be detected is input into a key point detection model to obtain the image key points of the object to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines the image key points according to the distance matrix.

2. The method according to claim 1, wherein the key point detection model comprises a distance prediction module and a key point calculation module; Inputting the image to be detected into a key point detection model to obtain the image key points of the object to be detected, including: Inputting the image to be detected into the distance prediction module to obtain a distance matrix output by the distance prediction module; The distance matrix is ​​input into the key point calculation module to obtain the image key points output by the key point calculation module.

3. The method according to claim 2, wherein the distance prediction module comprises a feature extraction submodule and a distance prediction submodule; Inputting the image to be detected into the distance prediction module to obtain a distance matrix output by the distance prediction module includes: Inputting the image to be detected into the feature extraction submodule, and obtaining a reference point set corresponding to the image to be detected based on a preset number of reference points; The preset number of key points and the reference point set are input into the distance prediction submodule to obtain a distance matrix.

4. The method according to claim 2, inputting the distance matrix into the key point calculation module to obtain the image key points output by the key point calculation module, comprises: determining a target distance in the distance matrix; Filtering a distance sub-matrix in the distance matrix based on the target distance; The image key points are determined according to the distance sub-matrix and the reference key points corresponding to the distance sub-matrix.

5. The method of claim 4, wherein determining the target distance in the distance matrix comprises: Sorting according to the distance value of each distance in the distance matrix; A target distance is determined among the distances based on the sorting result.

6. The method of claim 4, wherein screening a distance sub-matrix in the distance matrix based on the target distance comprises: Determining at least two sub-matrices to be processed in the distance matrix according to a matrix distance threshold and the target distance; Sort the sub-matrices to be processed based on the sum of the distance values ​​in the sub-matrices to be processed; The distance sub-matrix is ​​selected from each sub-matrix to be processed according to the sorting result.

7. The method of claim 4, wherein determining the image key points according to the distance sub-matrix and the reference key points corresponding to the distance sub-matrix comprises: Determine the reference key point coordinates of the reference key point corresponding to the distance sub-matrix; The coordinates of the image key points are calculated based on the reference key point coordinates and the distances in the distance sub-matrix.

8. The method according to claim 2, wherein the distance prediction module is trained based on the following steps: Determine the target sample pair in the image sample set, where The target sample pair is composed of a target sample image and a sample distance matrix of the target sample image; Inputting the target sample image into a distance prediction module to obtain a predicted distance matrix output by the distance prediction module; Calculate the model loss value according to the prediction distance matrix and the sample distance matrix; The model parameters of the distance prediction module are adjusted based on the model loss value until the model training stop condition is reached.

9. The method according to claim 8, before determining the target sample pair in the image sample set, further comprising: Determine a target sample image and sample key points corresponding to the target sample image; Generate a sample distance matrix based on the sample key points and the reference key points of the target sample image; The sample distance matrix is ​​added to the target sample image to obtain a target sample pair.

10. A training method for a distance prediction module, applied to a cloud-side device, comprising: Determine a target sample pair in the image sample set, wherein the target sample pair is composed of a target sample image and a sample distance matrix of the target sample image; Inputting the target sample image into a distance prediction module to obtain a predicted distance matrix output by the distance prediction module; Calculate the prediction loss value according to the prediction distance matrix and the sample distance matrix; Adjusting the model parameters of the distance prediction module based on the predicted loss value until a model training stop condition is reached to obtain the model parameters of the distance prediction module; The model parameters of the distance prediction module are sent to the end-side device.

11. A key point detection method, applied to a cloud-side device, comprising: Receiving an image to be detected sent by a client, wherein the image to be detected contains an object to be detected; Inputting the image to be detected into a key point detection model to obtain the image key points of the object to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines the image key points according to the distance matrix; The image key points are returned to the client.

12. A method for detecting key points of a face image, comprising: Acquire a face image to be detected, wherein the face to be detected is contained in the image to be detected; The face image to be detected is input into a key point detection model to obtain the image key points of the face to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines the image key points according to the distance matrix.

13. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 12 are implemented.

14. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 12.

Images