Training methods, devices, electronic equipment, and storage media for keypoint detection models

Abstract

This invention relates to a training method, apparatus, electronic device, and storage medium for a keypoint detection model. The method includes: acquiring a dense training sample set and a sparse training sample set, wherein each training sample in the dense training sample set is labeled with a dense set of keypoints, and each training sample in the sparse training sample set is labeled with a sparse set of keypoints; training an initial model using the dense training sample set to obtain a first model; and constructing a keypoint detection model using the sparse training sample set and the first model, wherein the keypoint detection model is used to detect keypoints in an image. This improves the accuracy of the keypoint detection model.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a training method, apparatus, electronic device and storage medium for a key point detection model. Background Technology

[0002] Facial landmark detection refers to the precise localization of facial regions, including facial features and contours such as cheeks, eyebrows, eyes, mouth, and nose. Since the first application of CNN (Convolutional Neural Network) technology to facial landmark detection in 2013, this technology has undergone rapid development, and the technical methods have become relatively mature. Both fast detection technologies applied to mobile devices and high-precision detection technologies applied to the cloud have achieved rapid progress.

[0003] With the development of facial landmark technology, it has been applied to various scenarios. For example, the landmark technology can be used to divide the face into areas such as eyebrows, eyes, mouth, and cheeks to achieve facial beautification effects. The outer contour area of ​​the face can be used for face slimming. In addition, there are many other application scenarios, which are widely distributed in the fields of short videos, long videos, and photography.

[0004] However, existing facial landmark detection technologies still suffer from inaccuracies and jitter in some landmarks in practical applications, especially in edge-test models. To meet the speed requirements of edge testing, the models are often lightweight. For example, with 106 landmarks, although the landmarks are marked on the edge of the face contour, the discreteness of the landmarks means that the gaps between the landmarks are large. This makes it difficult for the landmarks extracted by the landmark detection model to express accurate semantic information, resulting in low accuracy and a tendency to make errors. Summary of the Invention

[0005] In view of this, in order to solve the technical problem that the key points extracted by the key point detection model in the prior art often cannot express accurate semantic information, have low accuracy, and are prone to deviation, the embodiments of the present invention provide a training method, device, electronic device and storage medium for a key point detection model.

[0006] In a first aspect, embodiments of the present invention provide a training method for a keypoint detection model, the method comprising:

[0007] Obtain a dense training sample set and a sparse training sample set, wherein each training sample in the dense training sample set is labeled with a dense keypoint set, and each training sample in the sparse training sample set is labeled with a sparse keypoint set.

[0008] The initial model is trained using the dense training sample set to obtain the first model;

[0009] Using the sparse training sample set and the first model, a key point detection model is constructed, which is used to detect key points in an image.

[0010] As one possible implementation, constructing a keypoint detection model using the sparse training sample set and the first model includes:

[0011] The basic network layers in the initial model are re-initialized using the basic network layers in the first model to obtain the re-initialized initial model;

[0012] The re-initialized initial model is trained using the sparse training sample set to obtain the key point detection model.

[0013] As one possible implementation, training the initial model using the dense training sample set to obtain the first model includes:

[0014] The base network layer and dense keypoint prediction layer in the initial model are trained using the dense training sample set to obtain the first model;

[0015] Using the sparse training sample set and the first model, a keypoint detection model is constructed, including:

[0016] The sparse training sample set is used to train the basic network layer and sparse keypoint prediction layer in the first model to obtain the second model;

[0017] Based on the second model, a key point detection model is constructed.

[0018] As one possible implementation, the keypoint detection model is constructed based on the second model, including:

[0019] The basic network layer and sparse keypoint prediction layer in the second model are used to construct a keypoint detection model.

[0020] Secondly, embodiments of the present invention provide a method for detecting key points, the method comprising:

[0021] Acquire the target image;

[0022] The target image is input into a keypoint detection model trained according to any one of the methods described in the first aspect above, to obtain a keypoint set of the target image.

[0023] Thirdly, embodiments of the present invention provide a training apparatus for a keypoint detection model, the apparatus comprising:

[0024] The acquisition module is used to acquire a dense training sample set and a sparse training sample set. Each training sample in the dense training sample set is labeled with a dense keypoint set, and each training sample in the sparse training sample set is labeled with a sparse keypoint set.

[0025] The training module is used to train the initial model using the dense training sample set to obtain the first model;

[0026] A construction module is used to construct a key point detection model using the sparse training sample set and the first model, the key point detection model being used to detect key points in an image.

[0027] As one possible implementation, the building module includes:

[0028] An initialization submodule is used to reinitialize the basic network layers in the initial model using the basic network layers in the first model, so as to obtain a reinitialized initial model.

[0029] The first training submodule is used to train the re-initialized initial model using the sparse training sample set to obtain a key point detection model.

[0030] As one possible implementation, the training module includes:

[0031] The second training submodule is used to train the basic network layer and dense key point prediction layer in the initial model using the dense training sample set to obtain the first model;

[0032] The building module includes:

[0033] The third training submodule is used to train the basic network layer and sparse keypoint prediction layer in the first model using the sparse training sample set to obtain the second model.

[0034] A submodule is constructed to build a keypoint detection model based on the second model.

[0035] As one possible implementation, the construction submodule is specifically used for:

[0036] The basic network layer and sparse keypoint prediction layer in the second model are used to construct a keypoint detection model.

[0037] Fourthly, embodiments of the present invention provide a key point detection device, the device comprising:

[0038] The image acquisition module is used to acquire the target image;

[0039] An input module is used to input the target image into a keypoint detection model trained according to any one of the methods described in the first aspect above, to obtain a keypoint set of the target image.

[0040] Fifthly, embodiments of the present invention provide an electronic device, including: a processor and a memory, wherein the processor is configured to execute a training program for a keypoint detection model or a keypoint detection program stored in the memory, so as to implement the training method for the keypoint detection model as described in the first aspect, or the keypoint detection method as described in the second aspect.

[0041] In a sixth aspect, embodiments of the present invention provide a storage medium storing one or more programs, which can be executed by one or more processors to implement the training method of the key point detection model described in any one aspect of the first aspect, or the key point detection method described in the second aspect.

[0042] The technical solution provided by this invention involves acquiring a dense training sample set and a sparse training sample set. An initial model is trained using the dense training sample set to obtain a first model. Then, a keypoint detection model is constructed using the sparse training sample set and the first model. This keypoint detection model is used to detect keypoints in an image. In this technical solution, when training the keypoint detection model, the initial model is first trained using a dense training sample set, and then trained using a sparse training sample set. The base network layer trained with the dense training sample set can more effectively constrain the position of keypoints, making the keypoints extracted from the target image by the obtained keypoint detection model more accurate, thus improving the accuracy of the keypoint detection model. Attached Figure Description

[0043] Figure 1 A flowchart illustrating an embodiment of a training method for a keypoint detection model provided by the present invention;

[0044] Figure 2 is a schematic diagram of facial key points provided in an embodiment of the present invention;

[0045] Figure 3 A flowchart illustrating an embodiment of a key point detection method provided by this invention;

[0046] Figure 4 This is a schematic diagram of the initial model structure of a key point detection model provided in an embodiment of the present invention;

[0047] Figure 5 A block diagram illustrating an embodiment of a training device for a keypoint detection model provided in this invention;

[0048] Figure 6A block diagram illustrating an embodiment of a key point detection device provided in this invention;

[0049] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] The training method of the key point detection model provided by the present invention will be further explained below with reference to the accompanying drawings and specific embodiments. The embodiments do not constitute a limitation on the embodiments of the present invention.

[0052] See Figure 1 This is a flowchart illustrating an embodiment of a training method for a keypoint detection model provided by an embodiment of the present invention. Figure 1 As shown, the process may include the following steps:

[0053] Step 101: Obtain the dense training sample set and the sparse training sample set.

[0054] Each training sample in the aforementioned dense training sample set is labeled with a dense set of keypoints. For example, in the facial keypoint training sample set, the training sample can be labeled with a dense set of facial keypoints, for example... Figure 2A The 826 points of the face shown.

[0055] Each training sample in the aforementioned sparse training sample set is labeled with a sparse set of keypoints. For example, in the face keypoint training sample set, the training samples can be labeled with a sparse set of face keypoints, such as... Figure 2B The face shown has 106 key points. The number of key points in the dense keypoint cluster is greater than the number of key points in the sparse keypoint cluster.

[0056] In one embodiment, the executing entity of this invention may obtain the aforementioned dense training sample set and sparse training sample set from a preset database.

[0057] Step 102: Train the initial model using a dense training sample set to obtain the first model.

[0058] Step 103: Using the sparse training sample set and the first model, construct a key point detection model, which is used to detect key points in the image.

[0059] The following provides a unified explanation of steps 102 and 103:

[0060] The aforementioned keypoint detection model is used to detect keypoints in images, such as facial keypoints in face images.

[0061] The initial model mentioned above refers to the untrained model.

[0062] In one embodiment, the initial model may include a base network layer and a fully connected layer. The base network layer, which may be a MobileNetV3 model, is used to extract basic image features. The fully connected layer is used to reduce the dimensionality of the extracted image features.

[0063] The executing entity of this invention can use the aforementioned dense training sample set to train the aforementioned basic network layer and fully connected layer to obtain the trained first model. Therefore, the first model includes the basic network layer and fully connected layer trained using the dense training sample set.

[0064] Furthermore, the executing entity of this embodiment can re-initialize the basic network layers in the initial model using the basic network layers in the first model described above, to obtain a re-initialized initial model. It is understood that the re-initialized initial model may include basic network layers trained using a dense training sample set and untrained fully connected layers. Subsequently, the re-initialized initial model can be trained using the obtained sparse training sample set to obtain a keypoint detection model.

[0065] Since the base network layers in the re-initialized initial model are trained using a dense training sample set, and these base network layers are responsible for extracting basic features from the image, training the re-initialized initial model using a sparse training sample set not only accelerates the convergence speed of the keypoint detection model but also effectively constrains the positions of the keypoints. For example, when training a face keypoint detection model, it can make the positions of the identified keypoints more closely match the facial contour edges.

[0066] In another embodiment, the initial model may include a base network layer, a dense keypoint prediction layer, and a sparse keypoint prediction layer. The dense keypoint prediction layer can be used to train a dense keypoint detection model, and the sparse keypoint prediction layer can be used to train a sparse keypoint detection model.

[0067] Based on this, the execution entity of this embodiment of the invention can use the aforementioned dense training sample set to train the base network layer and dense keypoint prediction layer in the initial model to obtain the trained first model. It is understood that the first model here includes the base network layer and dense keypoint prediction layer trained using the dense training sample set, as well as the untrained sparse keypoint prediction layer.

[0068] Then, the base network layer and sparse keypoint prediction layer in the first model can be trained using a sparse training sample set to obtain the second model. Based on this second model, a keypoint detection model is constructed.

[0069] Optionally, the second model may include a dense keypoint prediction layer trained using a dense training sample set, a base network layer trained first using a dense training sample set and then using a sparse training sample set, and a sparse keypoint prediction layer trained using a sparse training sample set.

[0070] Because the base network layer trained on a dense training sample set can more effectively constrain the location of keypoints compared to the base network layer trained on a sparse training sample set, the base network layer and sparse keypoint prediction layer in the second model can be directly used to construct a keypoint detection model.

[0071] The technical solution provided by this invention involves acquiring a dense training sample set and a sparse training sample set. An initial model is trained using the dense training sample set to obtain a first model. Then, a keypoint detection model is constructed using the sparse training sample set and the first model. This keypoint detection model is used to detect keypoints in an image. In this technical solution, when training the keypoint detection model, the initial model is first trained using a dense training sample set, and then trained using a sparse training sample set. The base network layer trained with the dense training sample set can more effectively constrain the position of keypoints, making the keypoints extracted from the target image by the obtained keypoint detection model more accurate, thus improving the accuracy of the keypoint detection model.

[0072] See Figure 3 The above is a flowchart of an embodiment of a key point detection method provided by the present invention. Figure 3 The process shown is in Figure 1 Based on the process shown, using Figure 1 The keypoint detection model obtained from the illustrated process enables the detection of keypoints. For example... Figure 3 As shown, the process may include the following steps:

[0073] Step 301: Obtain the target image.

[0074] The target image mentioned above refers to the acquired object image for which key point extraction is to be performed.

[0075] In one embodiment, the executing entity of this invention may be an electronic device, which may be hardware or software that supports network connectivity to provide various network services. When the control terminal is hardware, it may be various electronic devices with displays, including but not limited to smartphones, tablets, laptops, desktop computers, etc. When the control terminal is software, it may be installed in the electronic devices listed above.

[0076] In one embodiment, the executing entity of this invention can acquire the target image using a preset image acquisition device.

[0077] In another embodiment, the executing entity of this invention may obtain the target image input by the user.

[0078] Step 302: Input the target image into the trained key point detection model to obtain the key point set of the target image.

[0079] The above key point detection model can be used for Figure 1 The process shown demonstrates the training of a keypoint detection model. This model can be applied to smart terminals to detect sparse keypoint sets in target images.

[0080] Based on this, the execution subject of this embodiment of the invention can input the acquired target image into the key point detection model to obtain the key point set of the target image.

[0081] The technical solution provided in this embodiment of the invention acquires a target image and inputs the target image into... Figure 1 The keypoint detection model trained using the illustrated process yields a keypoint set for the target image. This technical solution, by inputting the target image into the trained keypoint detection model (which is trained on a dense training sample set followed by a sparse training sample set), effectively constrains the positions of keypoints, thus improving the accuracy of the keypoint detection model. This achieves more accurate extraction of the keypoint set from the target image.

[0082] See Figure 4 This is a schematic diagram of the initial model structure of a key point detection model provided in an embodiment of the present invention. Figure 4 As shown, the initial model structure may include the following:

[0083] Existing sparse keypoint detection models may include Figure 4 The diagram shows a basic network layer and a sparse keypoint prediction layer, which may include a fully connected layer and a sparse keypoint prediction result module.

[0084] Based on this, Figure 4 The structure shown adds a dense keypoint prediction layer, which may include a fully connected layer and a dense keypoint prediction result module.

[0085] In this context, D can represent a higher feature dimension for keypoints, N1 represents a sparse keypoint feature dimension, and N2 represents a dense keypoint feature dimension. For example, with 826 dense keypoints and 106 sparse keypoints, N2 could be 826*2, meaning 826 dense keypoints are predicted, including both x and y coordinate parameters; N1 could be 106*2, meaning 106 points are predicted with their x and y coordinates. The feature dimension D is typically 256.

[0086] Training the keypoint detection model based on the initial model described above can be divided into two stages: In the first stage, a dense training sample set can be used to train the dense keypoint detection model, without adding any sparse training sample set. During training, only the base network layers and the dense keypoint prediction layer can be trained, without training the sparse keypoint prediction layer. This continues until the initial model fully converges on the dense training sample set.

[0087] In the second stage, after the dense keypoint prediction layer has been trained, it can be deleted. The keypoint detection model is then trained using only the base network layer and the sparse keypoint prediction layer. During training, only the sparse training sample set is used; no dense training sample set is added. Furthermore, the base network layer trained in the first stage is used to initialize the base network layer in this training process.

[0088] The initial model and training method for the keypoint detection model provided in this invention involve first training the base network layer and the dense keypoint prediction layer using a dense training sample set. After the dense keypoint prediction layer converges, it is deleted, and then a sparse keypoint prediction layer is trained. During training, the base network layer trained with the dense training sample set is used to initialize the model, and then the sparse keypoint detection layer is trained using the sparse training sample set until the model converges. This method, based on the learned dense training sample set, then learns from the sparse training sample set, which can effectively accelerate the convergence speed of the sparse keypoint prediction layer training. It also effectively constrains the positions of discrete points, strengthening the spatial distance and positional constraints of discrete points, thereby improving the accuracy of the keypoint detection model. Furthermore, it has strong applicability, does not modify the model structure in the inference stage, and only affects the training process.

[0089] See Figure 5 This is a block diagram illustrating an embodiment of a training device for a keypoint detection model provided by an embodiment of the present invention. Figure 5 As shown, the device includes:

[0090] The acquisition module 51 is used to acquire a dense training sample set and a sparse training sample set, wherein each training sample in the dense training sample set is labeled with a dense key point set, and each training sample in the sparse training sample set is labeled with a sparse key point set.

[0091] Training module 52 is used to train the initial model using the dense training sample set to obtain the first model;

[0092] The construction module 53 is used to construct a key point detection model using the sparse training sample set and the first model, the key point detection model being used to detect key points in an image.

[0093] As one possible implementation, the building module 53 includes (not shown in the figure):

[0094] An initialization submodule is used to reinitialize the basic network layers in the initial model using the basic network layers in the first model, so as to obtain a reinitialized initial model.

[0095] The first training submodule is used to train the re-initialized initial model using the sparse training sample set to obtain a key point detection model.

[0096] As one possible implementation, the training module 52 includes (not shown in the figure):

[0097] The second training submodule is used to train the basic network layer and dense key point prediction layer in the initial model using the dense training sample set to obtain the first model;

[0098] The building module includes (not shown in the figure):

[0099] The third training submodule is used to train the basic network layer and sparse keypoint prediction layer in the first model using the sparse training sample set to obtain the second model.

[0100] A submodule is constructed to build a keypoint detection model based on the second model.

[0101] As one possible implementation, the construction submodule is specifically used for:

[0102] The basic network layer and sparse keypoint prediction layer in the second model are used to construct a keypoint detection model.

[0103] See Figure 6 This is a block diagram illustrating an embodiment of a key point detection device provided by an embodiment of the present invention. Figure 6 As shown, the device includes:

[0104] Image acquisition module 61 is used to acquire the target image;

[0105] The input module 62 is used to input the target image into the key point detection model trained according to the method described above, so as to obtain the key point set of the target image.

[0106] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 7 The illustrated electronic device 700 includes at least one processor 701, a memory 702, at least one network interface 704, and a user interface 703. The various components in the electronic device 700 are coupled together via a bus system 705. It is understood that the bus system 705 is used to implement communication between these components. In addition to a data bus, the bus system 705 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 7 The general labeled all buses as Bus System 705.

[0107] The user interface 703 may include a display, keyboard, or clicking device (e.g., mouse, trackball, touchpad, or touchscreen).

[0108] It is understood that the memory 702 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 702 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0109] In some implementations, memory 702 stores elements, executable units or data structures, or subsets thereof, or extended sets thereof: operating system 7021 and application program 7022.

[0110] The operating system 7021 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 7022 includes various applications, such as a media player and a browser, used to implement various application functions. The program implementing the method of this embodiment can be included in the application program 7022.

[0111] In this embodiment of the invention, by calling the program or instructions stored in the memory 702, specifically the program or instructions stored in the application program 7022, the processor 701 executes the method steps provided in each method embodiment, including, for example:

[0112] Obtain a dense training sample set and a sparse training sample set, wherein each training sample in the dense training sample set is labeled with a dense keypoint set, and each training sample in the sparse training sample set is labeled with a sparse keypoint set.

[0113] The initial model is trained using the dense training sample set to obtain the first model;

[0114] Using the sparse training sample set and the first model, a key point detection model is constructed, which is used to detect key points in an image.

[0115] or,

[0116] Acquire the target image;

[0117] The target image is input into the keypoint detection model trained according to the method described above to obtain the keypoint set of the target image.

[0118] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 701. Processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 701 or by instructions in software form. The processor 701 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software units may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 702. Processor 701 reads the information in memory 702 and, in conjunction with its hardware, completes the steps of the above method.

[0119] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0120] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0121] The electronic device provided in this embodiment may be as follows: Figure 7 The electronic device shown can perform the following: Figure 1 All steps of the training method for the keypoint detection model, or Figure 3 All steps of the key point detection method are implemented to achieve... Figure 1 Training methods for key point detection models or Figure 3 For details on the technical effectiveness of the detection method for key points, please refer to [link / reference]. Figure 1 and Figure 3 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0122] This invention also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; the memory may also include combinations of the above types of memory.

[0123] When one or more programs in the storage medium can be executed by one or more processors to implement the above-mentioned training method for the key point detection model and the key point detection method executed on the electronic device side.

[0124] The processor is used to execute a training program for a keypoint detection model or a keypoint detection program stored in the memory, to implement the following steps of the keypoint detection model training method or keypoint detection method executed on the electronic device side:

[0125] Obtain a dense training sample set and a sparse training sample set, wherein each training sample in the dense training sample set is labeled with a dense keypoint set, and each training sample in the sparse training sample set is labeled with a sparse keypoint set.

[0126] The initial model is trained using the dense training sample set to obtain the first model;

[0127] Using the sparse training sample set and the first model, a key point detection model is constructed, which is used to detect key points in an image.

[0128] or,

[0129] Acquire the target image;

[0130] The target image is input into the keypoint detection model trained according to the method described above to obtain the keypoint set of the target image.

[0131] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0132] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0133] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A training method for a keypoint detection model, characterized in that, The method includes: Obtain a dense training sample set and a sparse training sample set, wherein each training sample in the dense training sample set is labeled with a dense keypoint set, and each training sample in the sparse training sample set is labeled with a sparse keypoint set. The initial model is trained using the dense training sample set to obtain the first model; Using the sparse training sample set and the first model, a key point detection model is constructed, which is used to detect key points in an image. The step of training the initial model using the dense training sample set to obtain the first model includes: The base network layer and dense keypoint prediction layer in the initial model are trained using the dense training sample set to obtain the first model; Using the sparse training sample set and the first model, a keypoint detection model is constructed, including: The sparse training sample set is used to train the basic network layer and sparse keypoint prediction layer in the first model to obtain the second model; Based on the second model, a key point detection model is constructed.

2. The method according to claim 1, characterized in that, The step of constructing a keypoint detection model using the sparse training sample set and the first model includes: The basic network layers in the initial model are re-initialized using the basic network layers in the first model to obtain the re-initialized initial model; The re-initialized initial model is trained using the sparse training sample set to obtain the key point detection model.

3. The method according to claim 1, characterized in that, The construction of the key point detection model based on the second model includes: The basic network layer and sparse keypoint prediction layer in the second model are used to construct a keypoint detection model.

4. A method for detecting key points, characterized in that, The method includes: Acquire the target image; The target image is input into a keypoint detection model trained according to any one of claims 1 to 3 above, to obtain a keypoint set of the target image.

5. A training device for a keypoint detection model, characterized in that, The device includes: The acquisition module is used to acquire a dense training sample set and a sparse training sample set. Each training sample in the dense training sample set is labeled with a dense keypoint set, and each training sample in the sparse training sample set is labeled with a sparse keypoint set. The training module is used to train the initial model using the dense training sample set to obtain the first model; The construction module is used to construct a key point detection model using the sparse training sample set and the first model, the key point detection model being used to detect key points in an image; The training module includes: The second training submodule is used to train the basic network layer and dense key point prediction layer in the initial model using the dense training sample set to obtain the first model; The building module includes: The third training submodule is used to train the basic network layer and sparse keypoint prediction layer in the first model using the sparse training sample set to obtain the second model. A submodule is constructed to build a keypoint detection model based on the second model.

6. The apparatus according to claim 5, characterized in that, The building module includes: An initialization submodule is used to reinitialize the basic network layers in the initial model using the basic network layers in the first model, so as to obtain a reinitialized initial model. The first training submodule is used to train the re-initialized initial model using the sparse training sample set to obtain a key point detection model.

7. A device for detecting key points, characterized in that, The device includes: The image acquisition module is used to acquire the target image; An input module is used to input the target image into a key point detection model trained according to the method described in any one of claims 1 to 3 above, to obtain a key point set of the target image.

8. An electronic device, characterized in that, include: A processor and a memory, the processor being configured to execute a training program for a keypoint detection model or a keypoint detection program stored in the memory, to implement the training method for the keypoint detection model according to any one of claims 1 to 3, or the keypoint detection method according to claim 4.

9. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the training method of the key point detection model according to any one of claims 1 to 3, or the key point detection method according to claim 4.

Images