Key point position determination method and device, equipment and storage medium

Abstract

The invention relates to the field of artificial intelligence, and provides a key point position determination method and device, equipment and a storage medium, and the method comprises the steps: obtaining an overall detection image corresponding to a target object; inputting the overall detection image into a trained first-class key point detection model to obtain an initial key point position result corresponding to the overall detection image; key point grouping is carried out according to the initial key point position result, the overall detection image is cut based on the grouping result, and a plurality of corresponding local detection images are obtained; inputting the plurality of local detection images into at least one trained second type key point detection model to obtain a local key point position result corresponding to each local detection image; and merging the plurality of local key point position results to obtain the key point position corresponding to the target object, thereby improving the accuracy of determining the key point position. The invention further relates to a block chain technology, and key point positions can be stored in block chain nodes.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method, device, equipment and storage medium for determining key point positions. Background Art

[0002] In the field of artificial intelligence, key point models can be used to determine the coordinates of key points. Generally, key point models require more than 10,000 sample data to train in simple scenarios. In complex scenarios or when the number of key points required increases, the demand for sample data may increase exponentially. Faced with such a large amount of sample data, it is difficult to collect enough sample data to train key point models in actual applications. The lack of sample data leads to unsatisfactory model training results. Using such a key point model to determine the coordinates of key points cannot guarantee the accuracy of the coordinates of key points.

[0003] Therefore, how to improve the accuracy of determining the location of key points becomes an urgent problem to be solved. Summary of the invention

[0004] The present application provides a method, device, equipment and storage medium for determining key point positions, aiming to improve the accuracy of determining key point positions.

[0005] To achieve the above object, the present application provides a key point position determination method, the key point position determination method comprising:

[0006] Obtaining the overall detection image corresponding to the target object;

[0007] Input the overall detection image into a trained first-type key point detection model to obtain an initial key point position result corresponding to the overall detection image;

[0008] Grouping key points according to the initial key point position results, and cropping the overall detection image based on the grouping results to obtain a corresponding plurality of local detection images;

[0009] Inputting the plurality of local detection images into at least one trained second-category key point detection model to obtain a local key point position result corresponding to each of the local detection images;

[0010] The multiple local key point position results are combined to obtain the key point position corresponding to the target object.

[0011] In addition, to achieve the above-mentioned purpose, the present application also provides a key point position determination device, the key point position determination device comprising:

[0012] An acquisition module is used to acquire an overall detection image corresponding to the target object;

[0013] A first analysis module is used to input the overall detection image into a trained first-type key point detection model to obtain an initial key point position result corresponding to the overall detection image;

[0014] An image cropping module, used for grouping key points according to the initial key point position results, and cropping the overall detection image based on the grouping results to obtain a corresponding plurality of local detection images;

[0015] A second analysis module is used to input the plurality of local detection images into at least one trained second-type key point detection model to obtain a local key point position result corresponding to each of the local detection images;

[0016] The processing module is used to merge the multiple local key point position results to obtain the key point position corresponding to the target object.

[0017] In addition, to achieve the above-mentioned purpose, the present application also provides a computer device, the computer device comprising a memory and a processor;

[0018] The memory is used to store computer programs;

[0019] The processor is used to execute the computer program and implement the key point position determination method as described above when executing the computer program.

[0020] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned key point position determination method are implemented.

[0021] The present application discloses a method, apparatus, device and storage medium for determining key point positions, which obtains an overall detection image corresponding to a target object, inputs the overall detection image into a trained first-type key point detection model, obtains an initial key point position result corresponding to the overall detection image, and groups key points according to the initial key point position result, crops the overall detection image based on the grouping result, obtains multiple corresponding local detection images, and then inputs the multiple local detection images into at least one trained second-type key point detection model to obtain local key point position results corresponding to each local detection image, merges multiple local key point position results, and obtains the key point position corresponding to the target object, that is, determines the key point position of the target object by combining multiple models, thereby improving the accuracy of determining the key point position. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 is a schematic flow chart of the steps of a method for determining a key point position provided in an embodiment of the present application;

[0024] Figure 2 It is a schematic flow chart of the steps of training a first type of key point detection model provided in an embodiment of the present application;

[0025] Figure 3 is a schematic diagram of a first type of training sample image provided in an embodiment of the present application;

[0026] Figure 4 It is a schematic flow chart of the steps of training a second type of key point detection model provided in an embodiment of the present application;

[0027] Figure 5 This is a method provided by the embodiment of the present application. Figure 3 Schematic diagram of dividing and cropping the first type of training sample images in;

[0028] Figure 6 It is a schematic flow chart of the steps of grouping key points according to the initial key point position results, cropping the overall detection image based on the grouping results, and obtaining corresponding multiple local detection images, provided by an embodiment of the present application;

[0029] Figure 7 It is a schematic diagram of a process for determining the key point position of a target object provided by an embodiment of the present application;

[0030] Figure 8 is a schematic block diagram of a key point position determination device provided in an embodiment of the present application;

[0031] Fig. 9 It is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0034] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0035] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0036] In the field of artificial intelligence, key point models can be used to determine the coordinates of key points. Generally, key point models require more than 10,000 sample data to train in simple scenarios. In complex scenarios or when the number of key points required increases, the demand for sample data may increase exponentially. Faced with such a large amount of sample data, it is difficult to collect enough sample data to train key point models in actual applications. The lack of sample data leads to unsatisfactory model training results. Using such a key point model to determine the coordinates of key points cannot guarantee the accuracy of the coordinates of key points.

[0037] In order to solve the above problems, the embodiments of the present application provide a key point position determination method, apparatus, device and storage medium for improving the accuracy of key point positions.

[0038] See also Figure 1 , Figure 1 1 is a flow chart of a key point location determination method provided in an embodiment of the present application. The method can be applied to a computer device, and the application scenario of the method is not limited in the present application.

[0039] like Figure 1 As shown, the key point position determination method specifically includes steps S101 to S105.

[0040] S101: Obtain an overall detection image corresponding to a target object.

[0041] It is understandable that the target object is different for different application scenarios. For example, the target object can be a person, an animal, a plant, a building, a vehicle, a device, etc.

[0042] First, an original image of the target object is obtained, for example, by photographing the target object with a camera, or by retrieving the original image of the target object from a photo album.

[0043] Afterwards, if there are no other objects except the target object in the original image, the original image can be directly used as the overall detection image of the target object. If there are other objects except the target object in the original image, the original image can be cropped to obtain the overall detection image of the target object.

[0044] In some embodiments, obtaining the overall detection image corresponding to the target object includes: marking the target object with a detection frame on the original image, and cropping the original image based on the detection frame of the target object to obtain the overall detection image.

[0045] For example, image recognition processing is performed on the original image to detect the target object in the original image, and the target object is marked with a detection frame, for example, the target object is marked with a red rectangular frame, and the original image is cropped based on the marked red rectangular frame to obtain an overall detection image of the target object.

[0046] S102: Input the overall detection image into a trained first-category key point detection model to obtain an initial key point position result corresponding to the overall detection image.

[0047] Among them, the first type of key point detection model can be a deep learning model. Of course, the first type of key point detection model can also be other types of neural network models. The initial key point position result of the target object is obtained by calling the first type of key point detection model. Before calling the first type of key point detection model for key point detection, the first type of key point detection model is first trained. Exemplarily, the first type of key point detection model can be trained based on a supervised learning algorithm, for example, using a regression loss function.

[0048] In some embodiments, Figure 2 As shown, step S106 to step S109 may be included before step S102.

[0049] S106, obtaining an initial sample image;

[0050] S107, marking key points of the sample objects in the initial sample image, and generating an initial detection frame including all the marked key points;

[0051] S108, performing at least one frame expansion process on the initial detection frame, cropping the initial sample image based on the detection frame after each frame expansion process to obtain at least one cropped image, and using each of the cropped images as a first type of training sample image;

[0052] S109: Input the first type of training sample images into the first type of key point detection model, perform model training on the first type of key point detection model, and obtain a trained first type of key point detection model.

[0053] In order to train the first type of key point detection model, samples required for model training are first obtained, for example, by photographing with a camera to obtain initial sample images, or by retrieving initial sample images from a photo album.

[0054] Afterwards, the sample object in the initial sample image is annotated with key points, and an initial detection frame corresponding to the sample object is generated based on all the annotated key points, wherein the initial detection frame includes all the annotated key points. For example, a red rectangular frame including all the annotated key points is generated.

[0055] After generating the initial detection frame corresponding to the sample object, the initial detection frame is randomly expanded. For example, the initial detection frame is randomly expanded by a corresponding multiple from a preset multiple range (such as 0-0.5 times). The expansion process can be performed only once or multiple times. After each expansion process, the initial sample image is cropped based on the detection frame after each expansion process to obtain a corresponding cropped image, and each cropped image is used as the first type of training sample image. The amount of training data is increased by expanding the frame and performing data augmentation.

[0056] For example, taking the sample object as a pig, Figure 3 is a corresponding first-category training sample image. Figure 3 The small dots in the figure are the labels of the annotated key points, and the red rectangle is the detection box obtained by expanding the initial detection box containing all the annotated key points.

[0057] After obtaining multiple first-category training sample images in the above manner, each first-category training sample image is input into the first-category key point detection model, and the first-category key point detection model is trained until the first-category key point detection model converges to obtain a trained first-category key point detection model.

[0058] The first-class key point detection model is trained so that the key point position output by the first-class key point detection model is near the annotated key point position label. For example, a regression loss function is used, and the training goal is to make the loss change slightly when the regression key point is near the label, so that the first-class key point detection model pays more attention to the overall information of the image, ensures the correct position relationship of each key point during inference and prediction, and allows a certain range of position errors. For example, for Figure 3For the key point position labels marked in the figure, it is only required that the first type of key point detection model infers and predicts that the position is within the circle corresponding to the small dot. The accuracy is as small as possible with the deviation within a preset ratio (such as 10%, or a maximum of 15%).

[0059] In some embodiments, Figure 4 As shown, the key point position determination method further includes step S110 and step S111.

[0060] S110, grouping key points based on the output results of the first type of key point detection model, and cropping the first type of training sample images according to the grouping results to obtain a corresponding plurality of second type of training sample images;

[0061] S111, input each of the second-category training sample images into the corresponding second-category key point detection model, perform model training on the second-category key point detection model, and obtain a trained second-category key point detection model.

[0062] According to the key point positions corresponding to the first type of training sample images output by the trained first type of key point detection model, the key points are grouped, and the key points with close distances are grouped together, so that multiple groups (such as n groups) of key points are obtained. Then, according to the multiple groups of key points, the first type of training sample images are cropped, and the first type of training sample images are divided into multiple blocks (such as n blocks). Each divided block corresponds to a key point group, and overlapping areas are allowed between adjacent blocks to ensure that the key points contained in the key point group corresponding to each divided block must be within the block. According to the result of being divided into multiple blocks, the first type of training sample images are cropped to obtain multiple corresponding second type training sample images, that is, each divided block corresponds to a second type training sample image.

[0063] For example, Figure 3 Taking the first type of training sample image shown in FIG. 1 as an example, the key points are grouped into 5 groups according to the distance between the key points. According to the 5 groups of key points, the first type of training sample image is divided into 5 blocks, such as Figure 5 The blue rectangular boxes correspond to areas 1, 2, 3, 4, and 5. There are overlapping areas between the blue rectangular boxes, and each blue rectangular box area corresponds to a cropped second-category training sample image.

[0064] According to the above operations, multiple second-category training sample images can be obtained, and each second-category training sample image is input into at least one second-category key point detection model, wherein the second-category key point detection model can be a deep learning model. Of course, the second-category key point detection model can also be other types of neural network models. Based on the second-category training sample images, at least one second-category key point detection model is trained until each second-category key point detection model converges, thereby obtaining at least one trained second-category key point detection model.

[0065] Compared with training the first type of key point detection model, when training the second type of key point detection model, for example, a heatmap regression algorithm is used to make key point position inference predictions, which ultimately makes the second type of key point detection model pay more attention to local feature information such as texture corner points, and obtain more accurate key point positions in the local range.

[0066] Exemplarily, the second category training sample images are divided into multiple groups. For example, based on the situation that the first category training sample images are divided into n blocks, the obtained second category training sample images are divided into n groups. According to each group of second category training sample images, a second category key point detection model is trained. In this way, n trained second category key point detection models are obtained.

[0067] When the amount of data is not large enough, the model functions can be separated and divided into multiple models by operations such as block division, so that each model belongs to a simple scenario. A separate targeted loss can be designed on each model to facilitate the use of smaller models and simpler models. After completing the entire training process, more images can be automatically annotated through multi-level models to obtain more samples, so that a single model can be trained using automatically annotated images when the system needs to be simplified.

[0068] Based on the above operations, a trained first-category key point detection model and at least one second-category key point detection model are obtained. After obtaining the overall detection image of the target object, the overall detection image of the target object is input into the trained first-category key point detection model, and the first-category key point detection model outputs the initial key point position results corresponding to the overall detection image. The initial key point position results include multiple key point positions of the target object as a whole inferred and predicted by the first-category key point detection model.

[0069] S103, grouping key points according to the initial key point position results, and cropping the overall detection image based on the grouping results to obtain a corresponding plurality of local detection images.

[0070] After obtaining the positions of multiple key points of the target object as a whole, the key points are grouped and the key points with close distances are grouped together. In this way, multiple groups of key points corresponding to the target object are obtained. Then, the overall detection image of the target object is cropped according to the multiple groups of key points to obtain multiple cropped local detection images.

[0071] In some embodiments, Figure 6 As shown, step S103 may include sub-steps S1031 to S1033.

[0072] S1031, determining the distance between key points according to the initial key point position result;

[0073] S1032, grouping the key points according to the distance, wherein the distance between key points in the same group is smaller than the distance between key points in different groups;

[0074] S1033, cropping the overall detection image according to each group of key points to obtain a plurality of local detection images, each of which contains a corresponding group of key points.

[0075] After obtaining the positions of multiple key points of the target object as a whole through inference and prediction by the first type of key point detection model, the distance between each key point can be determined based on the positions of multiple key points of the target object as a whole. Of course, the distance between each key point can be close or far.

[0076] According to the distance between each key point, the key points with close distance are divided into the same group, and the key points with far distance are divided into different groups, so that multiple groups of key points corresponding to the target object are obtained. Afterwards, according to each group of key points, the overall detection image of the target object is divided into multiple blocks, and each block corresponds to a key point group of the target object. Overlapping areas are allowed between adjacent blocks to ensure that the key points contained in the key point group of the target object corresponding to each divided block must be within the block. According to the result of division into multiple blocks, the overall detection image of the target object is cropped to obtain multiple local detection images corresponding to the target object. It can be understood that each local detection image must contain a group of key points corresponding to the target object.

[0077] S104: Input the plurality of local detection images into at least one trained second-category key point detection model to obtain a local key point position result corresponding to each of the local detection images.

[0078] After obtaining multiple local detection images corresponding to the target object, each local detection image is input into the corresponding second-type key point detection model respectively, and the local key point position results corresponding to each local detection image are output through the corresponding second-type key point detection model. The local key point position results include multiple local key point positions of the target object inferred and predicted by the second-type key point detection model.

[0079] In some embodiments, the inputting of multiple local detection images into at least one trained second-category key point detection model to obtain the local key point position results corresponding to each local detection image includes: respectively determining the target model corresponding to each local detection image, wherein the target model is one of multiple second-category key point detection models; inputting each local detection image into the corresponding target model to obtain the local key point position results corresponding to each local detection image.

[0080] For example, assuming that the trained second-category key point detection models include model A, model B, model C, model D, and model E, the local detection images corresponding to the target object obtained include local detection image a, local detection image b, local detection image c, local detection image d, and local detection image e, determine that the target model corresponding to local detection image a is model A, the target model corresponding to local detection image b is model B, the target model corresponding to local detection image c is model C, the target model corresponding to local detection image d is model D, and the target model corresponding to local detection image e is model E, input the local detection image a into model A, input the local detection image b into model B, input the local detection image c into model C, input the local detection image d into model D, and input the local detection image e into model E, output the local key point position result corresponding to the local detection image a through model A, output the local key point position result corresponding to the local detection image b through model B, output the local key point position result corresponding to the local detection image c through model C, output the local key point position result corresponding to the local detection image d through model D, and output the local key point position result corresponding to the local detection image e through model E.

[0081] S105: Merge the multiple local key point position results to obtain the key point position corresponding to the target object.

[0082] The local key point position results output by all second-category key point detection models are merged. For instance, still taking the above-mentioned example, the key point position results corresponding to local detection image a, local detection image b, local detection image c, local detection image d, and local detection image e are merged. That is, all key points inferred and predicted by each second-category key point detection model are merged to determine all key points corresponding to the target object and map them to the key point position of the target object on the original image.

[0083] In some embodiments, the merging of multiple local key point position results to obtain the key point position corresponding to the target object includes: if the local key point position results include multiple positions of the same key point, selecting one position from the multiple positions as the final position of the same key point, or performing weighted calculation processing on the multiple positions to obtain the final position of the same key point.

[0084] Since there may be overlapping areas between the local detection images, there may be multiple key point positions of the same key point in the local key point position results output by each second-category key point detection model. For non-repeated key points, the obtained key point position is directly used as the final position of the key point. For repeated key points, multiple key point positions of the key point will be obtained. In this case, one position can be directly selected from the multiple key point positions as the final position of the key point.

[0085] Alternatively, the weight of the key point position inferred and predicted by each second-category key point detection model can be set. If multiple key point positions of the same key point are obtained based on multiple second-category key point detection models, then the multiple key point positions of the same key point can be weightedly calculated based on the weights of the key point positions inferred and predicted by these multiple second-category key point detection models, and the calculated position can be used as the final position of the same key point.

[0086] The following takes the first type of key point detection model as model 1, and the second type of key point detection model includes model 2-1, model 2-2...model 2-n as an example. Figure 7 As shown, the process of determining the key point position of the target object is as follows:

[0087] Step a: Input the original image of the target object and mark the detection frame;

[0088] Step b, expanding the frame, and cropping the original image based on the expanded detection frame to obtain an overall detection image;

[0089] Step c: input the whole detection image into model 1 for inference prediction to obtain the key point position result of the whole target object;

[0090] Step d: group the key points according to the key point position results of the target object as a whole, divide the overall detection image into blocks based on multiple groups of key points, and obtain multiple corresponding local detection images after cropping;

[0091] Step e, inputting multiple local detection images into model 2-1, model 2-2, ..., model 2-n for inference prediction, and obtaining the local key point position results of the target object;

[0092] Step f: merge the inference prediction results of model 2-1, model 2-2, ..., model 2-n, and map all key points to the original image.

[0093] In the above embodiment, an overall detection image corresponding to the target object is obtained, the overall detection image is input into a trained first-category key point detection model to obtain an initial key point position result corresponding to the overall detection image, and key points are grouped according to the initial key point position result. The overall detection image is cropped based on the grouping result to obtain multiple corresponding local detection images. The multiple local detection images are then input into at least one trained second-category key point detection model to obtain local key point position results corresponding to each local detection image. The multiple local key point position results are merged to obtain the key point position corresponding to the target object. That is, the key point position of the target object is determined by jointly determining multiple models, thereby improving the accuracy of determining the key point position.

[0094] See also Figure 8 , Figure 8 It is a schematic block diagram of a key point position determination device provided in an embodiment of the present application. The key point position determination device can be configured in a computer device to execute the aforementioned key point position determination method.

[0095] like Figure 8 As shown, the key point position determination device 1000 includes: an acquisition module 1001 , a first analysis module 1002 , an image cropping module 1003 , a second analysis module 1004 , and a processing module 1005 .

[0096] An acquisition module 1001 is used to acquire an overall detection image corresponding to the target object;

[0097] The first analysis module 1002 is used to input the overall detection image into a trained first-type key point detection model to obtain an initial key point position result corresponding to the overall detection image;

[0098] An image cropping module 1003 is used to group key points according to the initial key point position results, and crop the overall detection image based on the grouping results to obtain a corresponding plurality of local detection images;

[0099] The second analysis module 1004 is used to input the plurality of local detection images into at least one trained second-type key point detection model to obtain a local key point position result corresponding to each of the local detection images;

[0100] The processing module 1005 is used to merge the multiple local key point position results to obtain the key point position corresponding to the target object.

[0101] In one embodiment, the image cropping module 1003 is further used for:

[0102] Determine the distance between key points according to the initial key point position result;

[0103] The key points are grouped according to the distances, wherein the distances between key points in the same group are smaller than the distances between key points in different groups;

[0104] The overall detection image is cropped according to each group of key points to obtain a plurality of local detection images, each of which contains a corresponding group of key points.

[0105] In one embodiment, the second analysis module 1004 is further configured to:

[0106] Determine a target model corresponding to each of the local detection images respectively, wherein the target model is one of a plurality of the second-type key point detection models;

[0107] Each of the local detection images is input into the corresponding target model to obtain the local key point position result corresponding to each of the local detection images.

[0108] In one embodiment, the acquisition module 1001 is further used for:

[0109] A detection frame is marked on the target object on the original image, and the original image is cropped based on the detection frame of the target object to obtain the overall detection image.

[0110] In one embodiment, the processing module 1005 is further configured to:

[0111] If the local key point position result includes multiple positions of the same key point, one position is selected from the multiple positions as the final position of the same key point, or the multiple positions are weightedly calculated to obtain the final position of the same key point.

[0112] In one embodiment, the key point position determination device 1000 further includes a first training module, which is used to:

[0113] Obtaining an initial sample image;

[0114] Annotating key points of sample objects in the initial sample image and generating an initial detection frame containing all the annotated key points;

[0115] Performing at least one frame expansion process on the initial detection frame, cropping the initial sample image based on the detection frame after each frame expansion process to obtain at least one cropped image, and using each of the cropped images as a first type of training sample image;

[0116] The first category of training sample images is input into the first category of key point detection model, and model training is performed on the first category of key point detection model to obtain a trained first category of key point detection model.

[0117] In one embodiment, the key point position determination device 1000 further includes a second training module, which is used to:

[0118] Grouping key points based on the output results of the first type of key point detection model, and cropping the first type of training sample images according to the grouping results to obtain a plurality of corresponding second type of training sample images;

[0119] Each of the second-category training sample images is input into the corresponding second-category key point detection model, and the second-category key point detection model is trained to obtain a trained second-category key point detection model.

[0120] Among them, each module in the above-mentioned key point position determination device 1000 corresponds to each step in the above-mentioned key point position determination method embodiment, and its functions and implementation processes are no longer repeated here one by one.

[0121] The method and apparatus of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0122] Exemplarily, the above method and apparatus may be implemented in the form of a computer program. The computer program may be implemented in Fig. 9 Runs on the computer device shown.

[0123] See also Fig. 9 , Fig. 9 It is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application.

[0124] See also Fig. 9 The computer device includes a processor and a memory connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0125] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0126] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any key point position determination method.

[0127] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0128] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0129] Obtaining the overall detection image corresponding to the target object;

[0130] Input the overall detection image into a trained first-type key point detection model to obtain an initial key point position result corresponding to the overall detection image;

[0131] Grouping key points according to the initial key point position results, and cropping the overall detection image based on the grouping results to obtain a corresponding plurality of local detection images;

[0132] Inputting the plurality of local detection images into at least one trained second-category key point detection model to obtain a local key point position result corresponding to each of the local detection images;

[0133] The multiple local key point position results are combined to obtain the key point position corresponding to the target object.

[0134] In one embodiment, when the processor implements the grouping of key points according to the initial key point position results, and cropping the overall detection image based on the grouping results to obtain the corresponding multiple local detection images, it is used to implement:

[0135] Determine the distance between key points according to the initial key point position result;

[0136] The key points are grouped according to the distances, wherein the distances between key points in the same group are smaller than the distances between key points in different groups;

[0137] The overall detection image is cropped according to each group of key points to obtain a plurality of local detection images, each of which contains a corresponding group of key points.

[0138] In one embodiment, when the processor implements the step of inputting the plurality of local detection images into at least one trained second-type key point detection model to obtain the local key point position result corresponding to each of the local detection images, it is configured to implement:

[0139] Determine a target model corresponding to each of the local detection images respectively, wherein the target model is one of a plurality of the second-type key point detection models;

[0140] Each of the local detection images is input into the corresponding target model to obtain the local key point position result corresponding to each of the local detection images.

[0141] In one embodiment, when the processor implements the acquisition of the overall detection image corresponding to the target object, it is used to implement:

[0142] A detection frame is marked on the target object on the original image, and the original image is cropped based on the detection frame of the target object to obtain the overall detection image.

[0143] In one embodiment, when the processor realizes merging the plurality of local key point position results to obtain the key point position corresponding to the target object, it is used to realize:

[0144] If the local key point position result includes multiple positions of the same key point, one position is selected from the multiple positions as the final position of the same key point, or the multiple positions are weightedly calculated to obtain the final position of the same key point.

[0145] In one embodiment, before implementing the step of inputting the overall detection image into the trained first-type key point detection model, the processor is configured to implement:

[0146] Obtaining an initial sample image;

[0147] Annotating key points of sample objects in the initial sample image and generating an initial detection frame containing all the annotated key points;

[0148] Performing at least one frame expansion process on the initial detection frame, cropping the initial sample image based on the detection frame after each frame expansion process to obtain at least one cropped image, and using each of the cropped images as a first type of training sample image;

[0149] The first category of training sample images is input into the first category of key point detection model, and model training is performed on the first category of key point detection model to obtain a trained first category of key point detection model.

[0150] In one embodiment, the processor is further configured to implement:

[0151] Grouping key points based on the output results of the first type of key point detection model, and cropping the first type of training sample images according to the grouping results to obtain a plurality of corresponding second type of training sample images;

[0152] Each of the second-category training sample images is input into the corresponding second-category key point detection model, and the second-category key point detection model is trained to obtain a trained second-category key point detection model.

[0153] An embodiment of the present application also provides a computer-readable storage medium.

[0154] The computer-readable storage medium of the present application stores a computer program, and when the computer program is executed by a processor, the steps of the key point position determination method as described above are implemented.

[0155] Wherein, the computer-readable storage medium may be an internal storage unit of the key point position determination device or computer device described in the aforementioned embodiment, such as a hard disk or memory of the key point position determination device or computer device. The computer-readable storage medium may also be an external storage device of the key point position determination device or computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital card (Secure Digital Card, SD Card), a flash card (Flash Card), etc., equipped on the key point position determination device or computer device.

[0156] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0157] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0158] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0159] The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and these modifications or substitutions should be included in the protection scope of the present application.

Claims

1. A method for determining the position of key points, It is characterized in that The key point position determination method comprises: Obtaining the overall detection image corresponding to the target object; Input the overall detection image into a trained first-type key point detection model to obtain an initial key point position result corresponding to the overall detection image; Grouping key points according to the initial key point position results, and cropping the overall detection image based on the grouping results to obtain a corresponding plurality of local detection images; Inputting the plurality of local detection images into at least one trained second-category key point detection model to obtain a local key point position result corresponding to each of the local detection images; The multiple local key point position results are combined to obtain the key point position corresponding to the target object.

2. The key point position determination method according to claim 1, It is characterized in that The key point grouping is performed according to the initial key point position result, and the overall detection image is cropped based on the grouping result to obtain a corresponding plurality of local detection images, including: Determine the distance between key points according to the initial key point position result; The key points are grouped according to the distances, wherein the distances between key points in the same group are smaller than the distances between key points in different groups; The overall detection image is cropped according to each group of key points to obtain a plurality of local detection images, each of which contains a corresponding group of key points.

3. The key point position determination method according to claim 1, It is characterized in that Inputting the plurality of local detection images into at least one trained second-type key point detection model to obtain a local key point position result corresponding to each local detection image includes: Determine a target model corresponding to each of the local detection images respectively, wherein the target model is one of a plurality of the second-type key point detection models; Each of the local detection images is input into the corresponding target model to obtain the local key point position result corresponding to each of the local detection images.

4. The key point position determination method according to claim 1, It is characterized in that The step of obtaining an overall detection image corresponding to the target object includes: A detection frame is marked on the target object on the original image, and the original image is cropped based on the detection frame of the target object to obtain the overall detection image.

5. The key point position determination method according to claim 1, It is characterized in that The step of merging the plurality of local key point position results to obtain the key point position corresponding to the target object includes: If the local key point position result includes multiple positions of the same key point, then one position is selected from the multiple positions as the final position of the same key point, or the multiple positions are weightedly calculated to obtain the final position of the same key point.

6. The key point position determination method according to any one of claims 1 to 5, It is characterized in that Before inputting the overall detection image into the trained first-type key point detection model, the method includes: Obtaining an initial sample image; Annotating key points of sample objects in the initial sample image and generating an initial detection frame containing all the annotated key points; Performing at least one frame expansion process on the initial detection frame, cropping the initial sample image based on the detection frame after each frame expansion process to obtain at least one cropped image, and using each of the cropped images as a first type of training sample image; The first category of training sample images is input into the first category of key point detection model, and model training is performed on the first category of key point detection model to obtain a trained first category of key point detection model.

7. The key point position determination method as claimed in claim 6, It is characterized in that The method further comprises: Grouping key points based on the output results of the first type of key point detection model, and cropping the first type of training sample images according to the grouping results to obtain a plurality of corresponding second type of training sample images; Each of the second-category training sample images is input into the corresponding second-category key point detection model, and the second-category key point detection model is trained to obtain a trained second-category key point detection model.

8. A device for determining a key point position, It is characterized in that The key point position determination device comprises: An acquisition module is used to acquire an overall detection image corresponding to the target object; A first analysis module is used to input the overall detection image into a trained first-type key point detection model to obtain an initial key point position result corresponding to the overall detection image; An image cropping module, used for grouping key points according to the initial key point position results, and cropping the overall detection image based on the grouping results to obtain a corresponding plurality of local detection images; A second analysis module is used to input the plurality of local detection images into at least one trained second-type key point detection model to obtain a local key point position result corresponding to each of the local detection images; The processing module is used to merge the multiple local key point position results to obtain the key point position corresponding to the target object.

9. A computer device, It is characterized in that The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the key point position determination method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the key point position determination method according to any one of claims 1 to 7 are implemented.

Images