A data augmentation method, device and terminal device for human pose estimation

Abstract

This application is applicable to the field of data augmentation technology, and provides a data augmentation method, device and terminal device for human pose estimation. In the embodiments of this application, a target image is obtained, the pose of the human body in the above target image is estimated to obtain the initial heatmap of the above human body; target key points with confidence levels meeting a preset threshold range are determined from the above initial heatmap; the local components of the above human body corresponding to the above target key points are determined, and a preset number of component maps corresponding to the above local components are obtained from a preset component library; the initial heatmap is data-augmented according to the above preset number of component maps to obtain an enhanced human body heatmap, thereby improving the accuracy of extracting human key points.

Description

Technical Field

[0001] The present application belongs to the field of data enhancement technology, and in particular, relates to a data enhancement method, device and terminal equipment for human posture estimation. Background Art

[0002] With the development of society, the key points of the human body can be extracted by estimating the human body posture, so as to realize visual perception or other tasks based on the key points. For example, somatosensory game interaction can be realized through key point extraction, target tracking can be realized through key point extraction, behavior recognition can be realized through key point extraction, etc.

[0003] However, since the structure of the human body is non-rigid, it has various changes compared to rigid bodies such as cars, tables and chairs. For example, some parts of the human body rotate around the joints to complete complex behaviors or movements. In addition, there are some special scenes with severe occlusion or many obstacles, so it is not possible to estimate the posture of the human body well, which leads to low accuracy in extracting key points of the human body. Summary of the invention

[0004] The embodiments of the present application provide a data enhancement method, apparatus and terminal device for human posture estimation, which can solve the problem of low accuracy in extracting key points of the human body.

[0005] In a first aspect, an embodiment of the present application provides a data enhancement method for human posture estimation, comprising:

[0006] Acquire a target image, perform posture estimation on a human body in the target image, and obtain an initial heat map of the human body;

[0007] Determine the target key points whose confidence meets the preset threshold range from the above initial heat map;

[0008] Determine the local parts of the human body corresponding to the target key points, and obtain a preset number of component drawings corresponding to the local parts from a preset component library;

[0009] The initial thermal map is data enhanced according to the preset number of component maps to obtain an enhanced human thermal map.

[0010] In a second aspect, an embodiment of the present application provides a data enhancement device for human posture estimation, comprising:

[0011] An image acquisition module is used to acquire a target image, estimate the posture of a human body in the target image, and obtain an initial thermal map of the human body;

[0012] A key point determination module is used to determine target key points whose confidence level meets a preset threshold range from the above initial heat map;

[0013] A component diagram acquisition module is used to determine the local components of the human body corresponding to the target key points, and to acquire a preset number of component diagrams corresponding to the local components from a preset component library;

[0014] The data enhancement module is used to perform data enhancement on the initial thermal map according to the preset number of component maps to obtain an enhanced human body thermal map.

[0015] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned data enhancement methods for human posture estimation when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned data enhancement methods for human posture estimation are implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes any one of the data enhancement methods for human posture estimation in the first aspect.

[0018] In the embodiment of the present application, a target image is obtained, and a posture of a human body in the target image is estimated to obtain an initial heat map of the human body. Then, target key points whose confidences meet a preset threshold range are determined from the initial heat map to determine key points with lower precision. Then, local components of the human body corresponding to the target key points are determined to facilitate processing of the local components and improve the accuracy of posture estimation of the human body. Specifically, a preset number of component drawings corresponding to the local components are obtained from a preset component library, and then data enhancement is performed on the initial heat map according to the preset number of component drawings to improve the accuracy of posture estimation of the human body, and finally an enhanced heat map of the human body is obtained, so that key points of the human body with higher precision are obtained through the enhanced heat map of the human body, thereby improving the accuracy of extracting key points of the human body by performing data enhancement on the heat map obtained by posture estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1This is a schematic diagram of a first flow chart of a data enhancement method for human posture estimation provided in an embodiment of the present application;

[0021] Figure 2 This is a second flow chart of the data enhancement method for human posture estimation provided in an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of the structure of a data enhancement device for human posture estimation provided in an embodiment of the present application;

[0023] Figure 4 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0025] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0026] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0027] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0028] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0029] Figure 1FIG. 1 is a flow chart of a data enhancement method for human posture estimation in an embodiment of the present application. The execution subject of the method may be a terminal device, such as Figure 1 As shown, the data enhancement method for human posture estimation may include the following steps:

[0030] Step S101: Acquire a target image, estimate the posture of a human body in the target image, and obtain an initial heat map of the human body.

[0031] In this embodiment, the terminal device can perform posture estimation on the human body in the target image using a preset posture estimation algorithm to obtain an initial heat map without any processing of the human body in the target image. The above-mentioned posture estimation algorithm can be implemented by a posture estimation model, and the posture estimation algorithm includes but is not limited to the FastPose algorithm, the LPN algorithm, etc.

[0032] In one embodiment, before estimating the posture of a human body, the terminal device may first detect the human body in the image using a preset human body detection algorithm, so as to obtain a human body detection frame for a single human body. For example, if there are three human bodies in the current target image, three human body detection frames will appear accordingly, and then the posture of the human body in the human body detection frame is estimated to obtain an initial heat map of the human body corresponding to the human body detection frame. The above-mentioned human body detection algorithm includes but is not limited to the YOLO algorithm, the SSD algorithm, etc.

[0033] Step S102: Determine target key points whose confidences meet a preset threshold range from the initial heat map.

[0034] In this embodiment, for the initial heat map obtained by the terminal device through the posture estimation model, the terminal device can process it through the Argmax function to obtain the key point coordinates corresponding to the initial heat map, and then process the obtained key points through the Softmax function to obtain the confidence of each key point, so as to judge the confidence of each key point, so as to select the target key point whose confidence meets the preset threshold range from each key point, that is, the key point that is easy to identify errors, so as to perform data enhancement on the key point that is easy to identify errors. Among them, the above threshold range can be 0.4 to 0.85.

[0035] It can be understood that if the confidence of the above-mentioned key point is less than the minimum value in the above-mentioned threshold range, it means that the key point is an invisible point, and the key point is filtered without display or corresponding processing; and if the confidence of the above-mentioned key point is greater than the maximum value in the above-mentioned threshold range, it means that the key point is an easy-to-learn point, and there is no need to perform data enhancement on the easy-to-learn point, and it can be directly displayed or processed accordingly.

[0036] In one embodiment, the number of the key points may vary according to the category of the data set in which the image is stored. For example, the MPII data set corresponds to 16 categories of key points, and the COCO data set corresponds to 17 categories of key points.

[0037] Step S103: determine the local parts of the human body corresponding to the target key points, and obtain a preset number of component drawings corresponding to the local parts from a preset component library.

[0038] In this embodiment, the terminal device can process the position of the component by acquiring a certain number of component images that are the same as the local parts of the human body corresponding to the target key points to improve the accuracy of the processed human body heat map, and realize data enhancement of the position through multiple component images, thereby realizing accurate extraction of human body key points.

[0039] Specifically, the above step S103 may include: the terminal device may determine the local human body component corresponding to the target key point from a preset key point comparison table according to the target key point. For example, if the position of the current target key point is the center point of the left wrist, the local human body component may be the part between the left wrist and the left elbow of the human body. Among them, the above key point comparison table may be a table generated according to the key point positions that may appear on the human body and the components with the lowest confidence corresponding to the positions, and the components with the lowest confidence among the components that can be connected to the key points can be determined by data analysis.

[0040] Specifically, the above step S103 may include: the terminal device may determine the adjacent key points connected to the target key point, thereby determining the local component according to the target key point and the adjacent key points. For example, the part connected to the target key point and the adjacent key point is determined as the local component that needs data enhancement.

[0041] In one embodiment, when there are at least two adjacent key points, the terminal device may select the adjacent key point with the smallest confidence from the at least two adjacent key points, and determine the component between the target key point and the adjacent key point with the smallest confidence as a local component. It is understandable that the confidence of the component composed of the adjacent key point with the smallest confidence and the target key point is also the smallest compared to the components corresponding to other key points connected to the target key point at the same time, so that the local component that needs to be enhanced can be accurately identified.

[0042] In one embodiment, before step S103, it may also include: the terminal device obtains a global map of the human body, that is, a picture containing only the human body, and then crops the global map according to a preset segmentation network to obtain each local component map, and calculates the confidence of each local component map, and stores the local component map with a confidence greater than a preset confidence threshold in a corresponding position in the component library, for example, according to the name of the local component map. The above-mentioned segmentation network may be a PSPNet network. It can be understood that the above-mentioned confidence threshold is the confidence critical value corresponding to the easy-to-learn point, so the confidence threshold may be the maximum value in the above-mentioned threshold range.

[0043] Step S104: perform data enhancement on the initial thermal map according to a preset number of component maps to obtain an enhanced human body thermal map.

[0044] In this embodiment, the terminal device performs local data enhancement on the human body parts with poor estimation effect in the initial heat map obtained by the posture estimation model according to a preset number of component maps, thereby obtaining a preset number of enhanced human body heat maps, and then obtains key points of the human body with higher accuracy through the enhanced human body heat maps, so as to perform corresponding tasks.

[0045] Exemplarily, if there are currently two target key points, namely point A and point B, and the above preset number is set to 3, then after data enhancement is performed on the three component images corresponding to point A and point B respectively, the human body thermal maps A1, A2, and A3 after the enhanced data corresponding to point A are obtained, and the human body thermal maps B1, B2, and B3 after the enhanced data corresponding to point B are obtained.

[0046] In one embodiment, step S104 includes: randomly pasting a preset number of component images on a human body in a target image, and if the human body detection frame exists, randomly pasting them in the human body detection frame to obtain a preset number of human body sample images. Then, the preset number of human body sample images are processed by a preset posture estimation algorithm, for example, the posture estimation algorithm is used to process the human body enhanced data in the target image, and the enhanced human body heat map corresponding to the enhanced human body data is obtained. In addition, the preset number of component images can also be enhanced based on the global data of the human body, for example, rotating a certain angle in a certain rotation direction, and changing the brightness contrast.

[0047] In one embodiment, before step S104, the terminal device may also calculate the component length of the local component according to the target key point, that is, the local limb length of the human body, and then normalize the preset number of component images according to the component length, so that the scale of the local component to be enhanced in the initial heat map and the local component corresponding to the enhanced human body heat map are consistent. For example, if the component length is 250 pixels, the lengths of the preset number of component images need to be normalized to 250 pixels.

[0048] Specifically, if the local component is determined only based on the target key point, the length of the local component can be estimated based on the current length of the human body. If the local component is determined based on the target key point and the adjacent key points adjacent to the target key point, the component length of the local component is determined by the horizontal and vertical coordinates of the target key point and the adjacent key points.

[0049] In one embodiment, Figure 2 As shown, after step S104, the following further includes:

[0050] Step S201, obtaining a label heat map of a human body, and determining a first loss value according to the label heat map and the initial heat map.

[0051] In this embodiment, the terminal device can use the mean square error to calculate the label heat map and the initial heat map to obtain the loss value between the label heat map and the initial heat map, that is, the difference between the label heat map and the initial heat map, which is the first loss value mentioned above.

[0052] Step S202: determining a second loss value according to the enhanced human body thermal map and the initial thermal map.

[0053] In this embodiment, the terminal device can calculate the enhanced human body thermal map and the initial thermal map using the mean square error to obtain the loss value between the enhanced human body thermal map and the initial thermal map, that is, the difference between the enhanced human body thermal map and the initial thermal map. It can be understood that since the number of enhanced human body thermal maps is the same as the number of component maps used for enhanced data, the second loss value is the sum of the loss values ​​corresponding to the preset number of enhanced human body thermal maps.

[0054] Exemplarily, based on the above example, point A corresponds to the human body heat map A1, A2, and A3 after the enhanced data, and point B corresponds to the human body heat map B1, B2, and B3 after the enhanced data. The loss values ​​of the initial heat map and A1, A2, A3, B1, B2, and B3 are calculated respectively to obtain the loss value Loss.A1 of the initial heat map and A1, the loss value Loss.A2 of the initial heat map and A2, the loss value Loss.A3 of the initial heat map and A3, the loss value Loss.B1 of the initial heat map and B1, the loss value Loss.B2 of the initial heat map and B2, and the loss value Loss.B3 of the initial heat map and B3. Then, the sum of Loss.A1, Loss.A2, Loss.A3, Loss.B1, Loss.B2, and Loss.B3 is calculated to obtain the second loss value loss 2 .

[0055] Step S203: weight the first loss value and the second loss value, and determine the total loss value according to the weighted first loss value and the second loss value. The calculation formula of the total loss value is as follows:

[0056] Loss a =α*Loss 2 +(1-α)Loss 1

[0057] Among them, the above α is the weight value, which can be selected from 0.3 to 0.5; the above loss 1 is the first loss value; the above loss 2 is the second loss value; the above loss a is the total loss value.

[0058] Step S204: update the model parameters of the human body posture estimation model according to the total loss value.

[0059] In this embodiment, the terminal device updates the model parameters of the above-mentioned posture estimation model by gradient feedback of the total loss value to act on the optimizer, so that the posture estimation model can learn better, improve the estimation effect of the posture estimation model, and further improve the model's recognition accuracy of key points of the human body.

[0060] In the embodiment of the present application, a target image is obtained, and a posture of a human body in the target image is estimated to obtain an initial thermal map of the human body. Then, target key points whose confidences meet a preset threshold range are determined from the initial thermal map to determine key points with lower precision. Then, local components of the human body corresponding to the target key points are determined to facilitate processing of the local components and improve the accuracy of posture estimation of the human body. Specifically, a preset number of component images corresponding to the local components are obtained from a preset component library, and then data enhancement is performed on the initial thermal map according to the preset number of component images to improve the accuracy of posture estimation of the human body, and finally an enhanced thermal map of the human body is obtained, so that key points of the human body with higher precision are obtained through the enhanced thermal map of the human body, thereby improving the accuracy of extracting key points of the human body by performing data enhancement on the thermal map obtained by posture estimation.

[0061] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0062] Corresponding to the data enhancement method for human posture estimation described above, Figure 3 FIG. 1 is a schematic diagram of a data enhancement device for human posture estimation in an embodiment of the present application. Figure 3 As shown, the data enhancement device for the above-mentioned human posture estimation may include:

[0063] The image acquisition module 301 is used to acquire a target image, estimate the posture of a human body in the target image, and obtain an initial thermal map of the human body.

[0064] The key point determination module 302 is used to determine the target key points whose confidence levels meet a preset threshold range from the initial heat map.

[0065] The component diagram acquisition module 303 is used to determine the local components of the human body corresponding to the target key points, and to acquire a preset number of component diagrams corresponding to the local components from a preset component library.

[0066] The data enhancement module 304 is used to perform data enhancement on the initial thermal map according to a preset number of component maps to obtain an enhanced human body thermal map.

[0067] In one embodiment, the data enhancement device for human body posture estimation may further include:

[0068] The heat map acquisition module is used to obtain a label heat map of the human body and determine a first loss value according to the label heat map and the initial heat map.

[0069] The loss value determination module is used to determine a second loss value based on the enhanced human body thermal map and the initial thermal map.

[0070] The processing module is used to perform weighted processing on the first loss value and the second loss value, and determine the total loss value according to the weighted first loss value and the second loss value.

[0071] The parameter updating module is used to update the model parameters of the human body posture estimation model according to the total loss value.

[0072] In one embodiment, the component diagram acquisition module 303 may include:

[0073] The key point determination unit is used to determine adjacent key points connected to the target key point.

[0074] The component determination unit is used to determine the local component according to the target key point and the adjacent key points.

[0075] In one embodiment, the component determination unit may include:

[0076] The key point selection subunit is used to select an adjacent key point with the smallest confidence from the at least two adjacent key points when there are at least two adjacent key points.

[0077] The component determination subunit is used to determine the component between the target key point and the adjacent key point with the smallest confidence as a local component.

[0078] In one embodiment, the data enhancement module 304 may include:

[0079] The component image pasting unit is used to randomly paste a preset number of component images on the human body in the target image to obtain a preset number of human body sample images.

[0080] The processing unit is used to process a preset number of human body sample images through a preset posture estimation algorithm to obtain an enhanced human body heat map.

[0081] In one embodiment, the data enhancement device for human body posture estimation may further include:

[0082] The length calculation unit is used to calculate the component length of the local component according to the target key point.

[0083] The normalization processing unit is used to perform normalization processing on a preset number of component drawings according to the component lengths.

[0084] In one embodiment, the data enhancement device for human body posture estimation may further include:

[0085] The cropping unit is used to obtain a global image of the human body, and to crop the global image according to a preset segmentation network to obtain images of various local components.

[0086] The confidence calculation unit is used to calculate the confidence of each local component diagram, and store the local component diagram with a confidence greater than a preset confidence threshold in a corresponding position in the component library.

[0087] In the embodiment of the present application, a target image is obtained, and a posture of a human body in the target image is estimated to obtain an initial thermal map of the human body. Then, target key points whose confidences meet a preset threshold range are determined from the initial thermal map to determine key points with lower precision. Then, local components of the human body corresponding to the target key points are determined to facilitate processing of the local components and improve the accuracy of posture estimation of the human body. Specifically, a preset number of component images corresponding to the local components are obtained from a preset component library, and then data enhancement is performed on the initial thermal map according to the preset number of component images to improve the accuracy of posture estimation of the human body, and finally an enhanced thermal map of the human body is obtained, so that key points of the human body with higher precision are obtained through the enhanced thermal map of the human body, thereby improving the accuracy of extracting key points of the human body by performing data enhancement on the thermal map obtained by posture estimation.

[0088] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned system embodiments and method embodiments, and will not be repeated here.

[0089] Figure 4This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0090] like Figure 4 As shown, the terminal device 4 of this embodiment includes: at least one processor 400 ( Figure 4 Only one is shown in the figure), a memory 401 connected to the above-mentioned processor 400, and a computer program 402 stored in the above-mentioned memory 401 and executable on the above-mentioned at least one processor 400, such as a data enhancement program for human posture estimation. When the above-mentioned processor 400 executes the above-mentioned computer program 402, the steps in the above-mentioned data enhancement method embodiments for human posture estimation are implemented, such as Figure 1 Alternatively, when the processor 400 executes the computer program 402, the functions of the modules in the above-mentioned device embodiments are realized, for example, Figure 3 Functions of modules 301 to 304 are shown.

[0091] Exemplarily, the computer program 402 may be divided into one or more modules, which are stored in the memory 401 and executed by the processor 400 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 402 in the terminal device 4. For example, the computer program 402 may be divided into an image acquisition module 301, a key point determination module 302, a component diagram acquisition module 303, and a data enhancement module 304, and the specific functions of each module are as follows:

[0092] The image acquisition module 301 is used to acquire a target image, estimate the posture of a human body in the target image, and obtain an initial thermal map of the human body;

[0093] A key point determination module 302 is used to determine target key points whose confidences meet a preset threshold range from the initial heat map;

[0094] A component diagram acquisition module 303 is used to determine the local components of the human body corresponding to the target key points, and to acquire a preset number of component diagrams corresponding to the local components from a preset component library;

[0095] The data enhancement module 304 is used to perform data enhancement on the initial thermal map according to a preset number of component maps to obtain an enhanced human body thermal map.

[0096] The terminal device 4 may include, but is not limited to, a processor 400 and a memory 401. Those skilled in the art will appreciate that Figure 4It is only an example of the terminal device 4 and does not constitute a limitation on the terminal device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, buses, etc.

[0097] The processor 400 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0098] In some embodiments, the memory 401 may be an internal storage unit of the terminal device 4, such as a hard disk or memory of the terminal device 4. In other embodiments, the memory 401 may also be an external storage device of the terminal device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 4. Further, the memory 401 may also include both an internal storage unit and an external storage device of the terminal device 4. The memory 401 is used to store an operating system, an application program, a boot loader, data, and other programs, such as the program code of the computer program. The memory 401 may also be used to temporarily store data that has been output or is to be output.

[0099] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For the part that is not described or recorded in detail in a certain embodiment, refer to the relevant description of other embodiments.

[0100] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0101] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the above modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0102] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0103] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0104] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A data augmentation method for human pose estimation, characterized in that, it includes: Obtain a target image, perform pose estimation on the human body in the target image through a preset pose estimation algorithm to obtain an initial heatmap of the human body; wherein, the pose estimation algorithm is implemented through a pose estimation model; Determine target key points in the initial heatmap whose confidence levels meet a preset threshold range; Determine the local components of the human body corresponding to the target key points, and obtain a preset number of component maps corresponding to the local components from a preset component library; Perform data augmentation on the initial heatmap according to the preset number of component maps to obtain a preset number of enhanced human body heatmaps.

2. The data augmentation method for human pose estimation according to claim 1, characterized in that, after obtaining the enhanced human body heatmap, it further includes: Obtain a label heatmap of the human body, and determine a first loss value according to the label heatmap and the initial heatmap; Determine a second loss value according to the enhanced human body heatmap and the initial heatmap; Perform weighted processing on the first loss value and the second loss value, and determine a total loss value according to the weighted first loss value and the second loss value; Update the model parameters of the human body pose estimation model according to the total loss value.

3. The data augmentation method for human pose estimation according to claim 1, characterized in that, the determination of the local components of the human body corresponding to the target key points includes: Determine adjacent key points connected to the target key points; Determine the local components according to the target key points and the adjacent key points.

4. The data augmentation method for human pose estimation according to claim 3, characterized in that, the determination of the local components according to the target key points and the adjacent key points includes: When there are at least two adjacent key points, select the adjacent key point with the lowest confidence level from the at least two adjacent key points; Determine the component between the target key point and the adjacent key point with the lowest confidence level as the local component.

5. The data augmentation method for human pose estimation according to claim 1, characterized in that, the performing data augmentation on the initial heatmap according to the preset number of component maps to obtain an enhanced human body heatmap includes: Randomly paste the preset number of component maps on the human body in the target image to obtain a preset number of human body sample maps; Process the preset number of human body sample maps through a preset pose estimation algorithm to obtain an enhanced human body heatmap.

6. The data augmentation method for human pose estimation according to claim 1, characterized in that, before performing data augmentation on the initial heatmap according to the preset number of component maps, it further includes: Calculate the component length of the local component according to the target key point; Perform normalization processing on the preset number of component maps according to the component length.

7. The data augmentation method for human pose estimation according to claim 1, characterized in that, before obtaining the preset number of component maps corresponding to the local components from a preset component library, it further includes: Obtain the global image of the human body, and crop the global image according to a preset segmentation network to obtain various local component images; Calculate the confidence levels of the various local component images, and store the local component images with confidence levels greater than a preset confidence threshold at corresponding positions in the component library.

8. A data augmentation device for human pose estimation, characterized in that, it includes: An image acquisition module, configured to acquire a target image, and perform pose estimation on the human body in the target image through a preset pose estimation algorithm to obtain an initial heat map of the human body; wherein, the pose estimation algorithm is implemented through a pose estimation model; A key point determination module, configured to determine target key points with confidence levels meeting a preset threshold range from the initial heat map; A component image acquisition module, configured to determine local components of the human body corresponding to the target key points, and acquire a preset number of component images corresponding to the local components from a preset component library; A data augmentation module, configured to perform data augmentation on the initial heat map according to the preset number of component images to obtain a preset number of enhanced human body heat maps.

9. A terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, the steps of a data augmentation method for human pose estimation according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, the steps of a data augmentation method for human pose estimation according to any one of claims 1 to 7 are implemented.

Images