Heat Map Generation Method and Device

Abstract

The present invention discloses a method and apparatus for generating a heat map. Wherein, the method includes: obtaining a target image and annotation information of the target image, wherein the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image; determining scale information of the target object based on the positions of at least one key point; obtaining a target Gaussian kernel corresponding to the scale information; and generating a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel. The present invention solves the technical problem that the generated heat map in the related art is difficult to effectively cover the key points.

Description

Technical Field

[0001] The present invention relates to the field of heat map generation, and in particular, to a method and device for generating a heat map. Background Art

[0002] In deep learning-based multi-person human pose estimation methods, heat maps of human key points are usually predicted instead of directly regressing the coordinates of the key points. When generating the ground truth of the task, a Gaussian kernel with a unified standard deviation is usually used to generate the heat map. However, due to the different scales of the human body and the different actual regional sizes of each key point of the human body, the heat maps generated with the same standard deviation will result in a large difference in the covered human body regions of different scales.

[0003] For the above problems, no effective solution has been proposed yet. Summary of the Invention

[0004] Embodiments of the present invention provide a method and device for generating a heat map, so as to at least solve the technical problem that the generated heat map in related technologies is difficult to effectively cover key points.

[0005] According to one aspect of the embodiments of the present invention, a method for generating a heat map is provided, including: obtaining a target image and annotation information of the target image, where the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image; determining scale information of the target object based on the positions of at least one key point; obtaining a target Gaussian kernel corresponding to the scale information; and generating a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel.

[0006] According to another aspect of the embodiments of the present invention, a method for generating a heat map is provided, including: displaying a target image and annotation information of the target image in an interaction interface, where the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image; and in response to an operation instruction detected in the interaction interface, displaying a target heat map corresponding to the target image in the interaction interface, where the target heat map is generated by the target image, the annotation information, and a target Gaussian kernel, the target Gaussian kernel corresponds to the scale information of the target object, and the scale information is determined by the positions of at least one key point.

[0007] According to another aspect of the embodiments of the present invention, there is also provided a method for generating a heat map, including: a cloud server receives original data uploaded by a client, where the original data includes: a target image and annotation information of the target image, the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image; the cloud server determines scale information of the target object based on the positions of at least one key point; the cloud server obtains a target Gaussian kernel corresponding to the scale information; the cloud server generates a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel.

[0008] According to another aspect of the embodiments of the present invention, there is also provided a device for generating a heat map, including: a first acquisition module for acquiring a target image and annotation information of the target image, where the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image; a determination module for determining scale information of the target object based on the positions of at least one key point; a second acquisition module for acquiring a target Gaussian kernel corresponding to the scale information; a generation module for generating a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel.

[0009] According to another aspect of the embodiments of the present invention, there is also provided a storage medium, where the storage medium includes a stored program, and when the program runs, it controls the device where the computer-readable storage medium is located to execute the method for generating a heat map in any one of the above embodiments.

[0010] According to another aspect of the embodiments of the present invention, there is also provided a computer terminal, including: a processor and a memory, the processor is used to run the program stored in the memory, and when the program runs, it executes the method for generating a heat map in any one of the above embodiments.

[0011] According to another aspect of the embodiments of the present invention, there is also provided a heat map generation system, including: a processor; and a memory, connected to the processor, for providing instructions for the processor to perform the following processing steps: acquiring a target image and annotation information of the target image, where the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image; determining scale information of the target object based on the positions of at least one key point; acquiring a target Gaussian kernel corresponding to the scale information; generating a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel.

[0012] Through the above steps, first, obtain the target image and the annotation information of the target image. Among them, the target image contains the target object, and the target object contains at least one key point. The annotation information includes the positions of at least one key point in the target image. Based on the positions of at least one key point, determine the scale information of the target object. Obtain the target Gaussian kernel corresponding to the scale information. Based on the target image, the annotation information, and the target Gaussian kernel, generate the target heat map corresponding to the target image, which improves the accuracy of the heat map covering the key points. It is easy to notice that the position in the target image can be determined according to at least one key point contained in the target object, and the scale information of the target object can be determined according to the positions of the key points, so as to generate the target heat map corresponding to the target object in the target image according to the Gaussian kernel corresponding to the scale information. Since the Gaussian kernel changes flexibly according to the scale information of the target object, the target heat map corresponding to the generated target image can better cover the key points according to the scale information of the target object, thereby solving the technical problem that the heat map generated in the related art is difficult to effectively cover the key points. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present invention, form a part of this application, and the schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0014] Figure 1 is a schematic diagram of a heat map in the related art;

[0015] Figure 2 is another schematic diagram of a heat map in the related art;

[0016] Figure 3 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a heat map generation method according to an embodiment of the present invention;

[0017] Figure 4 is a flowchart of a heat map generation method according to an embodiment of the present invention;

[0018] Figure 5 is a schematic diagram of a heat map according to an embodiment of the present invention;

[0019] Figure 6 is another schematic diagram of a heat map according to an embodiment of the present invention;

[0020] Figure 7 is a flowchart of another heat map generation method according to an embodiment of the present invention;

[0021] Figure 8 is a flowchart of another heat map generation method according to an embodiment of the present invention;

[0022] Figure 9 is a schematic diagram of a heat map generating device according to an embodiment of the present invention;

[0023] Figure 10 is a schematic diagram of another heat map generating device according to an embodiment of the present invention;

[0024] Figure 11 is a schematic diagram of another heat map generating device according to an embodiment of the present invention;

[0025] Figure 12 is a structural block diagram of a computer terminal according to an embodiment of the present invention. Detailed implementation manners

[0026] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0028] First, some nouns or terms that appear in the process of describing the embodiments of the present invention are applicable to the following explanations:

[0029] Human pose estimation: Also known as human key point detection, it locates and estimates the positions of key points such as the head, shoulder joints, and elbow joints of the human body in an image.

[0030] Multi-person human pose estimation: Input an image containing multiple people and estimate the positions of all key points of all people.

[0031] Heatmap: There are different values in different places on an image or feature map, and its distribution can represent the probability of having a target at that point.

[0032] Ground Truth of the task: The ground truth of the task here refers to the heat map.

[0033] Current solutions all directly use a Gaussian kernel with a fixed standard deviation to generate the Heatmap of key points, which is specifically implemented through the following formula where (x, y) are the coordinates of the key point, and (p x , p y ) is the area of the Gaussian kernel, where is the area of the Gaussian kernel.

[0034] As can be seen from the above, σ in the formula is a fixed value, without considering the regions of human bodies with different scales and different key point parts in the human body. This will result in the generated Heatmap having the same Gaussian kernel size for any human body and any part, leading to semantic confusion and being unfavorable for the training and learning of the network.

[0035] Figure 1 Figure 1 shows a schematic diagram of a heat map in the related art. As Figure 1 shown, for the larger human body in the figure, the Gaussian distribution value of the key point of the nose only covers the area around the nose. However, for the same Gaussian kernel size, for the smaller human body in the figure, the Gaussian distribution value of the key point of the nose will cover the entire face. At the same time, for different parts of a person, with the same Gaussian kernel size, when it just covers the nose, it may only cover a small part of the shoulder area.

[0036] Figure 2 Figure 2 shows another schematic diagram of a heat map in the related art. As Figure 2 shown, for the larger human body in the figure, the key points of each joint can be accurately marked. However, for the smaller human body in the figure, the Gaussian distribution value of the key point of the nose will cover the entire human body, and the coverage accuracy is relatively low.

[0037] Due to the problem of accurate annotation of the joint area, that is, any point in a large area near the human joint area can be regarded as the annotation of the key point. Therefore, when generating the Heatmap, it is best to cover the key point area with a suitable size as much as possible, neither too large nor too small. However, the current methods do not achieve this.

[0038] In this application, heat maps with different Gaussian kernel sizes can be generated through the scale information of the human body. At the same time, different Gaussian kernel sizes are also used for different key point parts of a person, so as to be able to generate the target heat map corresponding to the scale information of the human body, and improve the technical problem that the heat map generated in the related art is difficult to effectively cover the key points.

[0039] Embodiment 1

[0040] According to an embodiment of the present invention, an embodiment of a method for generating a heat map is further provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0041] The method embodiment provided by the first embodiment of the present invention can be executed on a mobile terminal, a computer terminal or a similar computing device. Figure 3 A hardware structure block diagram of a computer terminal (or mobile device) for implementing the heat map generation method is shown. As Figure 3 shown, the computer terminal 10 (or mobile device 10) may include one or more (shown as 102a, 102b,..., 102n in the figure) processors (the processor may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 3 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may further include more or fewer components than Figure 3 shown, or have a different configuration from Figure 3 shown.

[0042] It should be noted that the above one or more processors and / or other data processing circuits are generally referred to as "data processing circuits" in this article. The data processing circuit may be embodied in whole or in part as software, hardware, firmware, or any combination thereof. In addition, the data processing circuit may be a single independent processing module, or be incorporated in whole or in part into any one of the other elements in the computer terminal 10 (or mobile device). As involved in the embodiments of the present invention, the data processing circuit is used for processor control (such as the selection of a variable resistor terminal path connected to an interface).

[0043] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the heat map generation method in the embodiments of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above heat map generation method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely provided with respect to the processor, and these remote memories can be connected to the computer terminal 10 through a network. Examples of the above network include but are not limited to the Internet, enterprise intranets, local area networks, mobile communication networks, and combinations thereof.

[0044] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0045] The display can be, for example, a touch-screen liquid crystal display (LCD), and the liquid crystal display enables a user to interact with the user interface of the computer terminal 10 (or mobile device).

[0046] It should be noted here that in some alternative embodiments, the above Figure 3 shown computer device (or mobile device) may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware elements and software elements. It should be pointed out that Figure 3 is only an example of a specific specific instance, and is intended to show the types of components that may exist in the above computer device (or mobile device).

[0047] Under the above operating environment, the present application provides a heat map generation method as Figure 4 shown. Figure 4 is a flowchart of the heat map generation method according to Embodiment 1 of the present invention.

[0048] Step S402, obtain a target image and annotation information of the target image.

[0049] Among them, the target image contains a target object, the target object contains at least one key point, and the annotation information includes the positions of at least one key point in the target image.

[0050] The above-mentioned target image may contain a human face, a human body, etc. The above-mentioned target object may be a human face, a human body, etc.

[0051] The above-mentioned at least one key point can be used to represent the key points of the facial features in a human face, such as eyes, nose, mouth, etc. The above-mentioned at least one key point can also be used to represent the key points of the joints in a human body, such as the neck, hip, elbow, knee joint, etc.

[0052] The above-mentioned annotation information may be the position information of at least one key point pre-annotated on the human face or human body in the target image.

[0053] In an alternative embodiment, the target image and the annotation information of the target image can be obtained, so as to determine the positions of at least one key point corresponding to the target object in the target image through the annotation information in the target image. After determining the positions of at least one key point in the target image, the approximate scale of the human body in the target image can be determined simply and quickly according to this position, so that a relatively accurate heat map can be generated.

[0054] Step S404, determine the scale information of the target object based on the positions of at least one key point.

[0055] The above-mentioned scale information can represent the distances between various parts of the human body. For example, it can be the distance from the neck to the hip in the human body, or the distance from the head to the foot, etc. The scale information can also represent the distances between various organs of the human face. For example, it can be the distance from the eye to the mouth in the human face, or the distance between the two eyes in the human face, etc.

[0056] In an alternative embodiment, the scale information of the target object can be determined according to the positions of at least one key point, so as to determine the corresponding Gaussian kernel according to the scale information of the target object, so that the Gaussian kernel can better cover the key points.

[0057] Step S406, obtain the target Gaussian kernel corresponding to the scale information.

[0058] The above-mentioned target Gaussian kernel is mainly used to cover the positions of the key points in the target image.

[0059] In an alternative embodiment, a corresponding target Gaussian kernel can be determined according to the scale information. If the scale information indicates that the distances between various organs or parts of the target object in the target image are short, it means that the area occupied by the target object is small. At this time, a smaller target Gaussian kernel can be determined to cover the key points of the target object, avoiding the situation where a Gaussian kernel covers the entire face or the entire body, and also avoiding a large coverage area of a Gaussian kernel, resulting in inaccurate coverage.

[0060] In another alternative embodiment, if the scale information indicates that the distances between various organs or parts of the target object in the target image are long, it means that the area occupied by the target object is large. At this time, a larger target Gaussian kernel can be determined to cover the key points of the target object, avoiding the incomplete coverage of the key points due to a small Gaussian kernel.

[0061] By obtaining the target Gaussian kernel corresponding to the scale information, a Gaussian kernel adapted to the size of the target object in the target image can be obtained, so that the Gaussian kernel can accurately and effectively cover the key points of the target object, thereby improving the accuracy of the pose estimation algorithm for the target object in the target image according to the generated heat map.

[0062] Step S408: Generate a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel.

[0063] In an alternative embodiment, after obtaining the target Gaussian kernel, the target Gaussian kernel can be annotated on at least one key point corresponding to the target object in the target image based on the annotation information to generate a target heat map corresponding to the target image.

[0064] In an alternative embodiment, in the application scenario of a live video, a video frame containing various joints or facial organs of the anchor can be obtained, and the video frame and the annotation information corresponding to the video frame can be obtained. The annotation information is used to annotate the positions of various joints or facial organs of the anchor in the video frame. The scale information of the anchor can be determined according to the positions of various joints or facial organs of the anchor, and the scale information of the target Gaussian kernel required for generating the heat map can be determined according to the scale information of the anchor. A heat map of various joints or facial organs of the anchor in the video frame can be generated according to the video frame, the annotation information, and the target Gaussian kernel, so as to beautify the various organs of the anchor or beautify the various joints of the anchor according to the heat map, and send the beautified video frame to the client.

[0065] In another alternative embodiment, in the application scenario of short videos, a target image containing various joints of the short video blogger or various organs of the face can be obtained, and the annotation information corresponding to the target image and the target object can be obtained. The annotation information is used to annotate the positions of various joints of the short video blogger or various organs of the face in the target image. The scale information of the short video blogger in the video can be determined according to the positions of various joints of the short video blogger or various organs of the face, and the scale information of the target Gaussian kernel required for the heat map can be determined according to the scale information. A heat map of various joints of the short video blogger or various organs of the face in the target image can be generated according to the target image, the annotation information, and the target Gaussian kernel, so as to beautify various organs of the short video blogger or beautify various joints of the short video blogger according to the heat map, and send the beautified or body-beautified target image to the client.

[0066] Through the above steps, first, the target image and the annotation information of the target image are obtained, where the target image includes the target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image; based on the positions of at least one key point, the scale information of the target object is determined; the target Gaussian kernel corresponding to the scale information is obtained; based on the target image, the annotation information, and the target Gaussian kernel, the target heat map corresponding to the target image is generated, which improves the accuracy of the heat map covering the key points. It is easy to note that the position in the target image can be determined according to at least one key point included in the target object, and the scale information of the target object can be determined according to the position of the key point, so as to generate the target heat map corresponding to the target object in the target image according to the Gaussian kernel corresponding to the scale information. Since the Gaussian kernel changes flexibly according to the scale information of the target object, the target heat map corresponding to the generated target image can better cover the key points according to the scale information of the target object, thereby solving the technical problem that the heat map generated in the related art is difficult to effectively cover the key points.

[0067] In the above embodiments of the present application, when the target object includes multiple key points, based on the positions of at least one key point, determining the scale information of the target object includes: determining target key points among the multiple key points, where the distance between the target key points remains fixed relative to the target image; based on the positions of the target key points, determining the scale information of the target object.

[0068] In the case where the target object is a human body, the above-mentioned target key points can be the neck and hips in the human body. The reason for choosing the neck and hips as the target key points in the human body is that the distance between these two parts is relatively fixed and always perpendicular to the angle of image capture. It will not easily change the scale of the human body due to reasons such as side-facing. At the same time, these two parts are visible in most data cases. If other parts are selected, such as the distance between the left and right shoulders as the scale measurement standard, for people of the same size, the shoulder distance when facing forward and the shoulder distance when side-facing will vary greatly in the two-dimensional image, but subjectively, the size of the human body is the same whether it is facing forward or side-facing and should not change. If limbs such as arms are selected as the measurement standard, due to the complex movement of the limbs and frequent occlusion, it is not possible to measure well.

[0069] In an alternative embodiment, the target key points among multiple key points can be determined. Among them, there can be multiple target key points. For example, the target key points can be two. Since the distance between the target key points is fixed relative to the target image, therefore, based on the positions of the two target key points, the scale information of the target object can be determined, and the overall size of the target object at present can be judged, so as to be able to determine the corresponding target Gaussian kernel.

[0070] In the case where the target object is a human face, the above-mentioned target key points can be the nose and mouth in the human face. Since the positions of the mouth and nose in the human face are fixed relative to each other and always perpendicular to the angle of image capture, and will not easily change the scale of the human body due to reasons such as side-facing, therefore, the nose and mouth in the human face can be used as the target key points in the human face.

[0071] In the above embodiments of the present application, in the case where there are multiple target key points, determining the scale information of the target object based on the positions of the target key points includes: determining the distance between the multiple target key points based on the positions of the multiple target key points; obtaining the product of the preset coefficient and the distance to obtain the scale information.

[0072] The above-mentioned preset coefficient can be set by oneself.

[0073] In an alternative embodiment, the positions of multiple target key points can be obtained to obtain the distance between the multiple target key points. For example, the length from the neck to the hips in the human body can be obtained, and thus the scale information can be expressed by the following formula:

[0074] S p =λ*d(P neck ,P hip );

[0075] Among them, S p represents the scale information, d represents the distance between two key points, and Pneck represents the position of the neck, P hip represents the position of the hip, and λ is a set coefficient.

[0076] In the above embodiments of the present application, the target Gaussian kernel includes sub-Gaussian kernels corresponding to each key point. Among them, obtaining the target Gaussian kernel corresponding to the scale information includes: determining the weight coefficient corresponding to each key point based on the type of each key point; obtaining the product of the weight coefficient and the scale information to obtain the sub-Gaussian kernel corresponding to each key point.

[0077] The types of the above key points can represent different parts or different organs.

[0078] The above weight coefficients can be determined according to the size of the part or organ. If the area occupied by the part is large, a larger weight coefficient can be set. For example, since the area of the shoulder is large, a larger corresponding weight information can be set. For example, since the areas of the eyes and nose are small, smaller corresponding weight information can be set.

[0079] For different key point parts, the size of the corresponding Gaussian kernel can be adjusted. Optionally, it can be represented by the following Gaussian kernel standard deviation formula:

[0080] σ p =γ*5 p ;

[0081] where σ p can be the sub-Gaussian kernel of the key point, γ can be the coefficient of the Gaussian kernel corresponding to each key point, and S p represents the scale information.

[0082] It should be noted that for key points in areas with a small size such as the eyes and nose, γ can be 1, and for key point parts in other larger areas, a larger value can be taken. For example, γ can be 1.5.

[0083] Figure 5 is a schematic diagram of a heat map according to an embodiment of the present invention. As Figure 5 shown, for the larger human body in the figure, the Gaussian kernel of the key point of the nose covers the area around the nose. For the smaller human body in the figure, the Gaussian sum of the key point of the nose also covers the area around the nose. The coverage accuracy of its Gaussian kernel is relatively high, so the generated heat map has a relatively high accuracy, which is convenient for accurately estimating the human posture.

[0084] Figure 6 is another schematic diagram of a heat map according to an embodiment of the present invention. As Figure 6As shown, the key points of each joint in the larger human body in the figure can be accurately marked. Similarly, for the smaller human body in the figure, the key points of each joint can also be accurately marked. The coverage accuracy of its Gaussian kernel is relatively high, so the generated heat map has a relatively high accuracy, which is convenient for accurately estimating the human pose.

[0085] In the above embodiments of the present application, after generating the target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel, the method further includes: generating training data based on the target image and the target heat map; training the target object detection model using the training data, where the target object detection model is used to detect at least one key point in the input image and determine the predicted position of the at least one key point in the input image.

[0086] The above-mentioned target object detection model is used to detect the pose of the target object in the target image.

[0087] In an alternative embodiment, training data can be generated according to the target image and its corresponding target heat map, so as to train the target object detection model according to the training data, so that the target object detection model can recognize the heat map corresponding to the input image, so as to determine the predicted position of the key points of the target object in the input image according to the heat map, and thus determine the pose of the target object according to the key points and the predicted positions corresponding to the key points.

[0088] In the above embodiments of the present application, before generating training data based on the target image and the target heat map, the method further includes: outputting the target image and the target heat map; receiving a feedback image generated by operating on the target heat map, where the feedback image is used to represent the image obtained after modifying the target heat map; generating training data based on the target image and the feedback image.

[0089] In an alternative embodiment, the target image and the target heat map can be output to the display interface of the client, so that the client can correct the target heat map corresponding to the target image to obtain a feedback image, the feedback image can be obtained, and training data can be generated according to the target image and the feedback image. By correcting the generated heat map, the accuracy of the training data can be further improved. Furthermore, it can be beneficial to network training and improve the accuracy of human pose estimation.

[0090] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0091] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that makes contributions to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of the various embodiments of the present invention.

[0092] Embodiment 2

[0093] According to an embodiment of the present invention, there is also provided an embodiment of a heat map generation method. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions. And, although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from here.

[0094] Figure 7 is a flowchart of a heat map generation method according to an embodiment of the present invention, as Figure 7 shown, the method may include the following steps:

[0095] Step S702, display a target image and annotation information of the target image in an interaction interface.

[0096] Among them, the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image.

[0097] Step S704, in response to an operation instruction detected in the interaction interface, display a target heat map corresponding to the target image in the interaction interface.

[0098] Among them, the target heat map is generated by the target image, the annotation information, and a target Gaussian kernel. The target Gaussian kernel corresponds to the scale information of the target object, and the scale information is determined by the positions of at least one key point.

[0099] In the above embodiments of the present application, when the target object includes multiple key points, the method further includes: determining a target key point among the multiple key points, where the distance between the target key points is fixed with respect to the target image; determining the scale information of the target object based on the positions of the target key points.

[0100] In the above embodiments of the present application, when the target object includes multiple key points, the method further includes: determining the distances between multiple target key points based on the positions of the multiple target key points; obtaining the product of a preset coefficient and the distance to obtain the scale information.

[0101] In the above embodiments of the present application, when the target object is a human body, the target key points include: the neck and the hips.

[0102] In the above embodiments of the present application, the target Gaussian kernel includes sub-Gaussian kernels corresponding to each key point, and the method further includes: determining the weight coefficient corresponding to each key point based on the type of each key point; obtaining the product of the weight coefficient and the scale information to obtain the sub-Gaussian kernel corresponding to each key point.

[0103] In the above embodiments of the present application, the method further includes: generating training data based on the target image and the target heat map; training the target object detection model using the training data, where the target object detection model is used to detect at least one key point in the input image and determine the predicted position of the at least one key point in the input image.

[0104] In the above embodiments of the present application, the method further includes: outputting the target image and the target heat map; receiving a feedback image generated by operating on the target heat map, where the feedback image is used to represent the image obtained after modifying the target heat map; generating training data based on the target image and the feedback image.

[0105] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0106] Embodiment 3

[0107] According to an embodiment of the present invention, an embodiment of a heat map generation method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0108] Figure 8 is a flowchart of a method for processing the emission flux of atmospheric pollutants according to an embodiment of the present invention, as Figure 8As shown, the method may include the following steps:

[0109] Step S802, the cloud server receives the original data uploaded by the client.

[0110] Among them, the original data includes: a target image and annotation information of the target image. The target image contains a target object, the target object contains at least one key point, and the annotation information includes the position of at least one key point in the target image.

[0111] Step S804, the cloud server determines the scale information of the target object based on the positions of at least one key point.

[0112] Step S806, the cloud server obtains the target Gaussian kernel corresponding to the scale information.

[0113] Step S808, the cloud server generates a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel.

[0114] In the above embodiments of the present application, when the target object contains multiple key points, the cloud server determines the scale information of the target object based on the positions of at least one key point, including: the cloud server determines the target key points among the multiple key points, where the distance between the target key points remains fixed relative to the target image; the cloud server determines the scale information of the target object based on the positions of the target key points.

[0115] In the above embodiments of the present application, when there are multiple target key points, the cloud server determines the scale information of the target object based on the positions of the target key points, including: the cloud server determines the distance between the multiple target key points based on the positions of the multiple target key points; the cloud server obtains the product of the preset coefficient and the distance to obtain the scale information.

[0116] In the above embodiments of the present application, when the target object is a human body, the target key points include: the neck and the hip.

[0117] In the above embodiments of the present application, the target Gaussian kernel contains sub-Gaussian kernels corresponding to each key point. Among them, when the cloud server obtains the target Gaussian kernel corresponding to the scale information, it includes: the cloud server determines the weight coefficient corresponding to each key point based on the type of each key point; the cloud server obtains the product of the weight coefficient and the scale information to obtain the sub-Gaussian kernel corresponding to each key point.

[0118] In the above embodiments of the present application, after the cloud server generates the target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel, the method further includes: the cloud server generates training data based on the target image and the target heat map; the cloud server uses the training data to train the target object detection model, where the target object detection model is used to detect at least one key point in the input image and determine the predicted positions of the at least one key point in the input image.

[0119] In the above embodiments of the present application, before the cloud server generates training data based on the target image and the target heat map, the method further includes: the cloud server outputs the target image and the target heat map; the cloud server receives the feedback image generated by operating on the target heat map, where the feedback image is used to represent the image obtained after modifying the target heat map; the cloud server generates training data based on the target image and the feedback image.

[0120] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0121] Embodiment 4

[0122] According to an embodiment of the present invention, there is also provided a heat map generation device for implementing the above heat map generation method, as Figure 9 shown. The device 900 includes: a first acquisition module 902, a determination module 904, a second acquisition module 906, and a generation module 908.

[0123] Among them, the first acquisition module is used to acquire the target image and the annotation information of the target image, where the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of the at least one key point in the target image; the determination module is used to determine the scale information of the target object based on the positions of the at least one key point; the second acquisition module is used to acquire the target Gaussian kernel corresponding to the scale information; the generation module is used to generate the target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel.

[0124] It should be noted here that the above first acquisition module 902, determination module 904, second acquisition module 906, and generation module 908 correspond to steps S202 to S208 in Embodiment 1. The instances and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to the content disclosed in the above Embodiment 1. It should be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0125] In the above embodiments of the present application, the determination module includes: a first determination unit.

[0126] Among them, the first determination unit is used to determine the target key points among multiple key points, where the distance between the target key points is fixed relative to the target image; the first determination unit is further used to determine the scale information of the target object based on the positions of the target key points.

[0127] In the above embodiments of the present application, the determination module includes: a second determination unit and a first acquisition unit.

[0128] Among them, the second determination unit is used to determine the distances between multiple target key points based on the positions of the multiple target key points; the first acquisition unit is used to obtain the product of a preset coefficient and the distance to obtain the scale information.

[0129] In the above embodiments of the present application, when the target object is a human body, the target key points include: the neck and the hip.

[0130] In the above embodiments of the present application, the target Gaussian kernel includes sub-Gaussian kernels corresponding to each key point, and the second acquisition module includes a third determination unit and a second acquisition unit.

[0131] Among them, the third determination unit is used to determine the weight coefficient corresponding to each key point based on the type of each key point; the second acquisition unit is used to obtain the product of the weight coefficient and the scale information to obtain the sub-Gaussian kernel corresponding to each key point.

[0132] In the above embodiments of the present application, the device further includes: a training module.

[0133] Among them, the generation module is further used to generate training data based on the target image and the target heat map; the training module is further used to train the target object detection model using the training data, where the target object detection model is used to detect at least one key point in the input image and determine the predicted positions of the at least one key point in the input image.

[0134] In the above embodiments of the present application, the device further includes: an output module and a receiving module.

[0135] Among them, the output module is used to output the target image and the target heat map; the receiving module is used to receive the feedback image generated by operating on the target heat map, where the feedback image is used to represent the image obtained after modifying the target heat map; the generation module is used to generate training data based on the target image and the feedback image.

[0136] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0137] Embodiment 5

[0138] According to an embodiment of the present invention, there is also provided a heat map generation device for implementing the above heat map generation method, as Figure 10 shown. The device 1000 includes: a first display module 1002 and a second display module 1004.

[0139] Among them, the first display module is used to display a target image and annotation information of the target image in an interaction interface, where the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image; the second display module is used to display a target heat map corresponding to the target image in the interaction interface in response to an operation instruction detected in the interaction interface, where the target heat map is generated by the target image, the annotation information, and a target Gaussian kernel, the target Gaussian kernel corresponds to the scale information of the target object, and the scale information is determined by the positions of at least one key point.

[0140] It should be noted here that the above first display module 1002 and second display module 1004 correspond to steps S702 to S704 of Embodiment 2. The two modules have the same implementation examples and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above Embodiment 1. It should be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0141] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0142] Embodiment 6

[0143] According to an embodiment of the present invention, there is also provided a heat map generation device for implementing the above heat map generation method, as Figure 11 shown. The device includes: a receiving module 1102, a determining module 1104, an obtaining module 1106, and a generating module 1108.

[0144] Among them, the receiving module is used to receive the original data uploaded by the client, where the original data includes: a target image and annotation information of the target image, the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image; the determining module is used to determine the scale information of the target object based on the positions of at least one key point; the obtaining module is used to obtain the target Gaussian kernel corresponding to the scale information; the generating module is used to generate a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel.

[0145] It should be noted that the above receiving module 1102, determining module 1004, obtaining module 1006, and generating module 1008 correspond to steps S802 to S808 of Embodiment 3. The instances and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to the content disclosed in the above-mentioned Embodiment 1. It should be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0146] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0147] Embodiment 7

[0148] According to an embodiment of the present invention, there is also provided a heat map generation system for implementing the above heat map generation method, including: a processor; and

[0149] a memory connected to the processor for providing instructions for the processor to perform the following processing steps: obtaining a target image and annotation information of the target image, where the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image; determining scale information of the target object based on the positions of at least one key point; obtaining a target Gaussian kernel corresponding to the scale information; and generating a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel.

[0150] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0151] Embodiment 8

[0152] An embodiment of the present invention can provide a computer terminal, and the computer terminal can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the above computer terminal can also be replaced with a terminal device such as a mobile terminal.

[0153] Optionally, in this embodiment, the above computer terminal can be located in at least one of multiple network devices in a computer network.

[0154] In this embodiment, the above computer terminal may execute the program code of the following steps in the heat map generation method: obtaining a target image and annotation information of the target image, where the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image; determining the scale information of the target object based on the positions of at least one key point; obtaining a target Gaussian kernel corresponding to the scale information; and generating a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel.

[0155] Optionally, Figure 12 is a structural block diagram of a computer terminal according to an embodiment of the present invention. As Figure 12 shown, the computer terminal 10 may include: one or more (only one is shown in the figure) processors and a memory.

[0156] Among them, the memory may be used to store software programs and modules, such as program instructions / modules corresponding to the heat map generation method and device in the embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the above heat map generation method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, a flash memory, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely disposed relative to the processor, and these remote memories may be connected to the terminal A through a network. Examples of the above network include but are not limited to the Internet, an enterprise internal network, a local area network, a mobile communication network, and combinations thereof.

[0157] The processor may call the information and application programs stored in the memory through a transmission device to execute the following steps: obtaining a target image and annotation information of the target image, where the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image; determining the scale information of the target object based on the positions of at least one key point; obtaining a target Gaussian kernel corresponding to the scale information; and generating a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel.

[0158] Optionally, the above processor may further execute the program code of the following steps: in the case where there are multiple target key points, determining a target key point among the multiple key points, where the distance between the target key points is fixed relative to the target image; and determining the scale information of the target object based on the position of the target key point.

[0159] Optionally, the above-mentioned processor may also execute the program code of the following steps: when there are multiple target key points, determine the distances between the multiple target key points based on the positions of the multiple target key points; obtain the product of the preset coefficient and the distance to obtain the scale information.

[0160] Optionally, the above-mentioned processor may also execute the program code of the following steps: when the target object is a human body, the target key points include: the neck and the hips.

[0161] Optionally, the above-mentioned processor may also execute the program code of the following steps: the target Gaussian kernel includes sub-Gaussian kernels corresponding to each key point. Determine the weight coefficient corresponding to each key point based on the type of each key point; obtain the product of the weight coefficient and the scale information to obtain the sub-Gaussian kernel corresponding to each key point.

[0162] Optionally, the above-mentioned processor may also execute the program code of the following steps: generate training data based on the target image and the target heat map; use the training data to train the target object detection model, where the target object detection model is used to detect at least one key point in the input image and determine the predicted position of at least one key point in the input image.

[0163] Optionally, the above-mentioned processor may also execute the program code of the following steps: output the target image and the target heat map; receive the feedback image generated by operating on the target heat map, where the feedback image is used to represent the image obtained after modifying the target heat map; generate training data based on the target image and the feedback image.

[0164] The processor may call the information and application programs stored in the memory through the transmission device to execute the following steps: display the target image and the annotation information of the target image in the interaction interface, where the target image includes the target object, the target object includes at least one key point, and the annotation information includes the position of at least one key point in the target image; in response to the operation instruction detected in the interaction interface, display the target heat map corresponding to the target image in the interaction interface, where the target heat map is generated by the target image, the annotation information, and the target Gaussian kernel, and the target Gaussian kernel corresponds to the scale information of the target object, and the scale information is determined by the position of at least one key point.

[0165] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: The cloud server receives the original data uploaded by the client. The original data includes: a target image and annotation information of the target image. The target image contains a target object, and the target object contains at least one key point. The annotation information includes the positions of at least one key point in the target image. The cloud server determines the scale information of the target object based on the positions of at least one key point. The cloud server obtains the target Gaussian kernel corresponding to the scale information. The cloud server generates a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel.

[0166] By adopting the embodiment of the present invention, first, the target image and the annotation information of the target image are obtained. The target image contains a target object, and the target object contains at least one key point. The annotation information includes the positions of at least one key point in the target image. The scale information of the target object is determined based on the positions of at least one key point. The target Gaussian kernel corresponding to the scale information is obtained. The target heat map corresponding to the target image is generated based on the target image, the annotation information, and the target Gaussian kernel, thereby improving the accuracy of the heat map covering the key points. It is easy to notice that the positions in the target image can be determined according to at least one key point included in the target object, and the scale information of the target object can be determined according to the positions of the key points, so as to generate the target heat map corresponding to the target object in the target image according to the Gaussian kernel corresponding to the scale information. Since the Gaussian kernel flexibly changes according to the scale information of the target object, the target heat map corresponding to the generated target image can better cover the key points according to the scale information of the target object, thereby solving the technical problem that the heat map generated in the related art is difficult to effectively cover the key points.

[0167] Those of ordinary skill in the art can understand that Figure 12 The structure shown is only for illustration. The computer terminal can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, and terminal devices such as Mobile Internet Devices (MID), PAD, etc. Figure 12 It does not limit the structure of the above electronic device. For example, the computer terminal 10 may also include more or fewer components (such as a network interface, a display device, etc.) than those shown Figure 12 or have a different configuration from that shown Figure 12

[0168] ​Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by a program instructing the hardware related to the terminal device. This program can be stored in a computer-readable storage medium, and the storage medium can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disc, etc.

[0169] Embodiment 9

[0170] An embodiment of the present invention also provides a storage medium. Optionally, in this embodiment, the above storage medium can be used to store the program code executed by the heat map generation method provided in the first embodiment above.

[0171] Optionally, in this embodiment, the above storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0172] Optionally, in this embodiment, the storage medium is set to store program code for performing the following steps: obtaining a target image and annotation information of the target image, where the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image; determining the scale information of the target object based on the positions of at least one key point; obtaining a target Gaussian kernel corresponding to the scale information; and generating a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel.

[0173] Optionally, the above storage medium is further set to store program code for performing the following steps: when there are multiple target key points, determining a target key point among the multiple key points, where the distance between the target key points remains fixed relative to the target image; and determining the scale information of the target object based on the position of the target key point.

[0174] Optionally, the above storage medium is further set to store program code for performing the following steps: when there are multiple target key points, determining the distance between the multiple target key points based on the positions of the multiple target key points; and obtaining the product of a preset coefficient and the distance to obtain the scale information.

[0175] Optionally, when the target object is a human body, the target key points include: the neck and the hip.

[0176] Optionally, the above storage medium is further configured to store program code for performing the following steps: The target Gaussian kernel includes sub-Gaussian kernels corresponding to each key point. Based on the type of each key point, determine the weight coefficient corresponding to each key point; Obtain the product of the weight coefficient and the scale information to obtain the sub-Gaussian kernel corresponding to each key point.

[0177] Optionally, the above storage medium is further configured to store program code for performing the following steps: Generate training data based on the target image and the target heat map; Use the training data to train the target object detection model, where the target object detection model is used to detect at least one key point in the input image and determine the predicted position of the at least one key point in the input image.

[0178] Optionally, the above storage medium is further configured to store program code for performing the following steps: Output the target image and the target heat map; Receive the feedback image generated by operating on the target heat map, where the feedback image is used to represent the image obtained after modifying the target heat map; Generate training data based on the target image and the feedback image.

[0179] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: Display the target image and the annotation information of the target image in the interaction interface, where the target image includes the target object, the target object includes at least one key point, and the annotation information includes the position of the at least one key point in the target image; In response to the operation instruction detected in the interaction interface, display the target heat map corresponding to the target image in the interaction interface, where the target heat map is generated by the target image, the annotation information, and the target Gaussian kernel, the target Gaussian kernel corresponds to the scale information of the target object, and the scale information is determined by the position of the at least one key point.

[0180] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: The cloud server receives the original data uploaded by the client, where the original data includes: the target image and the annotation information of the target image, the target image includes the target object, the target object includes at least one key point, and the annotation information includes the position of the at least one key point in the target image; The cloud server determines the scale information of the target object based on the position of the at least one key point; The cloud server obtains the target Gaussian kernel corresponding to the scale information; The cloud server generates the target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel.

[0181] By adopting the embodiment of the present invention, first, a target image and annotation information of the target image are obtained, wherein the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of at least one key point in the target image; based on the positions of at least one key point, the scale information of the target object is determined; a target Gaussian kernel corresponding to the scale information is obtained; based on the target image, the annotation information and the target Gaussian kernel, a target heat map corresponding to the target image is generated, which improves the accuracy of the heat map covering the key points. It is easy to note that the positions in the target image can be determined according to at least one key point included in the target object, and the scale information of the target object can be determined according to the positions of the key points, so as to generate a target heat map corresponding to the target object in the target image according to the Gaussian kernel corresponding to the scale information. Since the Gaussian kernel changes flexibly according to the scale information of the target object, the target heat map corresponding to the generated target image can better cover the key points according to the scale information of the target object, thereby solving the technical problem that the heat map generated in the related art is difficult to effectively cover the key points.

[0182] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0183] In the above embodiments of the present invention, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0184] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of units or modules can be in an electrical or other form.

[0185] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0186] In addition, in each embodiment of the present invention, each functional unit can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0187] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention. The foregoing storage medium includes: various media such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0188] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for generating a heat map, characterized in that, Including: Obtain a target image and annotation information of the target image, where the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of the at least one key point in the target image; Determine scale information of the target object based on the positions of the at least one key point; Determine a weight coefficient corresponding to each key point based on the type of each key point; Obtain the product of the weight coefficient and the scale information to obtain a sub-Gaussian kernel corresponding to each key point; Generate a target heat map corresponding to the target image based on the target image, the annotation information, and a target Gaussian kernel, where the target Gaussian kernel includes the sub-Gaussian kernel corresponding to each key point.

2. The method according to claim 1, wherein When the target object includes multiple key points, determining the scale information of the target object based on the positions of the at least one key point includes: Determine target key points among the multiple key points, where the distance between the target key points remains fixed with respect to the target image; Determine the scale information of the target object based on the positions of the target key points.

3. The method according to claim 2, wherein When there are multiple target key points, determining the scale information of the target object based on the positions of the target key points includes: Determine the distance between the multiple target key points based on the positions of the multiple target key points; Obtain the product of a preset coefficient and the distance to obtain the scale information.

4. The method according to claim 2, wherein When the target object is a human body, the target key points include: the neck and the hips.

5. The method according to any one of claims 1 to 4, characterized in that, After generating the target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel, the method further includes: Generate training data based on the target image and the target heat map; Train a target object detection model using the training data, where the target object detection model is used to detect the at least one key point in an input image and determine the predicted positions of the at least one key point in the input image.

6. The method according to claim 5, characterized in that, Before generating the training data based on the target image and the target heat map, the method further includes: Output the target image and the target heat map; Receive a feedback image generated by operating on the target heat map, where the feedback image is used to represent an image obtained after modifying the target heat map; Generate the training data based on the target image and the feedback image.

7. A method for generating a heat map, characterized in that, Including: Display the target image and the annotation information of the target image in an interaction interface, where the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of the at least one key point in the target image; In response to an operation instruction detected in the interaction interface, a target heat map corresponding to the target image is displayed in the interaction interface, where the target heat map is generated by the target image, the annotation information, and a target Gaussian kernel, the target Gaussian kernel corresponds to the scale information of the target object, the scale information is determined by the positions of the at least one key point, the target Gaussian kernel includes a sub-Gaussian kernel corresponding to each key point, and the sub-Gaussian kernel corresponding to each key point is obtained according to the product of the weight coefficient corresponding to each key point and the scale information, and the weight coefficient is determined based on the type of each key point.

8. A method for generating a heat map, characterized in that, It includes: The cloud server receives the original data uploaded by the client, where the original data includes: a target image and the annotation information of the target image, the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of the at least one key point in the target image; The cloud server determines the scale information of the target object based on the positions of the at least one key point; The cloud server determines the weight coefficient corresponding to each key point based on the type of each key point; The cloud server obtains the product of the weight coefficient and the scale information to obtain the sub-Gaussian kernel corresponding to each key point; The cloud server generates a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel, where the target Gaussian kernel includes a sub-Gaussian kernel corresponding to each key point.

9. A heat map generation device, characterized in that, It includes: A first acquisition module, configured to acquire a target image and the annotation information of the target image, where the target image includes a target object, the target object includes at least one key point, and the annotation information includes the positions of the at least one key point in the target image; A determination module, configured to determine the scale information of the target object based on the positions of the at least one key point; A second acquisition module, configured to determine the weight coefficient corresponding to each key point based on the type of each key point, obtain the product of the weight coefficient and the scale information, and obtain the sub-Gaussian kernel corresponding to each key point; A generation module, configured to generate a target heat map corresponding to the target image based on the target image, the annotation information, and the target Gaussian kernel, where the target Gaussian kernel includes a sub-Gaussian kernel corresponding to each key point.

Images