Method, device, storage medium and processor for determining key points of target object

Abstract

This invention discloses a method, apparatus, storage medium, and processor for determining key points of a target object. The method includes: acquiring an original image showing the target object to be identified; sampling the original image using multiple feature layers of a network model on a target device to obtain a feature map of the target object, wherein the feature map includes at least one initial key point of the target object, and the target device is a model with performance configuration below standard; performing point dithering post-processing on the at least one initial key point in the feature map, and outputting the target key points of the target object. This invention solves the technical problem of low speed and efficiency in determining human body key points using low-configuration mobile terminals in related technologies.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and more specifically, to a method, apparatus, storage medium, and processor for determining key points of a target object. Background Technology

[0002] Currently, the demand for determining key human body points is increasing in AI-powered fitness, body shaping, and event detection applications. The current technology for determining key human body points involves processing the original image on a high-performance mobile device. However, the low speed and efficiency of current low-configuration mobile devices in determining key human body points make it difficult to extract this information from the original image in real time.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a method, apparatus, storage medium, and processor for determining key points of a target object, to at least solve the technical problem of low speed and efficiency in determining key points of the human body using low-configuration mobile terminals in related technologies.

[0005] According to one aspect of the present invention, a method for determining key points of a target object is provided, comprising: acquiring an original image, wherein the original image shows a target object to be identified; sampling the original image using multiple feature layers of a network model on a target device to obtain a feature map of the target object, wherein the feature map includes at least one initial key point of the target object, and the target device is a model with performance configuration lower than the standard configuration; performing point dithering post-processing on the at least one initial key point in the feature map, and outputting the target key points of the target object.

[0006] Further, the plurality of feature layers includes a first type of feature layer and a second type of feature layer. Sampling the original image using the plurality of feature layers of the network model on the target device to obtain the feature map of the target object includes: downsampling the original image through the first type of feature layer in the network model to obtain a first feature map, wherein the first type of feature layer includes a plurality of feature layers with gradually decreasing length and width, and the length and width of the first feature map are smaller than the length and width of the original image; upsampling the first feature map through the second type of feature layer in the network model to obtain a second feature map, wherein the second type of feature layer includes a plurality of feature layers with gradually increasing length and width, and the length and width of the second feature map are larger than the length and width of the first feature map; and using the second feature map as the feature map of the target object.

[0007] Furthermore, the input of each feature layer in the second type of feature layer includes at least one of the following: a feature map in the target feature map set, the output of the previous feature layer in the second type of feature layer, wherein the target feature map set is the set of feature maps output by each feature layer in the first type of feature layer.

[0008] Furthermore, if the input to the target feature layer in the second type of feature layer is a feature map from the target feature map set and the output of the previous feature layer in the second type of feature layer, the method further includes: performing a convolution operation on a feature map from the target feature map set to obtain a result; fusing the result with the output of the previous feature layer in the current feature layer to obtain a fusion result; and upsampling the fusion result in the current feature layer in the second type of feature layer to obtain the feature map output by the current feature layer.

[0009] Furthermore, the post-processing of point dithering on the at least one initial key point in the feature map to output the target key point of the target object includes: determining multiple neighboring points in the adjacent domain based on the position of the initial key point; determining the target neighboring point from the multiple neighboring points; and adjusting the position of the initial key point based on the target neighboring point to obtain the target key point of the target object.

[0010] Further, determining the target neighboring point from the plurality of neighboring points includes: sorting the plurality of neighboring points according to their pixel values, selecting a first neighboring point and a second neighboring point, wherein the pixel value of the first neighboring point is greater than the pixel value of any other neighboring point among the plurality of neighboring points, and the pixel value of the second neighboring point is less than the pixel value of the first neighboring point, but greater than the pixel value of any other neighboring point among the plurality of neighboring points except for the first neighboring point; determining the midpoint between the first neighboring point and the second neighboring point; and taking the midpoint between the first neighboring point and the second neighboring point as the target neighboring point.

[0011] Further, adjusting the position of the initial keypoint based on the target neighboring points to obtain the target keypoint of the target object includes: determining the distance value between the initial keypoint and the target neighboring points; determining the magnification factor of the second feature map converted into the original image; determining the moving distance of the initial keypoint based on the distance value and the magnification factor; and moving the initial keypoint by the moving distance to obtain the target keypoint of the target object.

[0012] According to one aspect of the present invention, a method for determining key points of a target object is also provided, comprising: a cloud server receiving an original image uploaded from a target device, wherein the original image displays a target object to be identified, and the target device is a model with performance configuration lower than the standard configuration; the cloud server performing sampling processing on the original image based on multiple feature layers of a network model to obtain a feature map of the target object, performing point dithering post-processing on at least one initial key point in the feature map, and outputting target key points of the target object, wherein the feature map includes at least one initial key point of the target object; and the cloud server returning the target key points of the target object to the target device.

[0013] According to one aspect of the present invention, a method for determining key points of a target object is also provided, comprising: acquiring an original image of the target object during a fitness class, wherein the original image shows the target object to be identified; sampling the original image using multiple feature layers of a network model on a target device to obtain a feature map of the target object, wherein the feature map includes at least one initial key point of the target object, and the target device is a model with performance configuration lower than the standard configuration; performing point dithering post-processing on the at least one initial key point in the feature map, and outputting the target key points of the target object.

[0014] According to another aspect of the present invention, an apparatus for determining key points of a target object is also provided, comprising: a first acquisition unit for acquiring an original image, wherein the original image shows a target object to be identified; a first processing unit for sampling the original image using multiple feature layers of a network model on a target device to obtain a feature map of the target object, wherein the feature map includes at least one initial key point of the target object, and the target device is a model with performance configuration lower than the standard configuration; and a second processing unit for performing post-processing of point dithering on the at least one initial key point in the feature map, and outputting the target key points of the target object.

[0015] Further, the plurality of feature layers includes a first type of feature layer and a second type of feature layer. The first processing unit includes: a first sampling module, used to downsample the original image through the first type of feature layer in the network model to obtain a first feature map, wherein the first type of feature layer includes a plurality of feature layers with gradually decreasing length and width, and the length and width of the first feature map are smaller than the length and width of the original image; a second sampling module, used to upsample the first feature map through the second type of feature layer in the network model to obtain a second feature map, wherein the second type of feature layer includes a plurality of feature layers with gradually increasing length and width, and the length and width of the second feature map are larger than the length and width of the first feature map; and a first determining module, used to use the second feature map as the feature map of the target object.

[0016] Furthermore, the input of each feature layer in the second type of feature layer includes at least one of the following: a feature map in the target feature map set, the output of the previous feature layer in the second type of feature layer, wherein the target feature map set is the set of feature maps output by each feature layer in the first type of feature layer.

[0017] Furthermore, if the input to the target feature layer in the second type of feature layer is a feature map from the target feature map set and the output of the previous feature layer in the second type of feature layer, the device further includes: a first calculation unit, configured to perform a convolution operation on a feature map from the target feature map set to obtain a calculation result; a first merging unit, configured to fuse the calculation result with the output of the previous feature layer in the current feature layer to obtain a fusion result; and a third processing unit, configured to perform upsampling processing on the fusion result in the current feature layer in the second type of feature layer to obtain the feature map output by the current feature layer.

[0018] Further, the second processing unit includes: a second determining module, used to determine a plurality of neighboring points in the adjacent area based on the position of the initial key point; a third determining module, used to determine a target neighboring point from the plurality of neighboring points; and a first adjusting module, used to adjust the position of the initial key point based on the target neighboring point to obtain the target key point of the target object.

[0019] Further, the third determining module includes: a first processing submodule, used to sort the plurality of adjacent points according to their pixel values, and select a first adjacent point and a second adjacent point, wherein the pixel value of the first adjacent point is greater than the pixel value of any other adjacent point among the plurality of adjacent points, and the pixel value of the second adjacent point is less than the pixel value of the first adjacent point, but greater than the pixel value of any other adjacent point among the plurality of adjacent points except for the first adjacent point; a first determining submodule, used to determine the midpoint between the first adjacent point and the second adjacent point; and a second determining submodule, used to take the midpoint between the first adjacent point and the second adjacent point as the target adjacent point.

[0020] Further, the first adjustment module includes: a third determining submodule, used to determine the distance value between the initial key point and the target adjacent point; a fourth determining submodule, used to determine the magnification factor of the second feature map converted into the original image; a fifth determining submodule, used to determine the moving distance of the initial key point based on the distance value and the magnification factor; and a first obtaining submodule, used to move the initial key point by the moving distance to obtain the target key point of the target object.

[0021] According to another aspect of the present invention, an apparatus for determining key points of a target object is also provided, comprising: a first receiving unit, configured to receive an original image uploaded from a target device by a cloud server, wherein the original image shows the target object to be identified, and the target device is a model with performance configuration lower than the standard configuration; a fourth processing unit, configured to have the cloud server perform sampling processing on the original image based on multiple feature layers of a network model to obtain a feature map of the target object, perform post-processing of point dithering on at least one initial key point in the feature map, and output the target key points of the target object, wherein the feature map includes at least one initial key point of the target object; and a first returning unit, configured to have the cloud server return the target key points of the target object to the target device.

[0022] According to another aspect of the present invention, a storage medium is also provided, wherein the storage medium includes a stored program, wherein the program, when running, controls the device where the storage medium is located to execute the method described in any of the above embodiments.

[0023] According to another aspect of the present invention, a processor is also provided, wherein the processor is configured to run a program, wherein the program executes the method described in any of the preceding embodiments.

[0024] In this embodiment of the invention, multiple feature layers of the network model on the target device are used to sample the original image, followed by post-processing point dithering optimization. Specifically, an original image is acquired, showing the target object to be identified; multiple feature layers of the network model on the target device are used to sample the original image to obtain a feature map of the target object, wherein the feature map includes at least one initial key point of the target object, and the target device is a model with performance configuration lower than standard configuration; point dithering post-processing is performed on the at least one initial key point in the feature map to output the target key point of the target object, thereby improving the speed and efficiency of determining the target key point. This achieves the technical effect of quickly determining human body key points even with a low-configuration mobile terminal (corresponding to the aforementioned target device), thus solving the technical problem of low speed and efficiency in determining human body key points using low-configuration mobile terminals in related technologies. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0026] Figure 1 This is a hardware structure block diagram of a computer terminal according to an embodiment of the present invention;

[0027] Figure 2 This is a flowchart of a method for determining key points of a target object according to Embodiment 1 of the present invention;

[0028] Figure 3 This is a schematic diagram of the network model in the method for determining key points of a target object according to Embodiment 1 of the present invention;

[0029] Figure 4 This is a schematic diagram of key points in the method for determining key points of a target object according to Embodiment 1 of the present invention;

[0030] Figure 5 A flowchart of a method for determining key points of a target object according to Embodiment 2 of the present invention;

[0031] Figure 6 A flowchart of a method for determining key points of a target object according to Embodiment 3 of the present invention;

[0032] Figure 7 A schematic diagram of a device for determining key points of a target object according to Embodiment 4 of the present invention;

[0033] Figure 8A schematic diagram of a device for determining key points of a target object according to Embodiment 5 of the present invention;

[0034] Figure 9 This is a structural block diagram of an optional computer terminal provided according to Embodiment Six of the present invention. Detailed Implementation

[0035] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0037] Example 1

[0038] According to an embodiment of the present invention, a method embodiment for determining key points of a target object is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0039] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a method to determine key points of a target object is shown. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0040] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0041] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for determining key points of a target object in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the method for determining key points of a target object in the above-mentioned application. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0042] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0043] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0044] Under the aforementioned operating environment, this application provides the following: Figure 2 The method shown is for determining the key points of a target object. Figure 2 This is a flowchart of a method for determining key points of a target object according to Embodiment 1 of the present invention.

[0045] Step S201: Obtain the original image, wherein the original image shows the target object to be identified.

[0046] Step S202: The original image is sampled and processed using multiple feature layers of the network model on the target device to obtain a feature map of the target object. The feature map includes at least one initial key point of the target object. The target device is a model with performance configuration lower than the standard configuration.

[0047] The aforementioned models with performance configurations lower than the standard configuration can also be understood as Android phones on the market priced below a certain level, such as the OPPO R9M, with a CPU frequency of 1.9GHz and a CPU model of MediaTek MT6755.

[0048] Step S203: Perform post-processing of point dithering on at least one initial key point in the feature map, and output the target key points of the target object.

[0049] The target object mentioned above can be a person, and the target key points are the key points of the human body. The key points of the human body mainly refer to the coordinate points representing the joints of the human skeleton, such as the knee point, the arm and wrist point, etc.

[0050] By employing the above method, multiple feature layers of the network model on the target device are used to sample and process the original image. Then, post-processing point jitter optimization is performed, which improves the speed and efficiency of determining target key points. This achieves the technical effect of quickly determining human key points even with a low-configuration mobile terminal (corresponding to the target device mentioned above), thereby solving the technical problem of low speed and efficiency in determining human key points using low-configuration mobile terminals in related technologies.

[0051] To improve the accuracy of sampling and processing of the original image by multiple feature layers of the network model on the target device, optionally, in the method for determining key points of the target object provided in this application, the multiple feature layers include a first type of feature layer and a second type of feature layer. Sampling and processing the original image using the multiple feature layers of the network model on the target device to obtain a feature map of the target object includes: downsampling the original image through the first type of feature layer in the network model to obtain a first feature map, wherein the first type of feature layer includes multiple feature layers with gradually decreasing length and width, and the length and width of the first feature map are smaller than the length and width of the original image; upsampling the first feature map through the second type of feature layer in the network model to obtain a second feature map, wherein the second type of feature layer includes multiple feature layers with gradually increasing length and width, and the length and width of the second feature map are larger than the length and width of the first feature map; and using the second feature map as the feature map of the target object.

[0052] For example, such as Figure 3 As shown, the network model on the target device employs a downsampling followed by upsampling operation. Specifically, the first type of feature layer can be considered as a feature layer that gradually transforms a 3-channel image with dimensions of 128 into a 64-channel image with dimensions of 8 (corresponding to...). Figure 3 The first five layers from left to right in the image), the second type of feature layer can be considered as the feature layer that gradually transforms a 32-channel image with dimensions of 8 x 15 x 15 into a 32-channel image with dimensions of 15 x 15. Figure 3 (Floors 6 through 9 from left to right).

[0053] In addition, the input of each feature layer in the second type of feature layer includes at least one of the following: a feature map in the target feature map set, the output of the previous feature layer in the second type of feature layer, wherein the target feature map set is the set of feature maps output by each feature layer in the first type of feature layer.

[0054] For example, such as Figure 3 As shown, the 9th feature layer is the input of the output of the previous feature layer, and the 7th feature layer is the input of the features of the previous feature layer and the features of the output of the 4th feature layer.

[0055] Optionally, in the method for determining key points of a target object provided in this application, if the input of the target feature layer in the second type of feature layer is a feature map in the target feature map set and the output of the previous feature layer of the current feature layer in the second type of feature layer, the method further includes: performing a convolution operation on a feature map in the target feature map set to obtain a calculation result; fusing the calculation result with the output of the previous feature layer of the current feature layer to obtain a fusion result; and upsampling the fusion result in the current feature layer of the second type of feature layer to obtain a feature map output by the current feature layer.

[0056] For example, such as Figure 3 As shown, a convolution operation is performed on the feature map output by the 5th feature layer to obtain the result. The result is then fused with the output of the 5th feature layer to obtain the fused result. The fused result is then upsampled in the 6th feature layer to obtain the feature map output by the 6th feature layer.

[0057] In other words, the idea behind the above scheme is to fuse the downsampling and upsampling outputs. The deepest layer (e.g., Figure 3 During the fusion process (the 6th feature layer in the model), convolutional layers are added on the jumpers, and then the layers are added together; the next deeper layers (e.g., Figure 3 The 7th feature layer in the model is directly added and fused; the next shallower layer (e.g., Figure 3 The 8th feature layer in the model also performs a jumper convolution on the downsampled features, and then adds and fuses them with the output features. The purpose of this is to incorporate low-level features while preserving high-level features, resulting in better modeling not only of the overall human body but also of details such as limbs, before the final output. Thus, this approach can improve the accuracy of the original image sampling processing by multiple feature layers of the network model on the target device.

[0058] Optionally, in the method for determining key points of a target object provided in this application, post-processing of point dithering is performed on at least one initial key point in the feature map, and the target key points of the target object are output by: determining multiple neighboring points in the adjacent neighborhood based on the position of the initial key point; determining the target neighboring point from the multiple neighboring points; and adjusting the position of the initial key point based on the target neighboring point to obtain the target key points of the target object.

[0059] In the above scheme, after the network model outputs the feature map of the target object, it is necessary to take the coordinates of the globally maximum value of the feature map and use them as the coordinates of the corresponding key points. For example, taking a floating-point coordinate of the neighborhood around the maximum pixel value in the feature map as the initial key point can reduce magnification error. For example, as... Figure 4As shown, assuming the initial keypoint is point O, there are points A to H in its 8-neighborhood. The position of point O is adjusted based on points A to H to obtain point P (corresponding to the target keypoint of the target object mentioned above).

[0060] By performing post-processing such as position jitter on at least one initial key point in the feature map using the above method, the accuracy of the target key points of the determined target object can be guaranteed.

[0061] Optionally, in the method for determining key points of a target object provided in this application, determining the target neighboring point from multiple neighboring points includes: sorting the multiple neighboring points according to their pixel values, selecting a first neighboring point and a second neighboring point, wherein the pixel value of the first neighboring point is greater than the pixel value of any other neighboring point among the multiple neighboring points, and the pixel value of the second neighboring point is less than the pixel value of the first neighboring point but greater than the pixel value of any other neighboring point among the multiple neighboring points except for the first neighboring point; determining the midpoint between the first neighboring point and the second neighboring point; and taking the midpoint between the first neighboring point and the second neighboring point as the target neighboring point.

[0062] For example, for Figure 4 Sort the pixel values ​​of the 7 points A to H, find the first and second points, such as points A and D, and then find the midpoint Q between points A and D. Use the midpoint Q as the target neighbor point.

[0063] Optionally, in the method for determining key points of a target object provided in this application, adjusting the position of the initial key point based on the target's neighboring points to obtain the target key points of the target object includes: determining the distance value between the initial key point and the target's neighboring points; determining the magnification factor of the second feature map converted into the original image; determining the moving distance of the initial key point based on the distance value and the magnification factor; and moving the initial key point by the moving distance to obtain the target key points of the target object.

[0064] For example, to determine the distance from point O to the midpoint Q, the coordinates of point O are shifted towards Q by the reciprocal of the magnification factor multiplied by the distance from point O to point Q. For instance, if the magnification factor of the original image is four times, then point O is shifted towards point Q by 0.25 times the distance from point O to point Q. The resulting point is the final point P (corresponding to the target key point of the target object mentioned above).

[0065] The above scheme employs a network model structure that is more compatible with low-configuration devices. The model structure is compact yet fully utilizes both shallow and deep layer features, achieving a frame processing time of only 20ms, which is remarkably short. Furthermore, 8-neighbor interpolation is applied to the keypoints predicted in the model, significantly reducing magnification errors and resulting in a more stable visual perception. This achieves the technical effect of quickly determining human body keypoints even on low-configuration mobile terminals (corresponding to the target device mentioned above), thus solving the technical problem of low speed and efficiency in determining human body keypoints using low-configuration mobile terminals in related technologies.

[0066] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0067] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0068] Example 2

[0069] Under the operating environment of the above-described embodiment one, the present invention also provides, as follows: Figure 5 The method shown is for determining the key points of a target object. Figure 5 This is a flowchart of a method for determining key points of a target object according to Embodiment 2 of the present invention.

[0070] Step 501: The cloud server receives the original image uploaded from the target device, where the original image shows the target object to be identified, and the target device is a model with performance configuration lower than the standard configuration.

[0071] Step 502: The cloud server samples the original image based on multiple feature layers of the network model to obtain the feature map of the target object. It performs post-processing of point dithering on at least one initial key point in the feature map and outputs the target key point of the target object. The feature map includes at least one initial key point of the target object.

[0072] Step 503: The cloud server returns the target key points of the target object to the target device.

[0073] Through the above steps, the original image is sampled and processed using multiple feature layers of the network model on the cloud server, and then post-processed to optimize point jitter. This achieves the goal of improving the speed and efficiency of determining target key points, thereby realizing the technical effect of quickly returning the identified target key points to a low-configuration mobile terminal (corresponding to the target device mentioned above). This solves the technical problem of low speed and efficiency in determining human body key points using low-configuration mobile terminals in related technologies.

[0074] It should be noted that the optional technical solutions of the method for determining the key points of the target object provided in Embodiment 1 can all be implemented in the cloud server in Embodiment 2, and will not be described again here.

[0075] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0076] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0077] Example 3

[0078] Under the operating environment of the above-described embodiment one, the present invention also provides, as follows: Figure 6 The method shown is for determining the key points of a target object. Figure 6This is a flowchart of a method for determining key points of a target object according to Embodiment 3 of the present invention.

[0079] Step S601: During the process of the target object participating in the fitness class, the original image is acquired, wherein the original image shows the target object to be identified;

[0080] Step S602: The original image is sampled using multiple feature layers of the network model on the target device to obtain a feature map of the target object. The feature map includes at least one initial key point of the target object. The target device is a model with performance configuration lower than the standard configuration.

[0081] Step S603: Perform post-processing of point dithering on at least one initial key point in the feature map, and output the target key points of the target object.

[0082] Through the above steps, during the target object's participation in the fitness course, multiple feature layers of the network model are used to sample and process the original image. Then, the post-processing point jitter optimization method is used to improve the speed and efficiency of determining target key points. This achieves the technical effect of quickly returning the identified target key points to a low-configuration mobile terminal (corresponding to the target device mentioned above), thereby solving the technical problem of low speed and efficiency in determining human body key points using low-configuration mobile terminals in related technologies.

[0083] Optionally, in the method for determining key points of a target object provided in Embodiment 3 of the present invention, after identifying the target key points of the target object, the exercise status of the target object during the fitness course can be determined. For example, the exercise status includes poor exercise effect, non-standard exercise movements, etc. If the coordinates of the target key points of the target object differ from the preset coordinates by more than a preset distance value, a reminder can be triggered for the target object. If the exercise movements differ significantly from the standard movements, the target object can be reminded to adjust its exercise posture, etc.

[0084] In other words, the above solution can identify key points of the target object on a low-configuration mobile phone and remind the user to adjust their exercise posture, thereby improving the user's exercise experience.

[0085] It should be noted that the optional technical solutions of the method for determining the key points of the target object provided in Embodiment 1 can all be implemented in the application scenario of the fitness course in Embodiment 3, and will not be described again here.

[0086] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0087] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0088] Example 4

[0089] According to embodiments of the present invention, an apparatus is also provided for a method of implementing key points of a target object as described in Embodiment 1 above, such as... Figure 7 As shown, the device includes:

[0090] The first acquisition unit 701 is used to acquire an original image, wherein the original image shows the target object to be identified;

[0091] The first processing unit 702 is used to sample and process the original image using multiple feature layers of the network model on the target device to obtain a feature map of the target object. The feature map includes at least one initial key point of the target object. The target device is a model with a performance configuration lower than the standard configuration.

[0092] The second processing unit 703 is used to perform post-processing of point dithering on at least one initial key point in the feature map and output the target key point of the target object.

[0093] Optionally, in the apparatus for determining key points of a target object provided in Embodiment 3 of the present invention, the plurality of feature layers include a first type of feature layer and a second type of feature layer. The first processing unit 702 includes: a first sampling module, used to downsample the original image through the first type of feature layer in the network model to obtain a first feature map, wherein the first type of feature layer includes a plurality of feature layers with gradually decreasing length and width, and the length and width of the first feature map are smaller than the length and width of the original image; a second sampling module, used to upsample the first feature map through the second type of feature layer in the network model to obtain a second feature map, wherein the second type of feature layer includes a plurality of feature layers with gradually increasing length and width, and the length and width of the second feature map are larger than the length and width of the first feature map; and a first determining module, used to use the second feature map as the feature map of the target object.

[0094] Optionally, in the apparatus for determining key points of a target object provided in Embodiment 3 of the present invention, the input of each feature layer in the second type of feature layer includes at least one of the following: a feature map in the target feature map set, the output of the previous feature layer in the current feature layer of the second type of feature layer, wherein the target feature map set is a set of feature maps output by each feature layer in the first type of feature layer.

[0095] Optionally, in the apparatus for determining key points of a target object provided in Embodiment 3 of the present invention, if the input of the target feature layer in the second type of feature layer is a feature map in the target feature map set and the output of the previous feature layer of the current feature layer in the second type of feature layer, the apparatus further includes: a first calculation unit, used to perform a convolution operation on a feature map in the target feature map set to obtain a calculation result; a first merging unit, used to fuse the calculation result with the output of the previous feature layer of the current feature layer to obtain a fusion result; and a third processing unit, used to perform upsampling processing on the fusion result in the current feature layer of the second type of feature layer to obtain a feature map output by the current feature layer.

[0096] Optionally, in the apparatus for determining key points of a target object provided in Embodiment 3 of the present invention, the second processing unit 703 includes: a second determining module, used to determine a plurality of adjacent points in an adjacent field based on the position of an initial key point; a third determining module, used to determine a target adjacent point from the plurality of adjacent points; and a first adjusting module, used to adjust the position of the initial key point based on the target adjacent point to obtain the target key point of the target object.

[0097] Optionally, in the apparatus for determining key points of a target object provided in Embodiment 3 of the present invention, the third determining module includes: a first processing submodule, used to sort multiple adjacent points according to pixel value, select a first adjacent point and a second adjacent point, wherein the pixel value of the first adjacent point is greater than the pixel value of any other adjacent point among the multiple adjacent points, and the pixel value of the second adjacent point is less than the pixel value of the first adjacent point, but greater than the pixel value of any other adjacent point among the multiple adjacent points except the first adjacent point; a first determining submodule, used to determine the midpoint between the first adjacent point and the second adjacent point; and a second determining submodule, used to take the midpoint between the first adjacent point and the second adjacent point as the target adjacent point.

[0098] Optionally, in the apparatus for determining key points of a target object provided in Embodiment 3 of the present invention, the first adjustment module includes: a third determining submodule, used to determine the distance value between the initial key point and the target adjacent point; a fourth determining submodule, used to determine the magnification factor of the second feature map converted into the original image; a fifth determining submodule, used to determine the moving distance of the initial key point based on the distance value and the magnification factor; and a first obtaining submodule, used to move the initial key point by the moving distance to obtain the target key point of the target object.

[0099] It should be noted that the first acquisition unit 701, the first processing unit 702, and the second processing unit 703 mentioned above correspond to steps S201 to S203 in Embodiment 1. The two modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0100] Example 5

[0101] According to embodiments of the present invention, an apparatus is also provided for implementing a method for key points of a target object as described in Embodiment 2 above, such as... Figure 8 As shown, the device includes:

[0102] The first receiving unit 801 is used to receive raw images uploaded from the target device by the cloud server. The raw images show the target object to be identified, and the target device is a model with performance configuration lower than the standard configuration.

[0103] The fourth processing unit 802 is used by the cloud server to sample and process the original image based on multiple feature layers of the network model to obtain the feature map of the target object, perform post-processing of point dithering on at least one initial key point in the feature map, and output the target key point of the target object, wherein the feature map includes at least one initial key point of the target object.

[0104] The first return unit 803 is used by the cloud server to return the target key points of the target object to the target device.

[0105] It should be noted that the first receiving unit 801, the fourth processing unit 802, and the first returning unit 803 mentioned above correspond to steps S501 to S503 in Embodiment 2. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0106] Example 6

[0107] Embodiments of the present invention can provide a computer terminal, which can be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the computer terminal can also be replaced by a mobile terminal or other terminal device.

[0108] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0109] In this embodiment, the computer terminal described above can execute the following steps of the method for determining key points of a target object in an application: acquiring an original image, wherein the original image shows the target object to be identified; sampling the original image using multiple feature layers of a network model on the target device to obtain a feature map of the target object, wherein the feature map includes at least one initial key point of the target object, and the target device is a model with performance configuration lower than the standard configuration; performing point dithering post-processing on the at least one initial key point in the feature map, and outputting the target key points of the target object.

[0110] The aforementioned computer terminal can also execute program code for the following steps in the method for determining key points of a target object in an application: downsampling the original image through a first type of feature layer in the network model to obtain a first feature map, wherein the first type of feature layer includes multiple feature layers with gradually decreasing length and width, and the length and width of the first feature map are smaller than the length and width of the original image; upsampling the first feature map through a second type of feature layer in the network model to obtain a second feature map, wherein the second type of feature layer includes multiple feature layers with gradually increasing length and width, and the length and width of the second feature map are larger than the length and width of the first feature map; and using the second feature map as the feature map of the target object.

[0111] The aforementioned computer terminal can also execute program code for the following steps in the method for determining key points of a target object in an application: the input of each feature layer in the second type of feature layer includes at least one of the following: a feature map in the target feature map set, the output of the previous feature layer in the second type of feature layer, wherein the target feature map set is the set of feature maps output by each feature layer in the first type of feature layer.

[0112] The aforementioned computer terminal can also execute program code for the following steps in the method for determining key points of a target object in an application: if the input of the target feature layer in the second type of feature layer is a feature map in the target feature map set and the output of the previous feature layer in the current feature layer of the second type of feature layer, perform a convolution operation on a feature map in the target feature map set to obtain a calculation result; fuse the calculation result with the output of the previous feature layer in the current feature layer to obtain a fusion result; and perform upsampling processing on the fusion result in the current feature layer in the second type of feature layer to obtain the feature map output by the current feature layer.

[0113] The aforementioned computer terminal can also execute program code for the following steps in the method for determining key points of a target object in an application: determining multiple neighboring points in the adjacent field based on the position of the initial key point; determining a target neighboring point from the multiple neighboring points; adjusting the position of the initial key point based on the target neighboring point to obtain the target key point of the target object.

[0114] The aforementioned computer terminal can also execute program code for the following steps in the method for determining key points of a target object in an application: sorting the plurality of adjacent points according to their pixel values, selecting a first adjacent point and a second adjacent point, wherein the pixel value of the first adjacent point is greater than the pixel value of any other adjacent point among the plurality of adjacent points, and the pixel value of the second adjacent point is less than the pixel value of the first adjacent point but greater than the pixel value of any other adjacent point among the plurality of adjacent points except for the first adjacent point; determining the midpoint between the first adjacent point and the second adjacent point; and taking the midpoint between the first adjacent point and the second adjacent point as the target adjacent point.

[0115] The aforementioned computer terminal can also execute program code for the following steps in the method for determining key points of a target object in an application: determining the distance value between the initial key point and the adjacent point of the target; determining the magnification factor of converting the second feature map into the original image; determining the moving distance of the initial key point based on the distance value and the magnification factor; moving the initial key point by the moving distance to obtain the target key point of the target object.

[0116] The aforementioned computer terminal can also execute program code for the following steps in the method for determining key points of a target object in an application: The cloud server receives an original image uploaded from a target device, wherein the original image displays the target object to be identified, and the target device is a model with performance configuration lower than the standard configuration; the cloud server samples the original image based on multiple feature layers of a network model to obtain a feature map of the target object, performs post-processing of point dithering on at least one initial key point in the feature map, and outputs the target key points of the target object, wherein the feature map includes at least one initial key point of the target object; the cloud server returns the target key points of the target object to the target device.

[0117] The aforementioned computer terminal can also execute program code for the following steps in the method for determining key points of a target object in an application: acquiring an original image during the target object's participation in a fitness class, wherein the original image shows the target object to be identified; sampling the original image using multiple feature layers of a network model on the target device to obtain a feature map of the target object, wherein the feature map includes at least one initial key point of the target object, and the target device is a model with performance configuration lower than the standard configuration; performing post-processing of point dithering on the at least one initial key point in the feature map, and outputting the target key points of the target object.

[0118] Optionally, Figure 9 This is a structural block diagram of a computer terminal according to an embodiment of the present invention. Figure 9 As shown, the computer terminal may include: one or more ( Figure 9 (Only one is shown in the image) processor and memory.

[0119] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and apparatus for determining key points of a target object in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned method for determining key points of the target object. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0120] The processor can access information and applications stored in memory via a transmission device to perform the following steps: acquiring an original image, wherein the original image shows a target object to be identified; sampling the original image using multiple feature layers of a network model on the target device to obtain a feature map of the target object, wherein the feature map includes at least one initial key point of the target object, and the target device is a model with performance configuration lower than the standard configuration; performing point dithering post-processing on the at least one initial key point in the feature map, and outputting the target key points of the target object.

[0121] Optionally, the processor may also execute program code for the following steps: downsampling the original image through a first type of feature layer in the network model to obtain a first feature map, wherein the first type of feature layer includes multiple feature layers with gradually decreasing length and width, and the length and width of the first feature map are smaller than the length and width of the original image; upsampling the first feature map through a second type of feature layer in the network model to obtain a second feature map, wherein the second type of feature layer includes multiple feature layers with gradually increasing length and width, and the length and width of the second feature map are larger than the length and width of the first feature map; and using the second feature map as the feature map of the target object.

[0122] Optionally, the processor may also execute program code that includes the following steps: the input of each feature layer in the second type of feature layer includes at least one of the following: a feature map in the target feature map set, the output of the previous feature layer in the second type of feature layer, wherein the target feature map set is the set of feature maps output by each feature layer in the first type of feature layer.

[0123] Optionally, the processor may also execute program code for the following steps: if the input to the target feature layer in the second type of feature layer is a feature map in the target feature map set and the output of the previous feature layer in the current feature layer of the second type of feature layer, perform a convolution operation on a feature map in the target feature map set to obtain a result; fuse the result with the output of the previous feature layer in the current feature layer to obtain a fusion result; and upsample the fusion result in the current feature layer of the second type of feature layer to obtain the feature map output by the current feature layer.

[0124] Optionally, the processor may also execute program code that performs the following steps: determining multiple neighboring points in the adjacent domain based on the position of the initial key point; determining a target neighboring point from the multiple neighboring points; and adjusting the position of the initial key point based on the target neighboring point to obtain the target key point of the target object.

[0125] Optionally, the processor may also execute program code for the following steps: sorting the plurality of adjacent points according to their pixel values; selecting a first adjacent point and a second adjacent point, wherein the pixel value of the first adjacent point is greater than the pixel value of any other adjacent point among the plurality of adjacent points, and the pixel value of the second adjacent point is less than the pixel value of the first adjacent point but greater than the pixel value of any other adjacent point among the plurality of adjacent points except for the first adjacent point; determining the midpoint between the first adjacent point and the second adjacent point; and taking the midpoint between the first adjacent point and the second adjacent point as the target adjacent point.

[0126] Optionally, the processor may also execute program code that performs the following steps: determining the distance between the initial key point and the adjacent point of the target; determining the magnification factor of the second feature map to be converted into the original image; determining the moving distance of the initial key point based on the distance value and the magnification factor; and moving the initial key point by the moving distance to obtain the target key point of the target object.

[0127] Optionally, the processor may also execute program code for the following steps: the cloud server receives an original image uploaded from a target device, wherein the original image shows a target object to be identified, and the target device is a model with performance configuration lower than the standard configuration; the cloud server samples the original image based on multiple feature layers of a network model to obtain a feature map of the target object, performs post-processing of point dithering on at least one initial key point in the feature map, and outputs the target key points of the target object, wherein the feature map includes at least one initial key point of the target object; the cloud server returns the target key points of the target object to the target device.

[0128] Optionally, the processor may also execute program code for the following steps: acquiring an original image during the target object's participation in a fitness class, wherein the original image shows the target object to be identified; sampling the original image using multiple feature layers of a network model on the target device to obtain a feature map of the target object, wherein the feature map includes at least one initial key point of the target object, and the target device is a model with performance configuration lower than the standard configuration; performing post-processing of point dithering on the at least one initial key point in the feature map, and outputting the target key points of the target object.

[0129] This invention provides a method for determining key points of a target object. Specifically, it involves sampling the original image using multiple feature layers of a network model on the target device, followed by post-processing with point dithering optimization. The method involves acquiring an original image showing the target object to be identified; sampling the original image using multiple feature layers of a network model on the target device to obtain a feature map of the target object, wherein the feature map includes at least one initial key point of the target object; and performing point dithering post-processing on the at least one initial key point in the feature map to output the target key points of the target object. This improves the speed and efficiency of determining target key points, enabling rapid determination of human body key points even with a low-configuration mobile terminal (corresponding to the aforementioned target device). This solves the technical problem of low speed and efficiency in determining human body key points using low-configuration mobile terminals in related technologies.

[0130] Those skilled in the art will understand that Figure 9 The structure shown is for illustrative purposes only. The computer terminal can also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a mobile internet device (MID), a PAD, and other terminal devices. Figure 9 This does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include components that are more advanced than those described above. Figure 9 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 9 The different configurations shown.

[0131] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0132] Example 7

[0133] Embodiments of the present invention also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the method for determining key points of a target object provided in Embodiment 1.

[0134] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0135] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring an original image, wherein the original image shows a target object to be identified; sampling the original image using multiple feature layers of a network model on the target device to obtain a feature map of the target object, wherein the feature map includes at least one initial key point of the target object, and the target device is a model with performance configuration lower than the standard configuration; performing point dithering post-processing on the at least one initial key point in the feature map, and outputting the target key points of the target object.

[0136] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: downsampling the original image through a first type of feature layer in the network model to obtain a first feature map, wherein the first type of feature layer includes multiple feature layers with gradually decreasing length and width, and the length and width of the first feature map are smaller than the length and width of the original image; upsampling the first feature map through a second type of feature layer in the network model to obtain a second feature map, wherein the second type of feature layer includes multiple feature layers with gradually increasing length and width, and the length and width of the second feature map are larger than the length and width of the first feature map; and using the second feature map as the feature map of the target object.

[0137] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: the input of each feature layer in the second type of feature layer includes at least one of the following: a feature map in the target feature map set, the output of the previous feature layer in the current feature layer of the second type of feature layer, wherein the target feature map set is a set of feature maps output by each feature layer in the first type of feature layer.

[0138] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: if the input of the target feature layer in the second type of feature layer is a feature map in the target feature map set and the output of the previous feature layer in the current feature layer of the second type of feature layer, perform a convolution operation on a feature map in the target feature map set to obtain a calculation result; fuse the calculation result with the output of the previous feature layer in the current feature layer to obtain a fusion result; and perform upsampling processing on the fusion result in the current feature layer in the second type of feature layer to obtain the feature map output by the current feature layer.

[0139] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: determining a plurality of neighboring points in the adjacent area based on the position of the initial key point; determining a target neighboring point from the plurality of neighboring points; and adjusting the position of the initial key point based on the target neighboring point to obtain the target key point of the target object.

[0140] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: sorting the plurality of adjacent points according to their pixel values, selecting a first adjacent point and a second adjacent point, wherein the pixel value of the first adjacent point is greater than the pixel value of any other adjacent point among the plurality of adjacent points, and the pixel value of the second adjacent point is less than the pixel value of the first adjacent point and greater than the pixel value of any other adjacent point among the plurality of adjacent points except for the first adjacent point; determining the midpoint between the first adjacent point and the second adjacent point; and taking the midpoint between the first adjacent point and the second adjacent point as the target adjacent point.

[0141] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: determining the distance value between the initial key point and the target adjacent point; determining the magnification factor of the second feature map to be converted into the original image; determining the moving distance of the initial key point based on the distance value and the magnification factor; and moving the initial key point by the moving distance to obtain the target key point of the target object.

[0142] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: the cloud server receives an original image uploaded from a target device, wherein the original image shows a target object to be identified, and the target device is a model with performance configuration lower than the standard configuration; the cloud server samples the original image based on multiple feature layers of a network model to obtain a feature map of the target object, performs post-processing of point dithering on at least one initial key point in the feature map, and outputs the target key point of the target object, wherein the feature map includes at least one initial key point of the target object; the cloud server returns the target key point of the target object to the target device.

[0143] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring an original image during the target object's participation in a fitness class, wherein the original image shows the target object to be identified; sampling the original image using multiple feature layers of a network model on the target device to obtain a feature map of the target object, wherein the feature map includes at least one initial key point of the target object, and the target device is a model with performance configuration lower than the standard configuration; performing post-processing of point dithering on the at least one initial key point in the feature map, and outputting the target key point of the target object.

[0144] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0145] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0146] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0147] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0148] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0149] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0150] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method of determining a key point of a target object, characterized by, include: Acquire the original image, wherein the original image shows the target object to be identified; The original image is sampled and processed using multiple feature layers of the network model on the target device to obtain a feature map of the target object. The feature map includes at least one initial key point of the target object. The target device is a model with performance configuration lower than the standard configuration. Post-processing of point dithering is performed on at least one initial key point in the feature map to output the target key point of the target object; The method of sampling the original image using multiple feature layers of the network model on the target device to obtain the feature map of the target object includes: sampling the original image through multiple feature layers with gradually decreasing length and width to obtain a first feature map; then sampling the first feature map through multiple feature layers with gradually increasing length and width to obtain a second feature map; and determining the feature map of the target object based on the second feature map.

2. The method of claim 1, wherein, The plurality of feature layers includes a first type of feature layer and a second type of feature layer. The first type of feature layer includes the plurality of feature layers with gradually decreasing length and width, and the second type of feature layer includes the plurality of feature layers with gradually increasing length and width. The original image is sampled through the plurality of feature layers with gradually decreasing length and width to obtain a first feature map. Then, the first feature map is sampled through the plurality of feature layers with gradually increasing length and width to obtain a second feature map. The feature map of the target object is determined based on the second feature map, including: The original image is downsampled through the first type of feature layer to obtain the first feature map, wherein the length and width of the first feature map are smaller than the length and width of the original image; The first feature map is upsampled through the second type of feature layer to obtain the second feature map, wherein the length and width of the second feature map are greater than the length and width of the first feature map; The second feature map is used as the feature map of the target object.

3. The method of claim 2, wherein, The input of each feature layer in the second type of feature layer includes at least one of the following: a feature map in the target feature map set, or the output of the previous feature layer in the second type of feature layer, wherein the target feature map set is the set of feature maps output by each feature layer in the first type of feature layer.

4. The method of claim 3, wherein, If the input to the target feature layer in the second type of feature layer is a feature map from the target feature map set and the output of the previous feature layer in the second type of feature layer, the method further includes: Perform a convolution operation on one of the feature maps in the target feature map set to obtain the operation result; The calculation result is fused with the output of the previous feature layer to obtain the fusion result; The fusion result is upsampled in the current feature layer of the second type of feature layer to obtain the feature map output by the current feature layer.

5. The method of claim 2, wherein, Post-processing involving point dithering of at least one initial keypoint in the feature map to output the target keypoints of the target object includes: Based on the location of the initial key point, determine multiple adjacent points in the adjacent field; Determine the target neighboring point from the plurality of neighboring points; The positions of the initial key points are adjusted based on the target's neighboring points to obtain the target key points of the target object.

6. The method of claim 5, wherein, Determining the target neighbor point from the plurality of neighbor points includes: The plurality of adjacent points are sorted according to their pixel values, and a first adjacent point and a second adjacent point are selected. The pixel value of the first adjacent point is greater than the pixel value of any other adjacent point among the plurality of adjacent points, and the pixel value of the second adjacent point is less than the pixel value of the first adjacent point, but greater than the pixel value of any other adjacent point among the plurality of adjacent points except for the first adjacent point. Determine the midpoint between the first adjacent point and the second adjacent point; The midpoint between the first adjacent point and the second adjacent point is taken as the target adjacent point.

7. The method of claim 5, wherein, Based on the target's neighboring points, the positions of the initial key points are adjusted to obtain the target key points of the target object, including: Determine the distance between the initial key point and the target's adjacent points; Determine the magnification factor by which the second feature map is converted into the original image; Based on the distance value and the magnification, determine the movement distance of the initial key point; The initial key point is moved by the moving distance to obtain the target key point of the target object.

8. A method of determining a key point of a target object, the method comprising: include: The cloud server receives raw images uploaded from the target device, wherein the raw images show the target object to be identified, and the target device is a model with performance configuration lower than the standard configuration; The cloud server samples the original image based on multiple feature layers of the network model to obtain a feature map of the target object. It then performs post-processing of point dithering on at least one initial key point in the feature map and outputs the target key point of the target object. The feature map includes at least one initial key point of the target object. The cloud server returns the target key points of the target object to the target device; The cloud server samples the original image based on multiple feature layers of the network model to obtain the feature map of the target object, including: sampling the original image through multiple feature layers with gradually decreasing length and width to obtain a first feature map; then sampling the first feature map through multiple feature layers with gradually increasing length and width to obtain a second feature map; and determining the feature map of the target object based on the second feature map.

9. A method of determining a key point of a target object, the method comprising: include: During the process of the target object participating in a fitness class, original images are acquired, wherein the original images show the target object to be identified; The original image is sampled and processed using multiple feature layers of the network model on the target device to obtain a feature map of the target object. The feature map includes at least one initial key point of the target object. The target device is a model with performance configuration lower than the standard configuration. Post-processing of point dithering is performed on at least one initial key point in the feature map to output the target key point of the target object; The method of sampling the original image using multiple feature layers of the network model on the target device to obtain the feature map of the target object includes: sampling the original image through multiple feature layers with gradually decreasing length and width to obtain a first feature map; then sampling the first feature map through multiple feature layers with gradually increasing length and width to obtain a second feature map; and determining the feature map of the target object based on the second feature map.

10. An apparatus for determining a key point of a target object, the apparatus comprising: include: The first acquisition unit is used to acquire an original image, wherein the original image shows the target object to be identified; The first processing unit is used to sample the original image using multiple feature layers of the network model on the target device to obtain a feature map of the target object, wherein the feature map includes at least one initial key point of the target object, and the target device is a model with performance configuration lower than the standard configuration; The second processing unit is used to perform post-processing of point dithering on at least one initial key point in the feature map and output the target key point of the target object. The first processing unit is further configured to sample the original image through multiple feature layers with gradually decreasing length and width to obtain a first feature map, then sample the first feature map through multiple feature layers with gradually increasing length and width to obtain a second feature map, and determine the feature map of the target object based on the second feature map.

11. An apparatus for determining a key point of a target object, the apparatus comprising: include: The first receiving unit is used for the cloud server to receive the original image uploaded from the target device, wherein the original image shows the target object to be identified, and the target device is a model with performance configuration lower than the standard configuration; The fourth processing unit is used by the cloud server to sample the original image based on multiple feature layers of the network model to obtain the feature map of the target object, perform post-processing of point dithering on at least one initial key point in the feature map, and output the target key point of the target object, wherein the feature map includes at least one initial key point of the target object. The first return unit is used by the cloud server to return the target key points of the target object to the target device; The fourth processing unit is further configured to sample the original image through multiple feature layers with gradually decreasing length and width to obtain a first feature map, then sample the first feature map through multiple feature layers with gradually increasing length and width to obtain a second feature map, and determine the feature map of the target object based on the second feature map.

12. A storage medium, characterized by The storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the storage medium to perform the method described in any one of claims 1 to 9.

Images