Data enhancement method, device, equipment and medium based on posture detection

Abstract

The present invention relates to the field of artificial intelligence technology, and in particular to a data enhancement method, device, equipment and medium based on posture detection. The data enhancement method based on posture detection includes obtaining a pre-labeled sample image; wherein the sample image includes a lying posture image; rotating and transforming the lying posture image into an upright human posture image to update the sample image; training a lightweight posture detection model based on the updated sample image to perform posture detection on a mobile terminal based on the trained posture detection model. The method implements data enhancement processing on the sample image, thereby reducing the learning complexity of the lightweight posture detection model, and can also achieve fast operation on a mobile terminal CPU with poor performance.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a data enhancement method, device, equipment and medium based on posture detection. Background Art

[0002] At present, the human posture detection model can detect the position of the key points of the human body, but there are still other factors that lead to low accuracy of current human posture detection in some cases, such as key point occlusion and detection of human side posture. Among them, for the detection of side posture, for example, for yoga movements, shooting at a side angle can more intuitively show the movements, and by increasing the volume of the detection model and increasing the learning ability of the model, better detection results can be achieved for the above postures that are more difficult to detect.

[0003] However, currently, large-volume models often need to run on GPUs with better performance, and running on CPUs with poor performance basically cannot meet real-time requirements. In order to quickly run the detection model on the CPU of a mobile terminal with poor performance, only a smaller model can be used. However, due to the small size of the small model, the learning ability is relatively poor. In various postures (such as lying, lying prone, front upright, sideways upright, etc.), the detection effect of the model cannot be guaranteed even on the training set. Summary of the invention

[0004] The embodiments of the present invention provide a data enhancement method, device, equipment and medium based on posture detection to solve the problem that the current posture detection effect of small-volume models is poor.

[0005] A data enhancement method based on posture detection, comprising:

[0006] Acquire a pre-labeled sample image; wherein the sample image includes a lying posture image;

[0007] Rotate and transform the lying posture image into an upright human posture image to update the sample image; wherein the upright human posture image is a human posture image in which the human posture is in an upright state;

[0008] A lightweight posture detection model is trained based on the updated sample images, so that posture detection is performed on the mobile terminal based on the trained posture detection model.

[0009] A data enhancement device based on posture detection, comprising:

[0010] An image acquisition module, used to acquire pre-labeled sample images; wherein the sample images include lying posture images;

[0011] A rotation transformation module, used for rotationally transforming the lying posture image into an upright human posture image to update the sample image; wherein the upright human posture image is a human posture image in which the human posture is in an upright state;

[0012] The posture detection module is used to train a lightweight posture detection model based on the updated sample image, so as to perform posture detection based on the trained posture detection model in the mobile terminal.

[0013] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned data enhancement method based on posture detection when executing the computer program.

[0014] A computer storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned data enhancement method based on posture detection are implemented.

[0015] In the above-mentioned data enhancement method, device, equipment and medium based on posture detection, the lying posture image in the original sample image is converted into an upright human posture image to achieve data enhancement processing of the sample image, thereby reducing the learning complexity of the lightweight posture detection model. Then, the lightweight posture detection model is trained based on the updated sample image, so that posture detection is performed on the mobile terminal based on the trained posture detection model, so that the lightweight posture detection model can run quickly on the mobile terminal CPU with poor performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0017] Figure 1 is a schematic diagram of an application environment of a data enhancement method based on posture detection in one embodiment of the present invention;

[0018] Figure 2 is a flow chart of a data enhancement method based on posture detection in one embodiment of the present invention;

[0019] Figure 3 is a flow chart of a data enhancement method based on posture detection in one embodiment of the present invention;

[0020] Figure 4 yes Figure 2 A specific flow chart of step S202;

[0021] Figure 5 is a flow chart of a data enhancement method based on posture detection in one embodiment of the present invention;

[0022] Figure 6 yes Figure 2 A specific flow chart of step S203;

[0023] Figure 7 is a schematic diagram of a data enhancement device based on posture detection in one embodiment of the present invention;

[0024] Figure 8 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] The data enhancement method based on posture detection can be applied in Figure 1 The application environment is one in which the computer device communicates with the server through the network. The computer device may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server may be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0027] In one embodiment, if Figure 2 As shown, a data enhancement method based on posture detection is provided, and the method is applied in Figure 1 The server in the example is used as an example to illustrate the process, including the following steps:

[0028] S201: Acquire a pre-labeled sample image, wherein the sample image includes a lying posture image;

[0029] The sample image is annotated with position information corresponding to the key points of the human body. The position information corresponding to the key points of the human body includes but is not limited to the two-dimensional coordinate information of the key points of the human body. The key points of the human body include but are not limited to the head, neck, left and right hands, left and right knees, left and right ankles, left and right elbows or left and right wrists and other skeletal key points. Lying posture images refer to human posture images taken sideways in a lying or side-lying state, such as plank support, push-ups, etc.

[0030] It is understandable that, since the trained posture detection model in this embodiment is applied to the mobile terminal, and the current large-volume model needs to be run on a GPU with better performance, and the operation on a CPU with poor performance basically cannot meet the real-time requirements, a smaller model (lightweight model) is used in this embodiment to ensure that the posture detection model can run quickly on a mobile terminal CPU with poor performance. However, since the current lightweight model is small in size, its detection ability for different postures is insufficient, so it is necessary to perform data enhancement on the sample image to update the sample image and reduce the complexity of model learning.

[0031] S202: Rotate and transform the lying posture image into an upright human posture image to update the sample image. The upright human posture image is a human posture image in which the human posture is in an upright state.

[0032] Among them, the upright human posture image is a human posture image in which the human body action is in an upright state. Specifically, the method can be applied to a mobile terminal for identifying human postures in videos or images, such as various yoga movements. Since yoga movements are generally displayed by sideways shooting for the convenience of display, but since the volume of the lightweight posture detection model trained subsequently is too small and the learning ability is poor, for some human body movements such as plank support, push-ups, etc., the lying posture images obtained by sideways shooting, the detection performance cannot be guaranteed when performing posture detection. Therefore, in this embodiment, in order to ensure the detection performance of the posture detection model for the lying posture image, the lying posture image can be rotated and transformed to the corresponding upright human posture image to update the sample image, and then the lightweight posture detection model is trained according to the updated sample image, so as to perform posture detection on the mobile terminal based on the trained posture detection model.

[0033] Specifically, since the lying posture image is marked with head position information, the rotation transformation parameters of the rotation transformation can be automatically determined through the head position information, so as to convert the lying posture image into an upright human body posture image according to the rotation transformation parameters.

[0034] Exemplarily, if the head is on the left side, it can be rotated 90 degrees clockwise or 270 degrees counterclockwise; if the head is on the right side, it can be rotated 90 degrees counterclockwise or 270 degrees clockwise to obtain the upright human body posture image.

[0035] S203: training a lightweight posture detection model based on the updated sample image, so as to perform posture detection on the mobile terminal based on the trained posture detection model.

[0036] Specifically, after updating the annotation information corresponding to the sample image, the sample image and the corresponding updated annotation information can be used as training samples to train a lightweight posture detection model. The lightweight posture detection model can be constructed based on a network including but not limited to MobilenNet.

[0037] In this embodiment, the lying posture image in the sample image is converted into an upright human posture image to achieve data enhancement processing of the sample image, thereby reducing the learning complexity of the lightweight posture detection model. Then, the lightweight posture detection model is trained based on the updated sample image, so that posture detection is performed on the mobile terminal based on the trained posture detection model, so that the lightweight posture detection model can run quickly on the mobile terminal CPU with poor performance.

[0038] In one embodiment, the sample image corresponds to multiple target key points such as Figure 3 As shown, the data enhancement method based on posture detection also includes the following steps:

[0039] S301: Use the trained 3D pose estimation model to predict the depth information corresponding to the target key points.

[0040] Specifically, for some images with poor lighting, the left and right distinction of the key points of the human body in the image is insufficient. For example, the left hand is easily misdetected as the right hand, and the right hand is misdetected as the left hand. Therefore, it is necessary to perform data enhancement on only the key points that are easily misdetected.

[0041] The target key points refer to the key points of the human body that are easily misdetected due to interference from external factors (such as light), such as left and right knees, left and right ankles, left and right elbows, or left and right wrists. In this embodiment, in order to solve the above-mentioned misdetection problem, the depth information of the target key points is introduced to explicitly learn the depth information of each key point, which helps to distinguish the left and right key points and assists the model in accurate recognition.

[0042] The trained 3D pose estimation model can be a currently open source 3D pose estimation model, such as ThreeDpose. By using the trained 3D pose estimation model, the depth information corresponding to the target key points in the sample image is predicted to update the annotation information corresponding to the target key points, that is, the annotation information corresponding to the target key points at this time includes the pre-annotated two-dimensional coordinate information and depth information.

[0043] In this embodiment, a trained three-dimensional posture estimation model is used to predict the depth information corresponding to the target key points, so as to reduce the complexity of model learning while ensuring the model recognition accuracy.

[0044] S302: Based on the depth information corresponding to the target key points, perform secondary annotation on the target key points in the sample image to obtain the secondary annotated sample image, wherein the annotation information of the secondary annotated sample image includes the depth information of the target key points and the pre-annotated position information.

[0045] Specifically, based on the depth information corresponding to the target key points, the target key points in the sample image are secondary labeled to obtain a secondary labeled sample image. At this time, the labeling information corresponding to the sample image includes coordinate information of N human key points and depth information corresponding to M target key points.

[0046] S303: training a lightweight posture detection model based on the updated sample image, so as to perform posture detection on the mobile terminal based on the trained posture detection model.

[0047] Specifically, after updating the annotation information corresponding to the sample image, the sample image and the corresponding updated annotation information can be used as training samples to train a lightweight posture detection model.

[0048] It is understandable that due to the small size of the current lightweight model, its detection capability for different postures is insufficient. Therefore, in this embodiment, data enhancement can be performed on the sample image, that is, by introducing depth information through secondary annotation of the target key points, the model is assisted in learning the key depth features of the target, thereby improving the detection accuracy of the model.

[0049] In one embodiment, if Figure 4 As shown, the annotation information includes the head position information; in step 202, the lying posture image is rotated and transformed into an upright human posture image, which specifically includes the following steps:

[0050] S401: Determine rotation transformation parameters according to the head position information.

[0051] The rotation transformation parameters include the rotation transformation direction and the rotation transformation angle. Specifically, since the head and legs are on the same horizontal line (left and right) in the lying posture image, the rotation transformation direction can be determined by the marked head position, such as left or right. For example, if the head position is on the left, it can be rotated clockwise or counterclockwise; if the head position is on the right, it can be rotated counterclockwise or clockwise.

[0052] S402: performing rotation transformation on the lying posture image based on the rotation transformation parameters to obtain an upright human posture image.

[0053] Specifically, based on the rotation transformation parameters, the lying posture image is rotationally transformed to obtain an upright human posture image with the head on top and the legs on the bottom, wherein the head and the legs in the upright human posture image are on the same vertical line.

[0054] In one embodiment, if Figure 5 As shown, the updated sample image includes a sideways posture image; the updated sample image includes a sideways posture image; the sideways posture image corresponds to a posture orientation; the posture orientation is used to indicate the horizontal orientation of the sideways action in the sideways posture image; before step S203, the data enhancement method based on posture detection also includes the following steps:

[0055] S501: When the posture orientation is different from the preset orientation, the side posture image is mirror-transformed into a side posture image with the preset orientation, and the sample image is updated.

[0056] S502: training a lightweight posture detection model based on the updated sample image, so as to perform posture detection on the mobile terminal based on the trained posture detection model.

[0057] It can be understood that, in the upright human posture image obtained after the rotation transformation, if the lying posture image is in a flat state, in the upright human posture image after the rotation transformation, its human body movement becomes a sideways movement, so in this embodiment, the sideways posture image includes an image of the human body posture in the upright human posture image after the rotation transformation and a sideways posture image existing in the sample image before the rotation transformation. The sideways posture image is pre-annotated with a corresponding posture orientation, such as left or right. Since the model needs to distinguish the left and right directions of the key points of the human body when distinguishing the sideways movement, in order to further reduce the difficulty of learning, the sideways posture image can be data enhanced, that is, its corresponding posture orientation is transformed into a unified orientation (i.e., a preset orientation), so that the model only needs to learn the depth information of the target key points, without learning the orientation information of the human body, so as to achieve the purpose of reducing the learning difficulty of the model. Among them, the preset orientation can be set to the left or right orientation according to actual needs, which is not limited here.

[0058] Specifically, by mirroring the sample image, the side posture image is mirrored into the side posture image of the preset orientation, and the sample image is updated, so that a lightweight posture detection model can be trained based on the updated sample image to perform posture detection on the mobile terminal based on the trained posture detection model.

[0059] Furthermore, when a trained posture detection model is used for posture detection on a mobile terminal, if there is a lying posture image in the multiple frames of video images in the captured video stream or the uploaded offline shot images, it can be rotated and transformed into an upright human posture image. At this time, the upright human posture image is a side posture image, which corresponds to a posture orientation. If the posture orientation is different from the preset orientation, the side posture image is mirrored into a side posture image with a preset orientation. Finally, the side posture image with a preset orientation is input into the trained posture detection model for posture detection, so as to realize the rapid operation of the model on the mobile terminal through the above-mentioned data enhancement means.

[0060] In one embodiment, before step S203, that is, before the step of performing posture detection on the mobile terminal based on the trained posture detection model, the data enhancement method based on posture detection also includes performing channel pruning on the last convolutional layer in the trained posture detection model, removing the feature channels corresponding to the target key points, and obtaining a two-dimensional posture detection model, so as to perform posture detection on the mobile terminal based on the two-dimensional posture detection model.

[0061] Among them, since this embodiment is mainly used for two-dimensional detection tasks, when using the trained posture detection model for posture detection, there is no need to output the depth information corresponding to the target key points. Therefore, it is necessary to perform channel pruning on the last convolution layer in the trained posture detection model to remove the feature channels corresponding to the target key points to obtain a two-dimensional posture detection model. When posture detection is performed through the two-dimensional posture detection model, only the two-dimensional coordinate information of all human key points in the sample image is predicted, which reduces the model calculation amount and improves the model calculation efficiency. However, since the model has learned the depth features of the target key points through the depth information of the introduced target key points during training, the detection accuracy of the model can still be guaranteed.

[0062] In one embodiment, the human body key points include the target key points and non-target key points; Figure 6 As shown, in step S203, training a lightweight posture detection model based on the updated sample image in the mobile terminal specifically includes the following steps:

[0063] S601: Input the sample image into the posture detection model to obtain the prediction information corresponding to the key points of the human body.

[0064] Specifically, the sample image is input into the posture detection model, and the prediction information corresponding to different human key points in the sample image can be output. The prediction information includes the predicted coordinate information of N human key points and the predicted depth information corresponding to M target key points. It can be understood that, assuming that the coordinate information of N human key points needs to be predicted, the coordinate information of each human key point will have coordinates in the x and y directions, then 2*N position information (x1, x2...x N ,y1,y2……y N ), in the side posture image, since it is an upright human posture image taken sideways, in order to reduce the complexity of model learning and ensure the model recognition accuracy at the same time, it is also necessary to predict the depth information of M target key points (z1, z2...z M ,).

[0065] S602: Calculate the model loss based on the prediction information and the annotation information after secondary annotation, and update the posture detection model based on the model loss to obtain a trained posture detection model; wherein the model loss includes the position information loss corresponding to the key points of the human body and the depth information loss of the target key points.

[0066] Specifically, in this embodiment, since depth information is introduced, it is necessary to reconstruct the model loss calculation function, and the following calculation formula is obtained: Among them, α and β are hyperparameters; N represents the number of key points of the human body; M represents the number of key points of the target; (x i ,y i ) represents the pre-marked coordinate information of the key points of the human body, Represents the predicted coordinate information of the key points of the human body; z i Indicates the depth information after secondary annotation corresponding to the target key point; Represents the predicted depth information corresponding to the target key point.

[0067] It can be understood that after calculating the model loss, the model is optimized by the model loss, and after iterative training until the model converges, a trained posture detection model can be obtained. Among them, the model optimization here can be optimized by the traditional error back propagation algorithm, which will not be described in detail here.

[0068] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0069] In one embodiment, a data enhancement device based on posture detection is provided, and the data enhancement device based on posture detection corresponds one-to-one to the data enhancement method based on posture detection in the above embodiment. Figure 7As shown, the data enhancement device based on posture detection includes an image acquisition module 10, a rotation transformation module 20 and a posture detection module 30. Each functional module is described in detail as follows:

[0070] The image acquisition module 10 is used to acquire pre-labeled sample images; wherein the sample images include lying posture images;

[0071] A rotation transformation module 20, used for rotationally transforming the lying posture image into an upright human posture image to update the sample image;

[0072] The posture detection module 30 is used to train a lightweight posture detection model based on the updated sample image, so as to perform posture detection based on the trained posture detection model in the mobile terminal.

[0073] Specifically, the data enhancement device based on posture detection also includes a depth information prediction module and a secondary labeling module.

[0074] A depth information prediction module, used to predict the depth information corresponding to the target key points using a trained 3D posture estimation model;

[0075] A secondary annotation module is used to perform secondary annotation on the target key points in the sample image based on the depth information corresponding to the target key points to obtain a sample image after secondary annotation; wherein the annotation information of the sample image after secondary annotation includes the depth information of the target key points and the pre-annotated position information.

[0076] Specifically, the updated sample image includes a side posture image; the side posture image corresponds to a posture orientation; the posture orientation is used to indicate the horizontal orientation of the side movement in the side posture image; the data enhancement device based on posture detection also includes: a mirror transformation module, which is used to mirror the side posture image into a side posture image with the preset orientation when the posture orientation is different from the preset orientation, so as to update the sample image.

[0077] Specifically, the data enhancement device based on posture detection also includes a model reconstruction module, which is used to perform channel pruning on the last convolutional layer in the trained posture detection model, remove the feature channels corresponding to the target key points, and obtain a two-dimensional posture detection model, so as to perform posture detection based on the two-dimensional posture detection model on the mobile terminal.

[0078] Specifically, the annotation information includes head position information; the rotation transformation module includes a rotation transformation parameter determination unit and a rotation transformation unit.

[0079] A rotation transformation parameter determination unit, used to determine the rotation transformation parameter according to the head position information;

[0080] A rotation transformation unit is used to perform rotation transformation on the lying posture image based on the rotation transformation parameters to obtain the upright human posture image.

[0081] Specifically, the model training module includes a prediction unit and a loss calculation unit.

[0082] A prediction unit, used for inputting the sample image into the posture detection model to obtain prediction information corresponding to each human body key point in the sample image;

[0083] A loss calculation unit is used to calculate the model loss based on the prediction information and the annotation information after secondary annotation, so as to update the posture detection model based on the model loss to obtain a trained posture detection model; wherein the model loss includes the position information loss corresponding to the human body key points and the depth information loss of the target key points.

[0084] For the specific definition of the data enhancement device based on posture detection, please refer to the definition of the data enhancement method based on posture detection above, which will not be repeated here. Each module in the above-mentioned data enhancement device based on posture detection can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0085] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a computer storage medium and an internal memory. The computer storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the computer storage medium. The database of the computer device is used to store data generated or acquired during the execution of a data enhancement method based on posture detection, such as a liveness detection model. The network interface of the computer device is used to communicate with an external mobile terminal through a network connection. When the computer program is executed by the processor, a data enhancement method based on posture detection is implemented.

[0086] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the data enhancement method based on posture detection in the above embodiment are implemented, such as Figure 2Steps S201-S203 shown, or Figures 3 to 6 Alternatively, when the processor executes the computer program, the functions of each module / unit in the embodiment of the data enhancement device based on posture detection are realized, for example Figure 7 The functions of the modules / units shown are not described here in detail to avoid repetition.

[0087] In one embodiment, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data enhancement method based on posture detection in the above embodiment are implemented, for example: Figure 2 Steps S201-S203 shown, or Figures 3 to 6 Alternatively, when the computer program is executed by a processor, the functions of each module / unit in the embodiment of the data enhancement device based on posture detection are realized, for example Figure 7 The functions of the modules / units shown are not described here in detail to avoid repetition.

[0088] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0089] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0090] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A data enhancement method based on posture detection, characterized in that: include: Acquire a pre-annotated sample image; wherein the sample image includes a lying posture image; the pre-annotated information of the sample image includes head position information; Rotate and transform the lying posture image into an upright human posture image to update the sample image; wherein the upright human posture image is a human posture image in which the human posture is in an upright state; the rotation and transform of the lying posture image into the upright human posture image comprises: determining a rotation transformation parameter according to the head position information; and performing a rotation transformation on the lying posture image based on the rotation transformation parameter to obtain the upright human posture image; A lightweight posture detection model is trained based on the updated sample images, so that posture detection is performed on the mobile terminal based on the trained posture detection model.

2. The data enhancement method based on posture detection as claimed in claim 1, characterized in that: The sample image corresponds to a plurality of target key points; Before training the lightweight posture detection model based on the updated sample image, the posture detection-based data enhancement method further includes: Using a trained 3D pose estimation model, predicting the depth information corresponding to the target key points; Based on the depth information corresponding to the target key points, the target key points in the sample image are secondary labeled to obtain a secondary labeled sample image; wherein the annotation information of the secondary labeled sample image includes the depth information of the target key points and pre-annotated position information.

3. The data enhancement method based on posture detection as claimed in claim 1, characterized in that: The updated sample image includes a sideways posture image; the sideways posture image corresponds to a posture orientation; the posture orientation is used to indicate the horizontal orientation of the sideways action in the sideways posture image; Before training the lightweight posture detection model based on the updated sample image, the posture detection-based data enhancement method further includes: When the posture orientation is different from the preset orientation, the sideways posture image is mirror-transformed into a sideways posture image with the preset orientation to update the sample image.

4. The data enhancement method based on posture detection as claimed in claim 2, characterized in that: Before the step of performing posture detection on the mobile terminal based on the trained posture detection model, the data enhancement method based on posture detection also includes: The last convolutional layer in the trained posture detection model is subjected to channel pruning processing to remove the feature channels corresponding to the target key points, so as to obtain a two-dimensional posture detection model, so as to perform posture detection based on the two-dimensional posture detection model on the mobile terminal.

5. The data enhancement method based on posture detection as claimed in claim 2, characterized in that: The step of training a lightweight posture detection model based on the updated sample image comprises: Inputting the sample image into the posture detection model to obtain prediction information corresponding to each human body key point in the sample image; Based on the prediction information and the annotation information after secondary annotation, the model loss is calculated to update the posture detection model based on the model loss to obtain a trained posture detection model; wherein the model loss includes the position information loss corresponding to the human body key points and the depth information loss of the target key points.

6. A data enhancement device based on posture detection, characterized in that: include: An image acquisition module, used to acquire a pre-annotated sample image; wherein the sample image includes a lying posture image; the pre-annotated information of the sample image includes head position information; A rotation transformation module, used for rotationally transforming the lying posture image into an upright human posture image to update the sample image; wherein the upright human posture image is a human posture image in which the human posture is in an upright state; the rotational transformation of the lying posture image into the upright human posture image comprises: determining a rotational transformation parameter according to the head position information; and performing a rotational transformation on the lying posture image based on the rotational transformation parameter to obtain the upright human posture image; The posture detection module is used to train a lightweight posture detection model based on the updated sample image, so as to perform posture detection based on the trained posture detection model in the mobile terminal.

7. The data enhancement device based on posture detection as claimed in claim 6, characterized in that: Also includes: The depth information prediction module is used to predict the depth information corresponding to the target key points using the trained 3D pose estimation model; The secondary annotation module is used to perform secondary annotation on the target key points in the sample image based on the depth information corresponding to the target key points to obtain the sample image after secondary annotation; wherein the annotation information of the sample image after secondary annotation includes the depth information of the target key points and the pre-annotated position information.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the data enhancement method based on posture detection according to any one of claims 1 to 5 are implemented.

9. A computer storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the data enhancement method based on posture detection according to any one of claims 1 to 5 are implemented.

Images