Posture detection method and device and computer equipment

Abstract

The invention relates to a posture detection method and device and computer equipment. The method is used for examining body position evaluation of an endoscope and comprises the steps that a front view angle image and a side view angle image of a detected object in an endoscopic examination scene are obtained; determining human body key point information and visibility information of human body key parts in the front view angle image and human body key point information in the side view angle image; determining the posture of the detection object based on the human body key point information and the visibility information of the human body key part in the front view angle image and the human body key point information in the side view angle image; and comparing the posture of the detection object with a preset posture standard to obtain a posture detection result of the detection object. By adopting the method, the problem that the posture detection result is inaccurate due to the influence of experience and subjective factors of workers in the existing posture detection method can be solved.

Description

Technical Field

[0001] The present application relates to the field of visual analysis technology, and in particular to a posture detection method, apparatus and computer equipment. Background Art

[0002] With the development of electronic technology, endoscope technology has also developed to a certain extent. The process of using an endoscope is to insert the endoscope into the body to observe the lesion site and perform operations. Therefore, people have higher and higher requirements for the comfort and imaging quality during the use of the endoscope. However, during the use of the endoscope, whether the posture of the object of detection is accurate will directly affect the comfort and imaging quality during the use of the endoscope. Therefore, before using the endoscope to work, the posture of the object of detection needs to be detected to ensure the comfort and imaging quality during the use of the endoscope.

[0003] Currently, the staff mainly judges whether the posture of the object to be inspected is correct based on their experience. Affected by the staff's experience and subjective factors, there is a problem of inaccurate posture detection results.

[0004] With regard to existing posture detection methods, there is a problem of inaccurate posture detection results due to the influence of staff experience and subjective factors, and no effective solution has been proposed yet. Summary of the invention

[0005] Based on this, it is necessary to provide a posture detection method, device and computer equipment to address the above technical problems.

[0006] In a first aspect, the present application provides a posture detection method for evaluating the posture of an endoscope, the method comprising the following steps:

[0007] Acquire a front view image and a side view image of the detection object in the endoscope inspection scene;

[0008] Determine the key point information of the human body and the visibility information of the key parts of the human body in the front view image, and the key point information of the human body in the side view image;

[0009] Determine the posture of the detection object based on the human body key point information and the visibility information of the human body key parts in the front view image, and the human body key point information in the side view image; the posture of the detection object includes one or more of the head posture, body posture and limb posture of the detection object;

[0010] The posture of the detected object is compared with a preset posture standard to obtain a posture detection result of the detected object.

[0011] In one embodiment, the determining of the key point information of the human body and the visibility information of the key parts of the human body in the front view image, and the key point information of the human body in the side view image, comprises the following steps:

[0012] The trained key point detection model is used to perform key point detection on the front view image and the side view image respectively to obtain the human body key point information and the visibility information of the key parts of the human body in the front view image, as well as the human body key point information in the side view image.

[0013] In one embodiment, determining the posture of the detected object based on the human body key point information and the visibility information of the human body key parts in the front view image, and the human body key point information in the side view image, comprises the following steps:

[0014] Based on the key point information of the human body in the frontal perspective image, determine the ratio of the shoulder width to the body length, the angle of the limbs, and the position information of the two shoulders of the human body in the frontal perspective image;

[0015] Determine the angles of the limbs of the human body in the side perspective image based on the key point information of the human body in the side perspective image;

[0016] Determine the head posture and body posture of the inspection object based on the ratio of the shoulder width to the body length of the human body in the frontal perspective image, the position information of the two shoulders, and the visibility information of the key parts of the human body;

[0017] The postures of the limbs of the inspection object are determined based on the angles of the limbs of the human body in the front view image and the angles of the limbs of the human body in the side view image.

[0018] In one embodiment, determining the angles of the limbs comprises the following steps:

[0019] Based on the human body key point information in the front view image or the side view image, determine the main key points of the limbs of the human body in the front view image or the side view image; the main key points of the limbs include at least the key points at the shoulder joints, the key points at the elbow joints, and the key points at the wrist joints of each arm, and the key points at the hip joints, the key points at the knee joints, and the key points at the ankle joints of each leg;

[0020] Based on the main key points of the limbs of the human body in the front perspective image or the side perspective image, various angles of the limbs of the human body in the front perspective image or the side perspective image are determined.

[0021] In one of the embodiments, determining the angles of the limbs of the human body in the front view image or the side view image based on the main key points of the limbs of the human body in the front view image or the side view image comprises the following steps:

[0022] Based on the key points at the shoulder joints, the key points at the elbow joints, and the key points at the wrist joints of each arm of the human body in the front view image or the side view image, determine the first vector and the second vector of each arm of the human body in the front view image or the side view image; the first vector is a vector obtained by taking the key point at the elbow joint as the starting point and pointing to the key point at the shoulder joint, or a vector obtained by taking the key point at the shoulder joint as the starting point and pointing to the key point at the elbow joint; the second vector is a vector obtained by taking the key point at the elbow joint as the starting point and pointing to the key point at the wrist joint, or a vector obtained by taking the key point at the wrist joint as the starting point and pointing to the key point at the elbow joint;

[0023] Determine each angle of each arm of the human body in the frontal perspective image or the side perspective image based on the first vector and the second vector of each arm of the human body in the frontal perspective image or the side perspective image;

[0024] Based on the key points at the hip joints, the key points at the knee joints, and the key points at the ankle joints of each leg of the human body in the front view image or the side view image, determine the third vector and the fourth vector of each leg of the human body in the front view image or the side view image; the third vector is a vector obtained by taking the key point at the knee joint as the starting point and pointing to the key point at the hip joint, or a vector obtained by taking the key point at the hip joint as the starting point and pointing to the key point at the knee joint; the fourth vector is a vector obtained by taking the key point at the knee joint as the starting point and pointing to the key point at the ankle joint, or a vector obtained by taking the key point at the ankle joint as the starting point and pointing to the key point at the knee joint;

[0025] Based on the third vector and the fourth vector of each leg of the human body in the front perspective image or the side perspective image, each angle of each leg of the human body in the front perspective image or the side perspective image is determined.

[0026] In one embodiment, before using the trained key point detection model to perform key point detection on the front view image and the side view image respectively to obtain the human body key point information in the front view image and the human body key point information in the side view image, the following steps are included:

[0027] Acquire samples of front view images and side view images of a plurality of inspection objects in the endoscopic inspection scene to obtain a sample set;

[0028] Annotating key parts of the human body for each image in the sample set to obtain an annotated sample set;

[0029] Dividing the labeled sample set into a training set, a validation set, and a test set;

[0030] Based on the training set, the validation set and the test set, the YOLOv8 pre-trained model is trained to obtain the trained key point detection model.

[0031] In one embodiment, the comparison between the posture of the detected object and a preset posture standard to obtain the posture detection result of the detected object includes the following steps:

[0032] Based on the head posture, body posture and limb posture of the detection object, as well as preset human head posture standards, human body posture standards and human limb posture standards, posture detection is performed on the detection object to obtain the posture detection result of the detection object; the posture detection result includes standard posture and non-standard posture.

[0033] In one embodiment, after obtaining the posture detection result of the detection object by comparing the posture of the detection object with a preset posture standard, the following steps are included:

[0034] When the posture detection result of the detection object is that the posture is not standard, a posture error reminder is issued.

[0035] In a second aspect, the present application also provides a posture detection device for evaluating body position during endoscope inspection, the device comprising:

[0036] An image acquisition module, used to acquire a front view image and a side view image of the detection object in the endoscopic inspection scene;

[0037] An information determination module, used to determine the key point information of the human body and the visibility information of the key parts of the human body in the front view image, and the key point information of the human body in the side view image;

[0038] a posture determination module, configured to determine the posture of the detection object based on the human body key point information and the visibility information of the human body key parts in the front view image, and the human body key point information in the side view image; the posture of the detection object is one or more of the head posture, body posture and limb posture of the detection object;

[0039] And a detection module is used to compare the posture of the detection object with a preset posture to obtain a posture detection result of the detection object.

[0040] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the posture detection method described in the first aspect is implemented.

[0041] The above-mentioned posture detection method, device and computer equipment obtain the front view image and side view image of the detection object in the endoscopic inspection scene, and determine the human body key point information and the visibility information of the human body key parts in the front view image, as well as the human body key point information in the side view image, and then determine the posture of the detection object based on the human body key point information and the visibility information of the human body key parts in the front view image, as well as the human body key point information in the side view image, and then compare the posture of the detection object with the preset posture standard to obtain the posture detection result of the detection object. During the entire posture detection process, no human participation is required, and the posture of the detection object can be automatically detected. Through the preset posture standard, the posture detection result can be made more accurate, which solves the problem of inaccurate posture detection results in the existing posture detection method due to the influence of the staff's experience and subjective factors.

[0042] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0044] Figure 1 A hardware structure block diagram of a terminal of a posture detection method provided in an embodiment of the present application;

[0045] Figure 2 A flowchart of a posture detection method provided in one embodiment of the present application;

[0046] Figure 3 A flowchart of a posture detection method provided in a preferred embodiment of the present application;

[0047] Figure 4 This is a structural block diagram of a posture detection device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0049] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of the posture detection method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0051] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the posture detection method in the present embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0052] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0053] In this embodiment, a posture detection method is provided for evaluating the body position of an endoscope. Figure 2 is a flow chart of the posture detection method of this embodiment. Figure 2 As shown, the process includes the following steps:

[0054] Step S210, obtaining a front view image and a side view image of the detection object in an endoscopic inspection scene.

[0055] The above-mentioned endoscopic inspection scene may be an actual application scene of inspecting the target part of the human body by using an endoscope. It should be noted that the above-mentioned endoscopic inspection scene may be inspecting different parts of the human body by using an endoscope. The above-mentioned detection object may be an object that needs to be inspected by using an endoscope, that is, an object that needs to be inspected and evaluated for posture before the endoscopic inspection. The above-mentioned frontal perspective image may be an image of the detection object captured by a shooting device located directly in front of the detection object. The above-mentioned side perspective image may be an image of the detection object captured by a shooting device located on the left or right side of the detection object. Among them, the above-mentioned shooting device may be one or more of a video camera or a still camera.

[0056] Step S220, determining the key point information of the human body and the visibility information of the key parts of the human body in the front view image, and the key point information of the human body in the side view image.

[0057] In this step, the above-mentioned human key point information can be the information of human joints (such as shoulder joints, elbow joints, wrist joints, hip joints, knee joints, ankle joints), facial key points (such as nose, eyes, ears) or other key points (such as hips, neck, fingers) and the like. The above-mentioned key point information can be the position information of each key point in the same coordinate system. The above-mentioned determination of the human key point information and the visibility information of the key parts of the human body in the front view image, as well as the determination of the human key point information in the side view image, can be to use a preset method to determine the human key point information and the visibility information of the key parts of the human body in the front view image, as well as the human key point information in the side view image. The above-mentioned preset method can be to use a trained neural network model, a preset key point detection algorithm and the like. The above-mentioned trained neural network model can be one or more of a Transformer-based neural network model, an OpenPose neural network model, an HRNet (High-Resolution Network) neural network model, a Simple Baseline neural network model, and the like. The above-mentioned preset key point detection algorithm can be a human key point detection algorithm based on deep learning, such as one or more of the key point detection algorithms such as DeepPose, Hourglass, and AlphaPose.

[0058] The above-mentioned key parts of the human body are mainly key parts that can be used to identify the posture of the human body, such as key parts of the face used to identify the facial orientation of the human body, for example, the eyes, ears, nose, mouth, etc. of the human body, specifically, the left eye, right eye, left ear, right ear, etc. The visibility information of the above-mentioned key parts of the human body can be whether the key parts of the human body are visible. For example, when the eyes of the human body cannot be displayed in the frontal perspective image, it can be determined that the eyes are not visible in the visibility information of the key parts in the frontal perspective image.

[0059] Step S230, determining the posture of the detection object based on the human body key point information and the visibility information of the human body key parts in the frontal perspective image, as well as the human body key point information in the side perspective image; the posture of the detection object includes one or more of the detection object's head posture, body posture and limb posture.

[0060] Wherein, the above-mentioned determination of the posture of the detection object based on the human body key point information and the visibility information of the key parts of the human body in the front view image, and the human body key point information in the side view image, can be based on the human body key point information in the front view image, and the human body key point information in the side view image, determine the ratio of the shoulder width and body length of the human body in the front view image, the angle of the limbs and the position information of the two shoulders, and the angle of the limbs of the human body in the side view image, and then, based on the ratio of the shoulder width and body length of the human body in the front view image, the angle of the limbs, the position information of the two shoulders, the visibility information of the key parts of the human body, and the angle of the limbs of the human body in the side view image, determine the posture of the inspection object. The above-mentioned head posture can include postures such as whether the head is tilted up, buried, or sideways, and the direction of the head when sideways. The above-mentioned body posture can include postures such as whether the body is lying prone, lying flat, or sideways, and the direction of the body. The above-mentioned limb posture can include information such as whether the limbs are curled up, the angle of the limbs, and the curling direction of the limbs. The angles of the limbs include the angles of the limbs in the front view image and the angles of the limbs in the side view image. It should be noted that the angles of the limbs include the angle of the left arm, the angle of the right arm, the angle of the left leg and the angle of the right leg. The body refers to the part other than the head and limbs.

[0061] Preferably, the posture of the inspection object can also be determined based on the human body key point information and the visibility information of the human body key parts in the frontal perspective image, and the human body key point information and the visibility information of the human body key parts in the side perspective image. Specifically, the human body key point information and the visibility information of the human body key parts in the frontal perspective image can be used to determine the ratio of the human body's shoulder width to body length, the angle of the four limbs, and the position information of the two shoulders in the frontal perspective image, and the human body key point information and the visibility information of the human body key parts in the side perspective image can be used to determine the ratio of the human body's shoulder width to body length, the angle of the four limbs, and the position information of the two shoulders in the side perspective image, and then according to the ratio of the human body's shoulder width to body length, the angle of the four limbs, the position information of the two shoulders and the visibility information of the human body key parts in the frontal perspective image, and the ratio of the human body's shoulder width to body length, the angle of the four limbs, the position information of the two shoulders and the visibility information of the human body key parts in the side perspective image, the posture of the inspection object can be determined.

[0062] Step S240 , comparing the posture of the detected object with a preset posture standard to obtain a posture detection result of the detected object.

[0063] Because different parts of the endoscopic examination have different standard postures, different posture standards can be pre-set according to different endoscopic examination scenes, and the posture of the object to be detected is compared with the preset posture standard to determine whether the posture of the object to be detected is accurate. The posture detection result of the object to be detected is obtained by comparing the posture of the object to be detected with the preset posture standard. It can be based on the head posture, body posture and limb posture of the object to be detected. The posture detection result is compared with the preset posture standard corresponding to the current endoscopic examination scene to determine whether the current posture of the object to be detected is correct. If the current posture of the object to be detected is correct, the posture detection result of the object to be detected is the posture standard. If the current posture of the object to be detected is incorrect, the posture detection result of the object to be detected is the posture non-standard. It should be noted that, according to different needs, the preset posture standard can be set to one or more of the head posture, body posture and limb posture. If there are multiple preset posture standards, the posture detection result of the posture standard can only be obtained when all the postures of the object to be detected meet the preset posture standards. If any of the postures of the object to be detected does not meet the preset posture standard, a posture detection result of the posture non-standard is obtained.

[0064] In the above steps S210 to S240, by acquiring the front view image and the side view image of the detection object in the endoscopic inspection scene, and determining the human body key point information and the visibility information of the human body key parts in the front view image, as well as the human body key point information in the side view image, and then determining the posture of the detection object based on the human body key point information and the visibility information of the human body key parts in the front view image, as well as the human body key point information in the side view image, and then comparing the posture of the detection object with the preset posture standard to obtain the posture detection result of the detection object. In the entire posture detection process, no human participation is required, and the posture detection of the detection object can be automatically performed. Through the preset posture standard, the posture detection result can be made more accurate, which solves the problem of inaccurate posture detection results in the existing posture detection method due to the influence of the staff's experience and subjective factors.

[0065] In one embodiment, the above step S220, determining the key point information of the human body and the visibility information of the key parts of the human body in the front view image, and the key point information of the human body in the side view image, includes:

[0066] Step S222, using the trained key point detection model, perform key point detection on the front view image and the side view image respectively, to obtain the human body key point information and visibility information of key parts of the human body in the front view image, as well as the human body key point information in the side view image.

[0067] The above-mentioned step S222 determines the human body key point information and the visibility information of the human body key parts in the frontal perspective image, as well as the human body key point information in the side perspective image, through a preset method, so as to facilitate the subsequent determination of the posture of the detection object based on the human body key point information and the visibility information of the human body key parts in the frontal perspective image, as well as the human body key point information in the side perspective image.

[0068] Specifically, in one embodiment, based on step S230, based on the human body key point information and the visibility information of the key parts of the human body in the front view image, and the human body key point information in the side view image, determining the posture of the detection object includes:

[0069] Step S232, based on the key point information of the human body in the frontal perspective image, determine the ratio of the shoulder width to the body length, the angle of the limbs and the position information of the two shoulders of the human body in the frontal perspective image.

[0070] The above-mentioned determination of the ratio of the shoulder width and body length of the human body in the frontal perspective image, the angle of the limbs and the position information of the two shoulders based on the key point information of the human body in the frontal perspective image may be based on the key point information of the human body in the frontal perspective image to determine the shoulder width and body length of the human body in the frontal perspective image; based on the shoulder width and body length of the human body in the frontal perspective image, to determine the ratio of the shoulder width and body length of the human body in the frontal perspective image; based on the key points on the limbs of the human body in the frontal perspective image, to determine the angle of the limbs of the human body in the frontal perspective image; based on the position of the key points at each shoulder joint of the human body in the frontal perspective image, to determine the position information of the two shoulders. The above-mentioned angle of the limbs may include the angle between the upper arm and the lower arm of each arm, the angle between the thigh and the lower leg of each leg, and may also include the angle between each upper arm and the body, and the angle between each thigh and the body.

[0071] The above determination of the shoulder width of the human body can be to determine the position information of the key points at each shoulder joint of the human body in the frontal perspective image, and take the distance between the position of the key point at the left shoulder joint of the human body and the position of the key point at the right shoulder joint of the human body as the shoulder width of the human body. The above body length is the distance between the key point at the shoulder joint on either side of the human body and the key point at the hip joint on the same side. For example, the body length can be the distance between the key point at the left shoulder joint and the key point at the left hip joint. Based on this, when the shoulder width and body length of the human body in the frontal perspective image are determined, the ratio of the shoulder width to the body length of the human body is determined based on the shoulder width and body length of the human body in the frontal perspective image.

[0072] Step S234, determining the angles of the limbs of the human body in the side perspective image based on the key point information of the human body in the side perspective image.

[0073] The above-mentioned method of determining the angle of the limbs of the human body in the side perspective image based on the key point information of the human body in the side perspective image may be based on the key point information of the human body in the side perspective image to determine the key points on the limbs of the human body in the side perspective image, and based on the key points on the limbs of the human body in the side perspective image to determine the angle of the limbs of the human body in the side perspective image.

[0074] Step S236, determining the head posture and body posture of the inspection object based on the ratio of the shoulder width and body length of the human body in the frontal perspective image, the position information of the two shoulders, and the visibility information of the key parts of the human body.

[0075] The above-mentioned determination of the head posture and body posture of the inspection object based on the ratio of the shoulder width and body length of the human body in the frontal perspective image, the position information of the two shoulders, and the visibility information of the key parts of the human body can be based on the ratio of the shoulder width and body length of the human body in the frontal perspective image, the position information of the two shoulders, and the visibility information of the key parts of the human body, according to a preset table, to determine the head posture and body posture of the inspection object; based on the angles of the limbs of the human body in the frontal perspective image and the side perspective image, to determine the limb posture of the inspection object. Specifically, the head posture and body posture of the inspection object can be determined according to a preset table based on whether the ratio of the shoulder width and body length of the human body in the frontal perspective image is greater than a preset threshold value of the ratio of the shoulder width and body length, the position relationship information of the two shoulders (the size relationship of the coordinates of the key points of the two shoulders), and the visibility information of the eyes and ears.

[0076] The above preset table may be as shown in Table 1 below:

[0077] Table 1

[0078]

[0079] Among them, "large" in Table 1 means that the ratio of shoulder width to body length is greater than the preset threshold value of the ratio of shoulder width to body length, and "small" means that the ratio of shoulder width to body length is less than the preset threshold value of the ratio of shoulder width to body length. The above threshold value of the ratio of shoulder width to body length can be specifically set according to the specific situation. This embodiment does not make specific limitations here, as long as it can be judged whether the body is lying flat or lying on its side by the preset ratio of shoulder width to body length. The coordinates of the key points at the above-mentioned right shoulder joint are close to the coordinate positions of the key points at the left shoulder joint, and the difference between the coordinate positions of the key points at the right shoulder joint and the key points at the left shoulder joint can be less than the preset position distance threshold. The above-mentioned preset position distance threshold can be set according to specific needs, as long as it can be distinguished whether the body is lying on its side or lying flat by the preset position distance threshold.

[0080] Furthermore, the orientation of the head can be determined based on the visibility of the eyes and ears in the frontal view image.

[0081] Based on the relationship between the shoulder width in the frontal perspective image and the preset shoulder width threshold, it can be determined whether the body is lying on its side or lying flat. Furthermore, based on the positional relationship between the coordinates of the key points at the left shoulder joint and the coordinates of the key points at the right shoulder joint, it can be determined whether the body is lying prone or lying flat. The specific process can be seen in Table 1.

[0082] The above steps S232 to S236 determine the posture of the inspection object by determining the ratio of the shoulder width and body length of the human body in the frontal perspective image, the angle of the limbs and the position information of the two shoulders, and the angle of the limbs of the human body in the side perspective image, and then determine the posture of the inspection object based on the ratio of the shoulder width and body length of the human body in the frontal perspective image, the angle of the limbs, the position information of the two shoulders, the visibility information of the key parts of the human body, and the angle of the limbs of the human body in the side perspective image. By determining the posture of the body of the inspection object, it is convenient to compare with the preset posture standard subsequently to obtain the posture detection result of the inspection object.

[0083] In addition, in one embodiment, the above-mentioned determination of the angles of the four limbs includes:

[0084] Step S1, based on the human body key point information in the front view image or the side view image, determine the main key points of the limbs of the human body in the front view image or the side view image; the main key points of the limbs include at least the key points at the shoulder joints, the key points at the elbow joints, the key points at the wrist joints of each arm, and the key points at the hip joints, the key points at the knee joints, and the key points at the ankle joints of each leg.

[0085] Step S2, based on the main key points of the limbs of the human body in the frontal perspective image or the side perspective image, determine the various angles of the limbs of the human body in the frontal perspective image or the side perspective image.

[0086] The above steps S1 to S2 determine the main key points of the human body's limbs in the front perspective image or the side perspective image based on the human body key point information in the front perspective image or the side perspective image, and then determine the various angles of the human body's limbs in the front perspective image or the side perspective image. The calculation of the angles of the limbs facilitates the subsequent determination of the body posture of the detection object based on the angles of the limbs, so as to perform posture detection on the detection object based on the body posture of the detection object.

[0087] In one embodiment, based on step S2, based on the main key points of the limbs of the human body in the frontal perspective image or the side perspective image, determining the angles of the limbs of the human body in the frontal perspective image or the side perspective image includes:

[0088] Step S22, based on the key points at the shoulder joints, the key points at the elbow joints, and the key points at the wrist joints of each arm of the human body in the front view image or the side view image, determine the first vector and the second vector of each arm of the human body in the front view image or the side view image; the first vector is a vector obtained by taking the key point at the elbow joint as the starting point and pointing to the key point at the shoulder joint, or a vector obtained by taking the key point at the shoulder joint as the starting point and pointing to the key point at the elbow joint; the second vector is a vector obtained by taking the key point at the elbow joint as the starting point and pointing to the key point at the wrist joint, or a vector obtained by taking the key point at the wrist joint as the starting point and pointing to the key point at the elbow joint.

[0089] The above-mentioned determination of the first vector and the second vector of each arm of the human body in the front view image or the side view image based on the key points at the shoulder joints, the key points at the elbow joints, and the key points at the wrist joints of each arm of the human body in the front view image or the side view image can be explained by the following example:

[0090] For example, the coordinates of the key point P1 at the elbow joint of the human arm are (X1, Y1), the coordinates of the key point P2 at the shoulder joint are (X2, Y2), and the coordinates of the key point P3 at the wrist joint are (X3, Y3), then the first vector of the human arm is:

[0091]

[0092] The second vector of the human arm is:

[0093]

[0094] Step S24, determining each angle of each arm of the human body in the front perspective image or the side perspective image based on the first vector and the second vector of each arm of the human body in the front perspective image or the side perspective image.

[0095] According to the first vector and the second vector of the human arm, the calculation formula for determining the angle θ of the arm is:

[0096]

[0097] Although, the above only lists the case where the vector obtained by taking the key point at the elbow joint as the starting point and pointing to the key point at the shoulder joint is the first vector, and the vector obtained by taking the key point at the elbow joint as the starting point and pointing to the key point at the wrist joint is the second vector, and only lists a way to calculate the angle of the arm through the first vector and the second vector. It should be noted that as long as the vectors can be constructed through the key points of each arm of the human body, the angle of the arm can be determined according to the relationship between the vectors, and the specific calculation process is not specifically limited in this embodiment.

[0098] Step S26, based on the key points at the hip joints, the key points at the knee joints, and the key points at the ankle joints of each leg of the human body in the front view image or the side view image, determine the third vector and the fourth vector of each leg of the human body in the front view image or the side view image; the third vector is a vector obtained by taking the key point at the knee joint as the starting point and pointing to the key point at the hip joint, or a vector obtained by taking the key point at the hip joint as the starting point and pointing to the key point at the knee joint; the fourth vector is a vector obtained by taking the key point at the knee joint as the starting point and pointing to the key point at the ankle joint, or a vector obtained by taking the key point at the ankle joint as the starting point and pointing to the key point at the knee joint.

[0099] The above-mentioned determination of the third vector and the fourth vector of each leg of the human body in the front view image or the side view image based on the key points at the hip joints, the key points at the knee joints, and the key points at the ankle joints of each leg of the human body in the front view image or the side view image can be illustrated by the following example:

[0100] For example, the coordinates of the key point P4 at the knee joint of the human leg are (X4, Y4), the coordinates of the key point P5 at the hip joint are (X5, Y5), and the coordinates of the key point P6 at the ankle joint are (X6, Y6). Then the third vector of the human leg is:

[0101]

[0102] The fourth vector of the human leg is:

[0103]

[0104] Step S28, determining each angle of each leg of the human body in the front perspective image or the side perspective image based on the third vector and the fourth vector of each leg of the human body in the front perspective image or the side perspective image.

[0105] According to the third vector and the fourth vector of the human body's leg, the calculation formula for determining the angle α of the leg is:

[0106]

[0107] Although, the above only lists the case where the vector obtained by taking the key point at the knee joint as the starting point and pointing to the key point at the hip joint is the third vector, and the vector obtained by taking the key point at the knee joint as the starting point and pointing to the key point at the ankle joint is the fourth vector, and only lists a method of calculating the angle of the leg by using the third vector and the fourth vector. It should be noted that as long as the vectors can be constructed through the key points of each leg of the human body, the angle of the leg can be determined according to the relationship between the vectors, and the specific calculation process is not specifically limited in this embodiment.

[0108] The above steps S22 to S28 determine the various angles of the human body's limbs in the front perspective image or the side perspective image through the human body key point information in the front perspective image or the side perspective image. By determining the various angles of the human body's limbs in the front perspective image and the side perspective image, the limb postures of the detection object are determined, so as to evaluate whether the posture of the detection object is standard according to the different angles of the limbs at different shooting angles.

[0109] In addition, in one embodiment, before step S222, the following steps are included:

[0110] Step S221, training the YOLOv8 pre-trained model to obtain a trained key point detection model.

[0111] Among them, the YOLOv8 pre-trained model is the V8 version of the YOLO (You Only Look Once) series of target detection algorithms.

[0112] Furthermore, in one embodiment, the above step S221, training the YOLOv8 pre-trained model to obtain a trained key point detection model, includes:

[0113] Step S2212, acquiring samples of front view images and side view images of a plurality of inspection objects in an endoscopic inspection scene to obtain a sample set.

[0114] The above-mentioned samples of front view images and side view images of multiple inspection objects in the endoscopic inspection scene can be obtained by obtaining samples of front view images and side view images of different inspection objects in different endoscopic inspection scenes. Among them, different endoscopic inspection scenes can be endoscopic inspection scenes that require inspection of different parts of the object, and different endoscopic inspection scenes correspond to different postures of the inspection object. For example, gastroscopy examination position, colonoscopy examination position, laryngoscopy examination position, etc.

[0115] Step S2214: annotate key parts of the human body for each image in the sample set to obtain an annotated sample set.

[0116] For example, the key parts of the human body of each image in the sample set can be annotated in the format of ["nose", "left_eye", "right_eye", "left_ear", "right_ear", "left_shoulder", "right_shoulder", "left_elbow", "right_elbow", "left_wrist", "right_wrist", "left_hip", "right_hip", "left_knee", "right_knee", "left_ankle", "right_ankle"]. It should be noted that an XML file is generated for each annotated image, and then the XML file is converted into a TXT file. The TXT file may include categories, human detection boxes and key point information. The above categories may be digital numbers of the detection objects. For example, the number of a person is 0. In addition to people, the detection objects may also be other categories, such as cats, dogs, etc.

[0117] Step S2216, dividing the labeled sample set into a training set, a validation set and a test set.

[0118] In this step, the above-mentioned dividing the labeled sample set into the training set, the validation set and the test set may be dividing the labeled sample set into the training set, the validation set and the test set according to a preset ratio. For example, the labeled sample set may be divided into the training set, the validation set and the test set according to a ratio of 6:2:2.

[0119] In addition, you can create a dataset configuration file mydata.yaml to store the paths and categories of the training set, validation set, and test set.

[0120] Step S2218, based on the training set, the validation set and the test set, the YOLOv8 pre-trained model is trained to obtain a trained key point detection model.

[0121] Based on the training set, validation set and test set, the YOLOv8 pre-trained model is trained to obtain the trained key point detection model. The training set can be trained with the yolov8s-pose.pt pre-trained model and mydata.yaml, the performance of the model is evaluated on the validation set, the hyperparameters (learning rate, batch size and number of iterations) or the model structure are manually adjusted to prevent overfitting, and the training is continued and the performance of the model is re-evaluated with the validation set until the optimal hyperparameters are found and the model best.pt is saved, and the accuracy, precision, recall and generalization ability of the model best.pt are evaluated with the test set, and the best.pt model that meets the preset accuracy, precision, recall and generalization requirements is used as the trained key point detection model. In the above steps S2212 to S2218, the trained key point detection model is obtained by training the YOLOv8 pre-trained model, so as to facilitate the use of the trained key point detection model to determine the human key point information and the visibility information of the key parts of the human body in the frontal perspective image, as well as the human key point information in the side perspective image.

[0122] In one embodiment, based on step S240, the posture of the detected object is compared with a preset posture standard to obtain a posture detection result of the detected object, including:

[0123] Step S242, based on the head posture, body posture and limb posture of the detection object, as well as the preset human head posture standard, human body posture standard and human limb posture standard, posture detection is performed on the detection object to obtain the posture detection result of the detection object; the posture detection result includes standard posture and non-standard posture.

[0124] The above-mentioned preset standards for the head posture of the human body may be posture standards such as whether the head is tilted up, buried or sideways, and the direction of the head when sideways. The above-mentioned preset standards for the body posture of the human body may be posture standards such as whether the body is prone, lying flat or sideways, and the direction of the body when sideways. The above-mentioned preset standards for the limb posture of the human body may be standards such as the range of angles of the limbs. It should be noted that, because there are two images acquired, namely a frontal perspective image and a side perspective image, the range of angles of the limbs may include the range of angles of the limbs under the frontal perspective, and the range of angles of the limbs under the side perspective.

[0125] For example, in a certain endoscopy inspection scene, the object to be detected needs to have its head facing right, its body lying on its side, its legs curled up, the angle of its legs in the front view is between 60° and 90° (both the angles of its legs need to meet the conditions), and the angle of its legs in the side view is 180°. The above conditions are set as the preset human posture standards. Based on the ratio of the human body's shoulder width to body length in the front view image, the angle of its limbs, the height information of its two shoulders, the visibility information of its key parts, and the angle of its limbs in the side view image, it is determined that the head posture of the object to be detected is facing right, the body posture is lying on its side, the limb posture is curled up, the angle of its legs in the front view is 80°, and the angle of its legs in the side view is 180°. The detection result of the object to be detected is the posture standard.

[0126] In the above steps S242 to S244, the head posture, body posture and limb posture of the detection object are determined, and then, one or more of the head posture, body posture and limb posture of the detection object are compared with the preset head posture standard, body posture standard and limb posture standard of the human body, and posture detection is performed on the detection object to obtain the posture detection result of the detection object. In the whole posture detection process, no human participation is required, and the posture detection of the detection object can be automatically performed. Through the preset posture standard, the posture detection result can be made more accurate, which solves the problem of inaccurate posture detection results in the existing posture detection method due to the influence of the experience and subjective factors of the staff.

[0127] Specifically, in one embodiment, after step S240, the following steps are included:

[0128] Step S250, when the posture detection result of the detected object is that the posture is not standard, a posture error reminder is issued.

[0129] In this step, when the posture detection result of the detection object is not standard, a posture error reminder is issued to remind the detection object to adjust the posture in time to meet the posture requirements of the current endoscope inspection scene.

[0130] The present embodiment is described and illustrated below through preferred embodiments.

[0131] Figure 3 Flow chart of a posture detection method provided by a preferred embodiment of the present application. Figure 3 As shown, the posture detection method includes the following steps:

[0132] Step S301, obtaining a front view image and a side view image of a detection object in an endoscope inspection scene;

[0133] Step S302, training the YOLOv8 pre-trained model to obtain a trained key point detection model;

[0134] Step S303, using the trained key point detection model, performing key point detection on the front view image and the side view image respectively, to obtain the human body key point information and visibility information of key parts of the human body in the front view image, and the human body key point information in the side view image;

[0135] Step S304, based on the key point information and visibility information of the key parts of the human body in the frontal perspective image, and the key point information of the human body in the side perspective image, determine the ratio of the shoulder width to the body length, the angle of the limbs, and the position information of the two shoulders of the human body in the frontal perspective image, and the angle of the limbs of the human body in the side perspective image;

[0136] Step S305, determining the posture of the inspection object based on the ratio of the shoulder width to the body length of the human body in the front view image, the angle of the four limbs, the position information of the two shoulders, the visibility information of the key parts of the human body, and the angle of the four limbs of the human body in the side view image;

[0137] Step S306, comparing the posture of the detected object with a preset posture standard to obtain a posture detection result of the detected object;

[0138] Step S307, when the posture detection result of the detected object is that the posture is not standard, a posture error reminder is issued.

[0139] In the above steps S301 to S307, by acquiring the front view image and the side view image of the detection object in the endoscopic inspection scene, and determining the human body key point information and the visibility information of the human body key parts in the front view image, as well as the human body key point information in the side view image, and then determining the posture of the detection object based on the human body key point information and the visibility information of the human body key parts in the front view image, as well as the human body key point information in the side view image, and then comparing the posture of the detection object with the preset posture standard to obtain the posture detection result of the detection object. In the entire posture detection process, no human participation is required, and the posture detection of the detection object can be automatically performed. Through the preset posture standard, the posture detection result can be made more accurate, which solves the problem of inaccurate posture detection results in the existing posture detection method due to the influence of the staff's experience and subjective factors.

[0140] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0141] Based on the same inventive concept, a posture detection device is also provided in this embodiment for endoscope inspection posture assessment. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware for predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0142] In one embodiment, Figure 4 is a structural block diagram of a posture detection device provided in an embodiment of the present application, such as Figure 4 As shown, the posture detection device comprises:

[0143] An image acquisition module 42 is used to acquire a front view image and a side view image of a detection object in an endoscopic inspection scene;

[0144] An information determination module 44, used to determine the key point information of the human body and the visibility information of the key parts of the human body in the front view image, and the key point information of the human body in the side view image;

[0145] a posture determination module 46, configured to determine the posture of the detection object based on the human body key point information and the visibility information of the human body key parts in the front view image, and the human body key point information in the side view image; the posture of the detection object is one or more of the head posture, body posture and limb posture of the detection object;

[0146] And the detection module 48 is used to compare the posture of the detection object with the preset posture to obtain the posture detection result of the detection object.

[0147] The above-mentioned posture detection device obtains the front view image and the side view image of the detection object in the endoscopic inspection scene, and determines the human body key point information and the visibility information of the human body key parts in the front view image, as well as the human body key point information in the side view image, and then determines the posture of the detection object based on the human body key point information and the visibility information of the human body key parts in the front view image, as well as the human body key point information in the side view image, and then compares the posture of the detection object with the preset posture standard to obtain the posture detection result of the detection object. During the entire posture detection process, no human participation is required, and the posture of the detection object can be automatically detected. Through the preset posture standard, the posture detection result can be made more accurate, which solves the problem of inaccurate posture detection results in the existing posture detection method due to the influence of the staff's experience and subjective factors.

[0148] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0149] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, any one of the posture detection methods in the above embodiments is implemented.

[0150] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0151] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0152] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A posture detection method for endoscope inspection posture assessment, characterized in that: The method comprises: Acquire a front view image and a side view image of the detection object in the endoscope inspection scene; Determine the key point information of the human body and the visibility information of the key parts of the human body in the front view image, and the key point information of the human body in the side view image; Determine the posture of the detection object based on the human body key point information and the visibility information of the human body key parts in the front view image, and the human body key point information in the side view image; the posture of the detection object includes one or more of the head posture, body posture and limb posture of the detection object; The posture of the detected object is compared with a preset posture standard to obtain a posture detection result of the detected object.

2. The posture detection method according to claim 1, characterized in that: The determining of the key point information of the human body and the visibility information of the key parts of the human body in the front view image, and the key point information of the human body in the side view image, comprises: The trained key point detection model is used to perform key point detection on the front view image and the side view image respectively to obtain the human body key point information and the visibility information of the key parts of the human body in the front view image, as well as the human body key point information in the side view image.

3. The posture detection method according to any one of claims 1 or 2, characterized in that: The determining the posture of the detected object based on the human body key point information and the visibility information of the human body key parts in the front view image, and the human body key point information in the side view image, comprises: Based on the key point information of the human body in the frontal perspective image, determine the ratio of the shoulder width to the body length, the angle of the limbs, and the position information of the two shoulders of the human body in the frontal perspective image; Determine the angles of the limbs of the human body in the side perspective image based on the key point information of the human body in the side perspective image; Determine the head posture and body posture of the inspection object based on the ratio of the shoulder width to the body length of the human body in the frontal perspective image, the position information of the two shoulders, and the visibility information of the key parts of the human body; The postures of the limbs of the inspection object are determined based on the angles of the limbs of the human body in the front view image and the angles of the limbs of the human body in the side view image.

4. The posture detection method according to claim 3, characterized in that: Determining the angles of the four limbs comprises: Based on the human body key point information in the front view image or the side view image, determine the main key points of the limbs of the human body in the front view image or the side view image; the main key points of the limbs include at least the key points at the shoulder joints, the key points at the elbow joints, and the key points at the wrist joints of each arm, and the key points at the hip joints, the key points at the knee joints, and the key points at the ankle joints of each leg; Based on the main key points of the limbs of the human body in the front perspective image or the side perspective image, various angles of the limbs of the human body in the front perspective image or the side perspective image are determined.

5. The posture detection method according to claim 4, characterized in that: The determining, based on the main key points of the limbs of the human body in the frontal perspective image or the side perspective image, various angles of the limbs of the human body in the frontal perspective image or the side perspective image comprises: Based on the key points at the shoulder joints, the key points at the elbow joints, and the key points at the wrist joints of each arm of the human body in the front view image or the side view image, determine the first vector and the second vector of each arm of the human body in the front view image or the side view image; the first vector is a vector obtained by taking the key point at the elbow joint as the starting point and pointing to the key point at the shoulder joint, or a vector obtained by taking the key point at the shoulder joint as the starting point and pointing to the key point at the elbow joint; the second vector is a vector obtained by taking the key point at the elbow joint as the starting point and pointing to the key point at the wrist joint, or a vector obtained by taking the key point at the wrist joint as the starting point and pointing to the key point at the elbow joint; Determine each angle of each arm of the human body in the frontal perspective image or the side perspective image based on the first vector and the second vector of each arm of the human body in the frontal perspective image or the side perspective image; Based on the key points at the hip joints, the key points at the knee joints, and the key points at the ankle joints of each leg of the human body in the front view image or the side view image, determine the third vector and the fourth vector of each leg of the human body in the front view image or the side view image; the third vector is a vector obtained by taking the key point at the knee joint as the starting point and pointing to the key point at the hip joint, or a vector obtained by taking the key point at the hip joint as the starting point and pointing to the key point at the knee joint; the fourth vector is a vector obtained by taking the key point at the knee joint as the starting point and pointing to the key point at the ankle joint, or a vector obtained by taking the key point at the ankle joint as the starting point and pointing to the key point at the knee joint; Based on the third vector and the fourth vector of each leg of the human body in the front perspective image or the side perspective image, each angle of each leg of the human body in the front perspective image or the side perspective image is determined.

6. The posture detection method according to claim 2, characterized in that: Before performing key point detection on the front view image and the side view image using the trained key point detection model to obtain the human body key point information in the front view image and the human body key point information in the side view image, the method comprises: Acquire samples of front view images and side view images of a plurality of inspection objects in the endoscopic inspection scene to obtain a sample set; Annotating key parts of the human body for each image in the sample set to obtain an annotated sample set; Dividing the labeled sample set into a training set, a validation set, and a test set; Based on the training set, the validation set and the test set, the YOLOv8 pre-trained model is trained to obtain the trained key point detection model.

7. The posture detection method according to claim 1, characterized in that: The step of comparing the posture of the detected object with a preset posture standard to obtain a posture detection result of the detected object includes: Based on the head posture, body posture and limb posture of the detection object, as well as preset human head posture standards, human body posture standards and human limb posture standards, posture detection is performed on the detection object to obtain the posture detection result of the detection object; the posture detection result includes standard posture and non-standard posture.

8. The posture detection method according to claim 7, characterized in that: After comparing the posture of the detected object with a preset posture standard to obtain a posture detection result of the detected object, the method includes: When the posture detection result of the detection object is that the posture is not standard, a posture error reminder is issued.

9. A posture detection device for evaluating body position during endoscope inspection, characterized in that: The device comprises: An image acquisition module, used to acquire a front view image and a side view image of the detection object in the endoscopic inspection scene; An information determination module, used to determine the key point information of the human body and the visibility information of the key parts of the human body in the front view image, and the key point information of the human body in the side view image; a posture determination module, configured to determine the posture of the detection object based on the human body key point information and the visibility information of the human body key parts in the front view image, and the human body key point information in the side view image; the posture of the detection object is one or more of the head posture, body posture and limb posture of the detection object; And a detection module is used to compare the posture of the detection object with a preset posture to obtain a posture detection result of the detection object.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the gesture detection method according to any one of claims 1 to 8 are implemented.

Images