Back recognition method and device based on human body image and electronic equipment

Abstract

The invention discloses a back recognition method and device based on a human body image, electronic equipment, a storage medium and a massage robot. The method comprises the steps of determining human body key points in a human body image, and determining a human body region in the human body image according to the human body key points; determining a shoulder boundary point according to the shoulder key point and a first connecting line of the neck key point and the hip joint key point; determining an armpit point according to an included angle formed between the shoulder key point and a connecting line between the elbow key point and the hipbone key point and a distance between the shoulder key point and the hipbone key point; determining a chest point and a hip point according to the shoulder key points, the hipbone key points, the armpit points and the first connecting line; and determining a back area in the human body area according to the shoulder boundary point, the armpit point, the chest point and the hip point. By using the scheme provided by the invention, the back area can be simply and efficiently determined in the human body image, and the determined back area is high in precision.

Description

Technical Field

[0001] The present application relates to the field of digital image processing, and more particularly to a back recognition method, device, electronic device and computer-readable storage medium based on human body images. In addition, the present application also relates to a massage robot. Background Art

[0002] As an important part of the human trunk, the back area of ​​the human body is an important area covering the human nervous system, meridian system, and immune system. Therefore, applications based on the back area of ​​the human body are widespread in real life. For example, since there are many acupoints such as the Governor Vessel and the Foot-Taiyang Bladder Meridian distributed in the back area of ​​the human body, local massage through the back area of ​​the human body can be seen everywhere in daily life. During the above-mentioned local massage process, pressure can be applied periodically according to the identified back area of ​​the human body, so as to achieve the effect of relieving back acupoints and muscle fatigue. Since the back area of ​​the human body covers the shoulders, back, waist, buttocks, spine, and other parts of the human body, fitness enthusiasts can perform local strength training based on the back area of ​​the human body in daily fitness activities. From the above description, it can be seen that whether it is a local massage of the back of the human body or strength training of the back of the human body, the recognition and acquisition of the back area of ​​the human body is crucial. Usually, the recognition and acquisition of the back area of ​​the human body is completed based on the human body image.

[0003] Currently, the methods for identifying and acquiring the back area of ​​the human body based on human body images are mainly divided into two categories: deep learning methods and traditional computing methods.

[0004] As a method of obtaining the back area of ​​the human body with the help of machine models, the deep learning method collects a large amount of human image annotation data, inputs the above images into the machine model to carry out end-to-end back segmentation network training, and then identifies the back area of ​​the human body. However, this method has the defects of large machine model scale, large amount of data required for training, and cumbersome and complicated process of collecting human image annotation data. The traditional calculation method is to directly calculate the area where the back of the human body is located from the image based on the difference between human skin color and environment. In actual application, the logical rules used in this method are complex and highly limited, and there are obvious application limitations. Therefore, it is an urgent problem to provide a simple and effective back area recognition method based on human images, which has a wide range of applications and high accuracy in determining the back area. Summary of the invention

[0005] In view of this, the present application provides a back recognition method and device based on human body images, a massage robot, an electronic device and a computer-readable storage medium with a back recognition device based on human body images, which can simply and effectively identify and obtain the back area of ​​the human body in the human body image. The determined back area has high accuracy and the scheme is not limited by the human skin color, and has a wide range of applications.

[0006] A first aspect of an embodiment of the present application provides a back recognition method based on a human body image, the method comprising:

[0007] Determine the key points of a human body in a human body image, and determine the human body region in the human body image according to the key points of the human body; the key points of the human body include: shoulder key points, neck key points, hip joint key points, elbow key points and hip bone key points;

[0008] Determine a shoulder boundary point according to the shoulder key point and a first connecting line between the neck key point and the hip joint key point;

[0009] Determine the armpit point according to the angle formed by the connecting lines of the shoulder key point, the elbow key point and the hip key point, and the distance between the shoulder key point and the hip key point;

[0010] Determine a chest point and a hip point according to the shoulder key point, the hip key point, the armpit point, and the first connecting line;

[0011] A back region is determined in the human body region based on the shoulder boundary point, the armpit point, the thorax point, and the hip point.

[0012] A second aspect of the embodiment of the present application further provides a back recognition device based on a human body image, the device comprising:

[0013] A human body key point determination unit, used to determine the human body key points in the human body image, and determine the human body area in the human body image according to the human body key points; the human body key points include: shoulder key points, neck key points, hip joint key points, elbow key points and hip bone key points;

[0014] A shoulder boundary point determination unit, configured to determine the shoulder boundary point according to the shoulder key point and a first connecting line between the neck key point and the hip joint key point;

[0015] an axillary point determination unit, configured to determine the axillary point according to the angle formed by the connecting lines of the shoulder key point, the elbow key point and the hip key point, and the distance between the shoulder key point and the hip key point;

[0016] A chest point and hip point determination unit, used to determine a chest point and a hip point according to the shoulder key point, the hip key point, the armpit point, and the first connecting line;

[0017] A back region determining unit is used for the shoulder boundary point, the armpit point, the chest point and the hip point to determine a back region in the human body region.

[0018] A third aspect of an embodiment of the present application provides a massage robot, comprising a back recognition device based on a human body image.

[0019] A fourth aspect of the embodiments of the present application further provides an electronic device, comprising: a processor, a memory, and computer program instructions stored in the memory and executable on the processor;

[0020] When the processor executes the computer program instructions, the back recognition method based on human body images described in the first aspect is implemented.

[0021] The fifth aspect of the embodiments of the present application also provides a computer-readable storage medium on which one or more computer instructions are stored, characterized in that the instructions are executed by a processor to implement the back recognition method based on human body images described in any of the above technical solutions.

[0022] Compared with the prior art, this application has the following advantages:

[0023] The back recognition method based on human images provided in the embodiment of the present application determines the human key points in the human image in the process of determining the human back area based on the human image, and determines the human area in the human image according to the human key points; the human key points include: shoulder key points, neck key points, hip joint key points, elbow key points and hip key points; the shoulder boundary point is determined according to the shoulder key points and the first connecting line between the neck key points and the hip key points; the armpit point is determined according to the angle formed by the connecting lines of the shoulder key points, the elbow key points and the hip key points, and the distance between the shoulder key points and the hip key points; the chest point and the hip point are determined according to the shoulder key points, the hip key points, the armpit point, and the first connecting line; the back area is determined in the human area according to the shoulder boundary point, the armpit point, the chest point and the hip point. The scheme is simple and efficient in determining the back area of ​​the human body, and the determined back area has high accuracy; because the process does not rely on the judgment and processing of the human skin color, it has a wide range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 1 is a flow chart of a back recognition method based on human body images provided in an embodiment of the present application;

[0025] Figure 2 is a schematic diagram of a corresponding image of a human body in a prone position provided in an embodiment of the present application;

[0026] Figure 3 is a schematic diagram of human body key points obtained by processing the human body key point recognition model provided in an embodiment of the present application;

[0027] Figure 4 is a schematic diagram of a human body preprocessing image provided in an embodiment of the present application;

[0028] Figure 5 is a schematic diagram of a human body region in a human body image provided by an embodiment of the present application;

[0029] Figure 6 is a schematic diagram of determining the left acromion point according to the left shoulder key points provided in an embodiment of the present application;

[0030] Figure 7 It is a schematic diagram of determining the left shoulder boundary point and the right shoulder boundary point provided in an embodiment of the present application.

[0031] Figure 8 This is a schematic diagram of the second intersection after movement provided in an embodiment of the present application.

[0032] Fig. 9 It is a schematic diagram of determining the left axillary point provided in an embodiment of the present application.

[0033] Fig.10 It is a schematic diagram of determining the left chest point provided in an embodiment of the present application.

[0034] Fig.11 It is a schematic diagram of determining the left hip point provided in an embodiment of the present application.

[0035] Fig.12 It is a schematic diagram of the back area provided in an embodiment of the present application.

[0036] Fig.13 It is a schematic diagram of a back recognition device based on human body images provided in an embodiment of the present application.

[0037] Fig.14 It is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0039] The terms "a", "an", "the" and "said" are used in this specification to indicate the presence of one or more elements / components / etc.; the terms "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first" and "second" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0040] It should be understood that in the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" is merely a way to describe the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "Including A, B and / or C" means including any one, any two, or any three of A, B, and C.

[0041] It should be understood that in the embodiments of the present application, "B corresponding to A", "B corresponding to A", "A corresponds to B", or "B corresponds to A" means that B is associated with A, and B can be determined according to A. Determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0042] Before describing the implementation modes of the present application in detail, for ease of understanding, the terms involved in the embodiments of the present application are explained below.

[0043] Depth images, also known as RGB-D images, include RGB images and Depth images; RGB images are ordinary RGB three-channel color images, and Depth images are depth images, similar to grayscale images. Each pixel value in the image is the actual distance between the sensor and the object. Usually, there is a one-to-one correspondence between the pixels of the RGB image and the Depth image.

[0044] Point cloud, as a data set of points in a coordinate system, is a massive collection of points of target surface characteristics, which contains rich information, including three-dimensional coordinates X, Y, Z, color, classification value, intensity value, time, etc. When these points are combined together, a point cloud is formed, which is a collection of data points representing a 3D shape or object in space. Point cloud is a true expression of physical time in a digital way. The denser the point cloud, the more image details and information it reflects.

[0045] The human key point recognition model is mainly used to detect the key point positions of the human body from the input image. By shooting a large amount of data containing the human body, manually marking the positions of the key points, and inputting them into the model for training, the model can output the coordinate positions of several key points in the image by inputting an RGB image.

[0046] In the existing process of obtaining the corresponding image of the back area of ​​the human body based on the human body image, deep learning methods and traditional computing methods are usually used. As a method of identifying and obtaining the back area of ​​the human body with the help of a machine model, the deep learning method collects a large number of human body images and annotates the data according to the needs of the human body images, inputs the above images into the machine model to carry out end-to-end back segmentation network training, and then identifies and obtains the back area of ​​the human body. However, this method has the defects of large machine model scale, large amount of data required for training, and cumbersome and complicated process of collecting and annotating human body image data. The traditional computing method is a method of directly calculating the area where the back of the human body is located from the image based on the difference between the human skin color and the environment. In the actual operation process, the logical rules used in this method are complex and highly limited, and there are obvious application limitations.

[0047] In view of the above problems existing in the prior art, the present application provides a back recognition method, device, massage robot, electronic device and storage medium based on human body image. For ease of understanding, the application scenarios of the method embodiments of the present application are illustrated by way of example.

[0048] During the operation of the massage robot, it is often necessary to take a photo of the human back in order to accurately identify the human back area in the image. Accurate recognition of the back area based on the human body image has a relatively important impact on the operation of the massage robot. By identifying the determined human back area, the massage robot's mechanical arm is guided to simulate the human hand to perform precise massage in the corresponding area of ​​the real human body corresponding to the area, so as to periodically apply pressure to the identified human back, thereby relieving muscle fatigue. It can be seen that it is particularly critical to accurately identify the human back area based on the taken human body photos. In order to improve the recognition accuracy of the human back area, and then improve the massage effect of the massage robot, the present application provides a back recognition method based on human body images. However, it is easy for those skilled in the art to understand that the above application scenarios are only used for examples, and are not limited to this in this exemplary embodiment.

[0049] It should be understood that the execution subject of the back recognition method based on human body images provided by the present application can be an electronic device, and the electronic device can be a desktop computer, a laptop computer, a tablet computer, a mobile phone, a massage device, a massage robot, etc., or an electronic device with other data processing functions, which is not specifically limited by the present application. When the electronic device is a massage device, such as a massage robot, the electronic device can specifically be a processor with data processing capabilities in the massage device or the massage robot.

[0050] The following combination Figure 1 , introduces the overall situation of a back recognition method based on human body images provided in an embodiment of the present application. Figure 1 FIG. 1 is a schematic flow chart of a back recognition method based on a human body image provided in an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0051] Step S101, determining the key points of a human body in a human body image, and determining the human body area in the human body image according to the key points of the human body; the key points of the human body include: shoulder key points, neck key points, hip joint key points, elbow key points and hip bone key points.

[0052] This step is used to determine the key points of the human body in the human body image and the human body area in the human body image.

[0053] In this embodiment, the above human body image is an image corresponding to a human body in a prone position. Figure 2 The hint, Figure 2 A schematic diagram of the corresponding image of a human body in a prone posture provided in this embodiment.

[0054] like Figure 2 As shown in FIG. 1 , the human body in the figure is in a prone position facing the bed surface, and the back of the human body is directly displayed in the image. The figure also shows the bed surface, the ground, and the top of the head, limbs, and other parts of the human body other than the back.

[0055] In specific implementation, the above-mentioned human body image format type for back recognition is RGB-D type. As mentioned above, the RGB-D type image includes both RGB image and Depth image. In this embodiment, by setting a two-dimensional coordinate system in the RGB-D image, including X-axis coordinates and Y-axis coordinates, the pixels at different positions of the image correspond to different coordinates, thereby realizing the one-to-one determination of the pixels in the image. For example, Figure 2 The coordinates of the first pixel corresponding to the upper left corner of the RGB-D image are defined as (0, 0), the coordinates of the 100th pixel corresponding to the lower right corner of the image are defined as (100, 100), and so on.

[0056] In this embodiment, determining the key points of a human body in a human body image includes: inputting the human body image into a human body key point recognition model to determine the key points of a human body in the human body image.

[0057] During specific implementation, the above-mentioned RGB-D type human image is input into the human key point recognition model. The image can obtain the human key points after being processed by the model, which can also be understood as obtaining the coordinates corresponding to the pixel points at the human key points. In this embodiment, the human back key points identified by the human key point recognition model include: shoulder key points, neck key points, hip joint key points, elbow key points and hip bone key points. The human key point recognition model is a model obtained through training with a large amount of image data, and can be directly used in this embodiment.

[0058] In one embodiment, the human back key points identified by the human key point recognition model, in addition to the above-mentioned shoulder key points, neck key points, hip joint key points, elbow key points and hip bone key points, also include: top of the head key points, chest key points, wrist key points, knee key points, ankle key points, etc.

[0059] It should be understood that the human body key points obtained by the above-mentioned human body key point recognition model are not essentially the locations of the real human skeleton points in the image, but the model makes judgments and processes based on the image data information, and defines the pixel points at the coordinate position in the human body image as human body key points, such as neck key points.

[0060] In order to make the description concise and easy to understand in the embodiments of the present application, the human body key points obtained by processing the above-mentioned human body key point recognition model are illustrated in the form of drawings. Figure 3 Schematic diagram of human key points obtained by processing the human key point recognition model.

[0061] like Figure 3 As shown, the human body key points obtained by the above-mentioned model recognition process are marked with the symbol "☆" in the human body image of the RGB-D type. As a kind of indication, in the pixel point of the figure, the coordinates of the neck key point A point are (50, 20), the coordinates of the left shoulder key point B point are (40, 30), the coordinates of the right shoulder key point C point are (60, 25), the coordinates of the left elbow key point D point are (5, 40), the coordinates of the right elbow key point E point are (95, 45), the coordinates of the left hip bone key point F point are (45, 70), the coordinates of the right hip bone key point G point are (55, 72), the coordinates of the hip joint key point H point are (50, 70), and the coordinates of the top of the head key point I point are (50, 10). In the subsequent steps of the present embodiment, the human body key points involved all refer to the coordinates in the human body image, and there is a one-to-one correspondence between the coordinates and the pixel points of the human body image.

[0062] The coordinates of the key points of the human body in the human body image can be understood as the position of the pixel corresponding to the key point in the RGB-D image. It is easy to understand that when there are differences in the images input into the human body key point recognition model, the coordinates of the key points of the human body obtained by the model processing will also be different. For example, when human body image 1 is input into the human body key point recognition model, the coordinates of the neck key point obtained by the model processing are (50, 20); when human body image 2 is input into the human body key point recognition model, the coordinates of the neck key point obtained by the model processing are (55, 10).

[0063] In one implementation, in the above step S101, determining the human body region in the human body image according to the human body key points may be performed according to the following steps S101-1 to S101-3:

[0064] Step S101-1: generating a human body preprocessing image in the human body image according to the human body key points;

[0065] Step S101-2: performing plane detection on the preprocessed image to determine plane depth data;

[0066] Step S101 - 3 : determining a human body region in the human body image according to the depth data of the human body image and the planar depth data.

[0067] In one embodiment, the human key points obtained by the human key point recognition model include: shoulder key points, neck key points, hip joint key points, elbow key points and hip bone key points as image drawing key points, and a human preprocessing image is drawn. The human preprocessing image can be understood as a rough human region mask image for the human image.

[0068] In another embodiment, shoulder key points, neck key points, hip joint key points, elbow key points, hip bone key points, top of the head key points, chest key points, wrist key points, knee key points, and ankle key points can be used as image drawing key points to obtain a rough human body area mask image based on the above human body image.

[0069] To understand the above human body preprocessing image (rough human body area mask image), please refer to Figure 4 's hint. Figure 4 Schematic diagram of preprocessing images of the human body.

[0070] like Figure 4As shown, the human body preprocessing image (rough human body area mask image) includes the human body trunk and arm parts. In specific implementation, the approximate area of ​​the human body is determined by connecting the key point A of the neck, the key point B of the left shoulder, the key point C of the right shoulder, the key point D of the left elbow, the key point E of the right elbow, the key point F of the left hip bone, the key point G of the right hip bone, the key point H of the hip joint, and the key point I of the top of the head; in the RGB-D type human body image, the pixel points in other areas outside the above approximate area of ​​the human body are masked, and then the human body preprocessing image containing a small amount of bed surface and ground is obtained. The above human body preprocessing image is composed of point clouds.

[0071] It should be understood that the above human body preprocessing image (rough human body region mask image) is an indirect source for obtaining the human body region. By further processing the generated human body preprocessing image, the human body region can be finally obtained.

[0072] In specific implementation, plane detection is performed on the point cloud corresponding to the human body preprocessing image, and a plane whose distance from the human body is less than a preset threshold is selected in the human body preprocessing image, and it is determined as the plane where the human body is located; such as the bed surface plane corresponding to the human body in the prone posture, and then the plane depth data of the plane where the human body is located is obtained; according to the above plane depth data and the depth data of the human body image of the RGB-D type, in the human body preprocessing image, the depth data of the human body preprocessing image greater than the value corresponding to the above plane depth data is removed with the plane depth data as the boundary, and then the human body area is obtained. Since the human body is in a prone posture facing the bed surface, the human body and the bed surface can be easily distinguished according to the point cloud.

[0073] To understand the human body regions, refer to Figure 5 's hint. Figure 5 is a schematic diagram of the human body area in a human body image. Figure 5 As shown, in the process of obtaining the human body region by processing the human body preprocessing image, in the pixel points of the human body preprocessing image, Figure 4 The human body preprocessing image shown is subjected to plane detection to determine the plane depth data. According to the plane depth data, the pixels on the bed surface and below the bed surface are masked to obtain the human body area (precise human body area mask image). For example, the human body preprocessing image is used as an RGB-D image, and the depth data of the human body back plane and the plane depth data of the bed surface where the human body is located are determined by obtaining the depth data in the Depth image. According to the bed surface plane depth data, the pixels on the bed surface and below the bed surface are masked to obtain the human body area (precise human body area mask image).

[0074] Step S102, determining the shoulder boundary point according to the shoulder key point and the first connecting line between the neck key point and the hip joint key point.

[0075] This step is used to determine the shoulder boundary point. In this embodiment, there are two ways to determine the shoulder boundary point based on the shoulder key point, so they are introduced separately. It can also be understood that there are two forms of shoulder boundary points, the first is the shoulder boundary point corresponding to the first intersection point, and the second is the shoulder boundary point corresponding to the first boundary point and the second boundary point.

[0076] In one embodiment, determining the shoulder boundary point according to the shoulder key point and the first connecting line between the neck key point and the hip joint key point includes:

[0077] generating a first ray extending from the shoulder key point along a specified direction and parallel to the first connecting line;

[0078] Determining a first intersection point in the human body region according to an intersection point between the first ray and the human body region;

[0079] The first intersection point is determined as the shoulder boundary point.

[0080] The specified direction refers to the direction toward the position of the head in the human body image, and can also be understood as the direction toward the position of the key point on the top of the head.

[0081] In order to facilitate understanding of the above-mentioned first method of determining the shoulder boundary point, the left shoulder boundary point is determined by taking the left shoulder key point as an example for explanation. In the image corresponding to the human body area, a first ray is generated that extends from the left shoulder key point B along the direction of the top of the head key point I (specified direction) and is parallel to the first connecting line; the first connecting line is a connecting line determined based on the neck key point A and the hip joint key point H; the first ray intersects with the human body boundary of the human body area (precise human body area mask image), and the first intersection point where the first ray intersects with the human body area boundary is defined as the left acromion point (left shoulder boundary point). Similarly, the right acromion point (right shoulder boundary point) can be determined based on the right shoulder key point C.

[0082] For ease of understanding, refer to Figure 6 's hint. Figure 6 The figure is a schematic diagram for determining the left shoulder peak point based on the left shoulder key points. Figure 6As shown, in the image corresponding to the human body area, the neck key point A and the hip joint key point H are connected, and the line L1 between the above two key points is used as the first line representing the spine line. A first ray S1 parallel to the first line L1 is made through the left shoulder key point B, and the direction of the first ray S1 is toward the key point I on the top of the head. The first ray S1 intersects with the boundary of the human body area at point P1, and point P1 is defined as the left acromion point (left shoulder boundary point). Similarly, the right acromion point P2 (right shoulder boundary point) is obtained.

[0083] In another embodiment, determining the shoulder boundary point according to the shoulder key point and the first connecting line between the neck key point and the hip joint key point includes:

[0084] Obtain a shoulder key point connection line of the shoulder key point and a hip key point connection line of the hip key point;

[0085] Determine a second intersection point according to the intersection point of the shoulder key point connection line and the first connection line;

[0086] Generate a first parallel line extending from the second intersection point and parallel to the line connecting the hip bone key points;

[0087] Determine the intersection point of the first parallel line and the human body region as a boundary intersection point;

[0088] In response to the distance value of the line corresponding to the boundary intersection satisfying a preset distance condition, the boundary intersection is determined as a first boundary point and a second boundary point; in response to the distance value of the line corresponding to the boundary intersection not satisfying the preset distance condition, the second intersection is moved along a predetermined direction of the first line, and the boundary intersection follows the movement of the second intersection until the distance value of the line corresponding to the boundary intersection satisfies the preset distance condition.

[0089] The first boundary point and the second boundary point that meet the preset distance condition are determined as the shoulder boundary points.

[0090] In this embodiment, in response to the distance value of the line corresponding to the boundary intersection point satisfying a preset distance condition, determining the boundary intersection point as the first boundary point and the second boundary point includes:

[0091] Obtain the distance value of the shoulder key point connection line;

[0092] In response to the fact that the distance value of the line corresponding to the boundary intersection point does not exceed the distance value of the line connecting the shoulder key points, the boundary intersection point is determined as the first boundary point and the second boundary point.

[0093] In order to facilitate understanding of the second method of determining the shoulder boundary points, the left shoulder boundary points and the right shoulder boundary points are determined by using the left shoulder key points and the right shoulder key points as an example. Figure 7 's hint. Figure 7 The figure is a schematic diagram for determining the left shoulder boundary point and the right shoulder boundary point based on the left shoulder key point and the right shoulder key point.

[0094] like Figure 7 As shown, the left shoulder key point B is connected with the right shoulder key point C to obtain the shoulder key point connection line L2, and the length value of the shoulder key point connection line L2 is obtained; the left hip key point F is connected with the right hip key point G to obtain the hip key point connection line L3; the first connection line L1 obtained in the above steps intersects with the shoulder key point connection line L2 at the second intersection point a; a parallel line L4 (first parallel line) extending from the second intersection point a and parallel to the hip key point connection line L3 is generated; the parallel line L4 is at Figure 7 It is represented by a dotted line in the figure; the parallel line L4 intersects with the human body boundary of the human body area at the boundary intersection points b and c; the corresponding length value of the line L4 connecting the boundary intersection points b and c is obtained.

[0095] When judging whether the corresponding length of the line L4 connecting the boundary intersection points b and c meets the preset distance condition:

[0096] If the corresponding length of the line L4 (the corresponding line of the boundary intersection) is ≤ the length value of m*L2, the boundary intersection points b and c are determined as the first boundary point P1' and the second boundary point P2'. At the same time, the second intersection point a can also be defined as the seventh cervical vertebra point P5. In other words, when the corresponding length of the line L4 of the boundary intersection points b and c is less than or equal to the length value of the threshold m*L2 determined according to the shoulder key point line L2, the boundary intersection points b and c determined as the first boundary point P1' and the second boundary point P2' can be understood as the boundary points between the neck and shoulders of the human body in the image.

[0097] If the corresponding length of the line L4 (the line corresponding to the boundary intersection) is greater than the length value of m*L2, the second intersection point a is moved n pixels on the first line L1 toward the neck key point A (predetermined direction), that is, the coordinates of the second intersection point a are moved n coordinate points along the first line L1 toward the neck key point A; repeat the above judgment process until the preset distance condition is met.

[0098] In the above process, m is a coefficient value less than 1. It should be understood that when the second intersection point a is moved on the first line L1 toward the neck key point A, the line L4 where the second intersection point a is located moves accordingly. At the same time, the intersection point of the line L4 and the boundary of the human body area changes, and b is obtained. ’ dot and c ’Point; calculate b ’ dot and c ’ The corresponding length of the line connecting the two points is L4 ’ , L4 ’ The corresponding length of the neck key point A is repeatedly compared with the preset distance condition until it is determined that the first boundary point P1' and the second boundary point P2' correspond to the left shoulder boundary point and the right shoulder boundary point. ’ dot and c ’ For the process of clicking, please refer to Figure 8 The hint, Figure 8 This is a schematic diagram after the second intersection point a is moved. Figure 8 As shown, as the second intersection point a moves, the intersection point of the line L4 and the boundary of the human body area moves from point b to point b. ’ Point c moves to point c ’ point, and then regain the connection L4 ’ The length of the data gradually becomes smaller.

[0099] It should be understood that in the embodiment of the present application, there are two types of shoulder boundary points determined: the first type, the shoulder boundary points corresponding to the left acromion point P1 and the right acromion point P2; the second type, the shoulder boundary points corresponding to the first boundary point P1' and the second boundary point P2'. The above-mentioned different types of shoulder boundary points are used to determine the back area in the human body area in the subsequent steps. It should be understood that in actual application, due to differences in human body shape and posture, differences in human prone posture, such as hunchback, shrugging, side-lying, and well-developed trapezius muscles in some groups, the above-mentioned two types of shoulder boundary points determined in different ways are used to determine the back area of ​​the human body, and their application also has differences.

[0100] Step S103, determining the armpit point according to the angle formed by the connecting lines of the shoulder key point, the elbow key point and the hip key point, and the distance between the shoulder key point and the hip key point.

[0101] This step is used to determine the axillary point.

[0102] In this embodiment, in the above step S103, the axillary point is determined according to the angle formed by the connecting lines of the shoulder key point, the elbow key point and the hip key point, and the distance between the shoulder key point and the hip key point, and the following steps S103-1 to S103-5 can be performed:

[0103] Step S103-1: Generate a first vector extending from the shoulder key point to the elbow key point;

[0104] Step S103-2: Generate a second vector extending from the shoulder key point to the hip key point;

[0105] Step S103-3: generating a vector angle corresponding to the shoulder key point according to the first vector and the second vector;

[0106] Step S103-4: determining a second ray within the vector angle according to the angle of the vector angle;

[0107] Step S103-5: Determine the armpit point on the second ray according to the distance between the shoulder key point and the hip key point.

[0108] Wherein, in step S103-4, determining the second ray within the vector angle according to the angle of the vector angle includes:

[0109] Determine the proportional value corresponding to the vector angle;

[0110] According to the ratio value, a second ray with the shoulder key point as a vertex is determined on a side close to the second vector within the vector angle.

[0111] In order to facilitate the understanding of the process of obtaining the axillary point, the determination of the left axillary point is taken as an example. Fig. 9 The hint, Fig. 9 This is a schematic diagram for determining the left axillary point. Fig. 9 As shown, generate a first vector V1 extending from the left shoulder key point B to the left elbow key point D; generate a second vector V2 extending from the left shoulder key point B to the left hip bone key point F; at the left shoulder key point B, generate a corresponding vector angle θ1 according to the first vector V1 and the second vector V2; obtain the angle of the vector angle θ1, and determine the second ray S2 according to the angle of the vector angle θ1 within the vector angle θ1; obtain the length of the corresponding line L5 from the left shoulder key point B to the left hip bone key point F; on the second ray S2, determine the left armpit point P3 according to the length of the line L5. Similarly, obtain the right armpit point P4.

[0112] In specific implementation, the corresponding proportional value is determined according to the angle of the vector angle θ1. As shown in the figure, if the angle of the vector angle θ1 is an acute angle, the corresponding proportional value is 1 / 4. If the angle of the vector angle θ1 is an obtuse angle, the corresponding proportional value may be greater than 1 / 4.

[0113] The second ray S2 is set within the vector angle θ1, at 1 / 4 of the angle of the vector angle θ1, and close to the position of the second vector V2. When determining the left armpit point P3 according to the length of the connecting line L5, the point on the second ray S2 that is 1 / 4*L5 length away from the left shoulder key point B is defined as the left armpit point P3. Similarly, the right armpit point P4 is obtained. Of course, for the angle of the vector angle θ1, the proportional value 1 / 4 corresponding to the angle of the above vector angle θ1 can also be adjusted according to actual needs. For ease of understanding, in Fig. 9 In the illustration, when the angle of the vector angle θ1 is less than or equal to 90°, the proportional value corresponding to the vector angle θ1 is determined to be 1 / 4. If the angle of the vector angle θ1 is greater than 90°, the proportional value can be selected between 1 / 4 and 1 / 2. At the same time, when the axillary point is determined according to the length of L5 on the second ray S2, the proportional value in the 1 / 4*L5 length can also be adjusted according to actual conditions. This embodiment is an illustration and is not actually limited.

[0114] Step S104, determining the chest point and the hip point according to the shoulder key point, the hip key point, the armpit point, and the first connecting line.

[0115] This step is used to determine the ribcage point and hip point.

[0116] In this embodiment, in the above step S104, the chest point and the hip point are determined according to the shoulder key point, the hip key point, the armpit point, and the first connecting line, and the following steps S104-1 to S104-6 can be performed:

[0117] Step S104-1: generating a third ray extending from the hip key point along the specified direction and parallel to the first connecting line;

[0118] Step S104-2: determining a first middle point on the third ray according to the distance value of the first connecting line;

[0119] Step S104-3: generating a vertical line extending from the armpit point along a second specified direction and perpendicular to a line connecting the shoulder key point and the first middle point;

[0120] Step S104-4: determining a second middle point according to the intersection of the vertical line and the line connecting the shoulder key point and the first middle point;

[0121] Step S104-5: constructing a parallelogram by using the first middle point, the second middle point and the axillary point as vertices of the parallelogram to determine the thoracic point;

[0122] Step S104-6: construct a parallelogram using the hip key point, the first middle point and the chest point as the vertices of the parallelogram to determine the hip point.

[0123] Wherein, in step S104-2, determining the first intermediate point on the third ray according to the distance value of the first connecting line includes:

[0124] Determine, on the first connecting line, a distance value of a midpoint of the first connecting line according to the distance value of the first connecting line;

[0125] The first middle point is determined on the third ray according to the distance value of the midpoints of the first connecting line.

[0126] In order to facilitate understanding of the determination process of the chest point and the hip point, the determination process of the left chest point and the left hip point is taken as an example for explanation. Since the determination process of the left hip point depends on the determination of the left chest point, the determination process of the left chest point is first described in detail.

[0127] refer to Fig.10 The hint, Fig.10 A schematic diagram for determining the left thoracic point provided in an embodiment of the present application. As shown in the figure, the first line L1 between the neck key point A and the hip joint key point H is obtained; the midpoint A1 of the first line L1 is defined as the ninth thoracic vertebra position point, and the distance value between the ninth thoracic vertebra position point A1 and the hip joint key point H is obtained (the first line midpoint distance value), and the distance value is 1 / 2 of the length of the first line L1. The distance value of the above-mentioned first line L1 is represented by 1 / 2*L1. A third ray S3 parallel to the first line L1 is generated through the left hip key point F; the third ray S3 extends in the direction of the top of the head key point I (specified direction); on the third ray S3, a point with a distance of 1 / 2*L1 from the left hip key point F is taken, and it is marked as the first intermediate point d. Connect the left shoulder key point B and the first middle point d, and mark the line as L6; generate a vertical line extending from the left armpit point P3 along the neck key point A (the second specified direction) and perpendicular to the line L6, and mark the intersection of the vertical line and the line L6 as the second middle point e. Use the first middle point d, the second middle point e, and the left armpit point P3 as the vertices of the parallelogram to construct the parallelogram de-P3; determine the other vertex of the parallelogram de-P3 as the left chest point P6. Similarly, obtain the right chest point P7.

[0128] After the left chest point P6 is determined, the left hip point can be determined. Fig.11 The hint, Fig.11This is a schematic diagram of determining the left hip point provided by an embodiment of the present application. As shown in the figure, the left hip key point F, the first middle point d, and the left chest point P6 are used as the vertices of the parallelogram to construct a parallelogram Fd-P6; another vertex of the parallelogram Fd-P6 is determined as the left hip point P8. Similarly, the right hip point P9 is obtained.

[0129] Step S105, determining a back region in the human body region according to the shoulder boundary point, the armpit point, the chest point and the hip point.

[0130] This step is used to determine the back region in the human body region.

[0131] In a first embodiment, determining the back region in the human body region according to the shoulder boundary point, the armpit point, the chest point and the hip point includes:

[0132] A back region is determined in the human body region based on the first intersection point, the axilla point, the thorax point, and the hip point.

[0133] In the specific implementation, in the human body area, the coordinate points corresponding to the left acromion point P1, the left axilla point P3, the left chest point P6, the left hip point P8, the right hip point P9, the right chest point P7, the right axilla point P4, and the right acromion point P2 are connected by polygonal lines to determine Fig.12 The first human back area is shown in the blank area in the figure. It should be understood that when the processor based on the back recognition of the human body image determines that the human body shape posture and the human body prone posture in the human body image show one or more of the situations of hunchback, shrugging shoulders, lying on the side, etc., the back area (first human back area) can be obtained by connecting the left acromion point P1 and the right acromion point P2 as the boundary of the human back area, combined with other determined boundary points, such as the left axilla point P3, etc. For example, in the human body area (precise human body area mask image), the coordinate points corresponding to the left acromion point P1, the left axilla point P3, the left chest point P6, the left hip point P8, the right hip point P9, the right chest point P7, the right axilla point P4, and the right acromion point P2 are connected by polygonal lines, and then a polygonal area corresponding to P1-P3-P6-P8-P9-P7-P4-P2-P1 is determined. The pixel points corresponding to the above polygonal areas are masked in the precise human body area mask image to obtain Fig.12 The human back region (first human back region) is shown in the blank area.

[0134] In a second embodiment, determining the back region in the human body region according to the shoulder boundary point, the armpit point, the chest point and the hip point includes:

[0135] A back region is determined in the human body region according to the first boundary point, the second boundary point, the armpit point, the chest point, and the hip point.

[0136] During specific implementation, in the human body area, the first boundary point P1', the left axillary point P3, the left chest point P6, the left hip point P8, the right hip point P9, the right chest point P7, the right axillary point P4, and the second boundary point P2' are connected by polygonal lines to determine the second human back area. This situation is applied to the human body shape and posture and the human prone posture in the human body image, and the above-mentioned hunchback, shrugging, and side-lying situations do not occur. By using the line connecting the first boundary point P1' and the second boundary point P2' as the preset boundary line of the human body area, the pixel points above the preset boundary line in the human body area are masked to obtain the preset human body area. It should be understood that the first boundary point P1' and the second boundary point P2' are used as the boundary points between the neck and shoulders of the human body in the image. Masking the pixel points above the preset boundary line can achieve the removal of the pixel points corresponding to the neck and head. Furthermore, on the basis of obtaining the preset human body region, it is also necessary to perform mask processing on the pixels above the line connecting the left acromion point P1 and the right acromion point P2 in the preset human body region according to the line connecting the left acromion point P1 and the right acromion point P2, so as to accurately determine the back region in the human body region. The above process can also be understood as first performing image mask processing according to the preset boundary line of the first boundary point P1' and the second boundary point P2', and then performing secondary image mask processing according to the line connecting the left acromion point P1 and the right acromion point P2, so as to obtain the back region process in the human body region.

[0137] For ease of understanding, the second implementation method described above is illustrated by way of example. In the human body region (precise human body region mask image), the coordinate points corresponding to the first boundary point P1', the left axilla point P3, the left chest point P6, the left hip point P8, the right hip point P9, the right chest point P7, the right axilla point P4, and the second boundary point P2' are connected by polygonal lines, thereby determining a polygonal region corresponding to P1'-P3-P6-P8-P9-P7-P4-P2'-P1'; the pixel points of the polygonal region are masked in the precise human body region mask image to obtain a preset human body region; on the basis of obtaining the preset human body region, the coordinate points corresponding to the left acromion point P1 and the right acromion point P2 are connected by lines; according to the Y-axis coordinate of the line corresponding to the left acromion point P1 and the right acromion point P2, all pixel points in the preset human body region that are greater than the Y-axis coordinate of the line corresponding to the left acromion point P1 and the right acromion point P2 are subjected to image masking along the Y-axis direction, thereby obtaining a back region in the human body region.

[0138] It should be understood that due to the difference in individual back morphology, the line connecting the left acromion point P1 and the right acromion point P2, and the line connecting the first boundary point P1' and the second boundary point P2', have different coordinate values ​​in the Y-axis direction, that is, along the y-axis direction in the two-dimensional coordinate system of the image, the coordinate values ​​corresponding to the y-axis are different. Therefore, in the actual application process, it is necessary to make a comparison and judgment based on the coordinate values ​​corresponding to the y-axis, and then determine whether to adopt the above-mentioned first implementation or the second implementation.

[0139] For example, based on the y-axis coordinate value, if the y-axis mean coordinate of the line connecting the left acromion point P1 and the right acromion point P2 is greater than the y-axis mean coordinate of the line connecting the first boundary point P1' and the second boundary point P2', the first implementation method is adopted. If the y-axis mean coordinate of the line connecting the left acromion point P1 and the right acromion point P2 is less than the y-axis mean coordinate of the line connecting the first boundary point P1' and the second boundary point P2', the second implementation method is adopted. If they are equal, both implementation methods are acceptable.

[0140] Through the above-mentioned back recognition method based on human body images, the back area can be simply and efficiently determined in the human body area, and the determined back area has high accuracy. Since the process does not rely on the judgment and processing of human skin color, its application range is wide. It should be understood that as a simple and effective segmentation scheme for human body lying images, this scheme is more suitable for positioning and recognition on images. Furthermore, in the above-mentioned human body image, the human body is in a prone posture facing the bed surface, and the human body and the bed surface where the human body is located can be easily distinguished based on the point cloud. Through the geometric features of the human body, it is relatively easy to determine the key points of the human body in the human body key point recognition model, so as to facilitate the masking of the pixel points of the human body limbs and head in the image. The above-mentioned recognition and determination process does not require the collection of data sets for training, and the processing method is simple and efficient and can adapt to groups of different body shapes and skin colors.

[0141] The above-mentioned embodiment provides a back recognition method based on human body images. Correspondingly, the embodiment of the present application also provides a back recognition device based on human body images. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For details of the relevant technical features, please refer to the corresponding description of the method embodiment provided above. The following description of the device embodiment is merely illustrative.

[0142] Please refer to Fig.13 To understand this embodiment, Fig.13 This is a unit block diagram of a back recognition device based on a human body image provided in this embodiment. As shown in the figure, the back recognition device 1300 based on a human body image provided in this embodiment includes:

[0143] The human body key point determination unit 1301 is used to determine the human body key points in the human body image, and determine the human body area in the human body image according to the human body key points; the human body key points include: shoulder key points, neck key points, hip joint key points, elbow key points and hip bone key points.

[0144] The shoulder boundary point determining unit 1302 is used to determine the shoulder boundary point according to the shoulder key point and the first connecting line between the neck key point and the hip joint key point.

[0145] The armpit point determination unit 1303 is used to determine the armpit point according to the angle formed by the connecting lines of the shoulder key point, the elbow key point and the hip key point, and the distance between the shoulder key point and the hip key point.

[0146] The chest point and hip point determination unit 1304 is used to determine the chest point and hip point according to the shoulder key point, the hip key point, the armpit point, and the first connecting line.

[0147] The back region determining unit 1305 is used to determine the back region in the human body region according to the shoulder boundary point, the armpit point, the chest point and the hip point.

[0148] In one embodiment, determining the shoulder boundary point according to the shoulder key point and the first connecting line between the neck key point and the hip joint key point includes:

[0149] generating a first ray extending from the shoulder key point along a specified direction and parallel to the first connecting line;

[0150] Determining a first intersection point in the human body region according to an intersection point between the first ray and the human body region;

[0151] The first intersection point is determined as the shoulder boundary point.

[0152] In one embodiment, determining the shoulder boundary point according to the shoulder key point and the first connecting line between the neck key point and the hip joint key point includes:

[0153] Obtain a shoulder key point connection line of the shoulder key point and a hip key point connection line of the hip key point;

[0154] Determine the intersection of the shoulder key point connection line and the first connection line as the second intersection point;

[0155] Generate a first parallel line extending from the second intersection point and parallel to the line connecting the hip bone key points;

[0156] Determine the intersection point of the first parallel line and the human body region as a boundary intersection point;

[0157] In response to the distance value of the line corresponding to the boundary intersection meeting the preset distance condition, the boundary intersection is determined as the first boundary point and the second boundary point; in response to the distance value of the line corresponding to the boundary intersection not meeting the preset distance condition, the second intersection is moved along a predetermined direction of the first line, and the boundary intersection follows the movement of the second intersection until the distance value of the line corresponding to the boundary intersection meets the preset distance condition;

[0158] The first boundary point and the second boundary point satisfying a preset distance condition are determined as the shoulder boundary points.

[0159] In one implementation, in response to the distance value of the line corresponding to the boundary intersection point satisfying a preset distance condition, determining the boundary intersection point as the first boundary point and the second boundary point includes:

[0160] Obtain the distance value of the shoulder key point connection line;

[0161] In response to the fact that the distance value of the line corresponding to the boundary intersection point does not exceed the distance value of the line connecting the shoulder key points, the boundary intersection point is determined as the first boundary point and the second boundary point.

[0162] In one embodiment, determining the axillary point according to the angle formed by the connecting lines of the shoulder key point, the elbow key point and the hip key point, and the distance between the shoulder key point and the hip key point, comprises:

[0163] generating a first vector extending from the shoulder key point to the elbow key point;

[0164] generating a second vector extending from the shoulder key point to the hip key point;

[0165] Generate a vector angle corresponding to the shoulder key point according to the first vector and the second vector;

[0166] Determine a second ray within the vector angle according to the angle of the vector angle;

[0167] The axillary point is determined on the second ray according to the distance between the shoulder key point and the hip key point.

[0168] In one implementation, determining the second ray within the vector angle according to the angle of the vector angle includes:

[0169] Determine the proportional value corresponding to the vector angle;

[0170] According to the ratio value, a second ray with the shoulder key point as a vertex is determined on a side close to the second vector within the vector angle.

[0171] In one embodiment, determining the chest point and the hip point according to the shoulder key point, the hip key point, the armpit point, and the first connecting line includes:

[0172] generating a third ray extending from the hip key point along the specified direction and parallel to the first connecting line;

[0173] Determining a first intermediate point on the third ray according to the distance value of the first connecting line;

[0174] Generate a vertical line extending from the armpit point along a second specified direction and perpendicular to a line corresponding to the shoulder key point and the first middle point;

[0175] Determine a second middle point according to an intersection of the vertical line and a line corresponding to the shoulder key point and the first middle point;

[0176] Construct a parallelogram by using the first middle point, the second middle point and the axillary point as vertices of the parallelogram to determine the thoracic point;

[0177] A parallelogram is constructed by using the hip bone key point, the first middle point and the thoracic cavity point as vertices of the parallelogram to determine the hip point.

[0178] In one implementation, determining the first intermediate point on the third ray according to the distance value of the first connecting line includes:

[0179] Determine, on the first connecting line, a distance value of a midpoint of the first connecting line according to the distance value of the first connecting line;

[0180] The first middle point is determined on the third ray according to the distance value of the midpoints of the first connecting line.

[0181] In one implementation, determining a human body region in the human body image according to the human body key points includes:

[0182] Generating a human body preprocessing image in the human body image according to the human body key points;

[0183] Performing plane detection on the preprocessed image to determine plane depth data;

[0184] A human body region in the human body image is determined according to the depth data of the human body image and the planar depth data.

[0185] In one implementation, determining key points of a human body in a human body image includes:

[0186] The human body image is input into a human body key point recognition model to determine the human body key points in the human body image.

[0187] The above embodiment provides a back recognition device based on human body images. Correspondingly, the present application embodiment also provides a massage robot, including the back recognition device based on human body images described in the above embodiment. For details of the relevant technical features, please refer to the corresponding description of the device embodiment provided above, and this embodiment will not be repeated.

[0188] In specific implementation, during the operation of the massage robot, the massage robot sends prompt information such as voice, message pop-up window, image warning, etc. to the user, thereby reminding the user to correct the lying posture of the human body so that the front of the human back can be displayed to the image information acquisition device of the massage robot. When the user completes the correction of the lying posture of the human body according to the prompt information, the image information acquisition device of the massage robot collects the image of the human body.

[0189] In an optional embodiment, the human body image collected by the massage robot image information collection device can be directly used for image recognition processing by the massage robot processor to obtain the back area of ​​the human body. The massage robot's mechanical arm simulates a human hand to periodically apply pressure in the corresponding area of ​​the real human body according to the back area, thereby achieving the effect of relieving back acupuncture points and muscle fatigue.

[0190] In another optional embodiment, the human body image collected by the massage robot image information collection device can be sent to a cloud server via network communication. The cloud server performs back recognition on the human body image to obtain the back area corresponding to the human body area. The cloud server then transmits the recognition information of the back area to the massage robot via network communication. After the massage robot receives the recognition information of the back area, the robotic arm simulates the human hand and accurately applies pressure in the corresponding area of ​​the real human body according to the back area.

[0191] In the above embodiments, a variety of methods and devices corresponding to the methods are provided. In addition, the embodiments of the present application also provide electronic device embodiments corresponding to the above method embodiments and device embodiments. Since the electronic device embodiments are basically similar to the method embodiments, the description is relatively simple. For the details of the relevant technical features and the effects achieved, please refer to the corresponding description of the method embodiments provided above. The following description of the electronic device embodiments is only illustrative. The electronic device embodiments are as follows:

[0192] In this embodiment, an optional hardware structure of the electronic device 1400 can be as follows: Fig.14As shown, it includes: at least one processor 1401, at least one memory 1402 and at least one communication bus 1405; the memory 1402 contains a program 1403 and data 1404.

[0193] The bus 1405 may be a communication device for transmitting data between components inside the electronic device 1400, such as an internal bus (e.g., a CPU-memory bus, where the processor is a central processing unit, CPU for short), an external bus (e.g., a universal serial bus port, a peripheral component interconnect express port), etc.

[0194] In addition, the electronic device also includes: at least one network interface 1406, at least one peripheral interface 1407. The network interface 1406 provides wired or wireless communication related to an external network 1408 (for example, the Internet, an intranet, a local area network, a mobile communication network, etc.); in some embodiments, the network interface 1406 may include any number of network interface controllers (NICs), radio frequency (RF) modules, repeaters, transceivers, modems, routers, gateways, any combination of wired network adapters, wireless network adapters, Bluetooth adapters, infrared adapters, near field communication (NFC) adapters, cellular network chips, etc.

[0195] The peripheral interface 1407 is used to connect to the peripheral device, and the peripheral device can be the peripheral device 1 ( Fig.14 1409 in), peripheral 2 ( Fig.14 1410) and peripheral 3 ( Fig.14 1411 in the figure). Peripherals are peripheral devices, which may include but are not limited to cursor control devices (such as mice, touch pads or touch screens), keyboards, displays (such as cathode ray tube displays, liquid crystal displays). Displays or light emitting diode displays, video input devices (such as cameras or communication interfaces coupled to video files), etc.

[0196] The processor 1401 may be a CPU, or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0197] The memory 1402 may include a high-speed RAM (full name: Random Access Memory) memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0198] The processor 1401 calls the program and data stored in the memory 1402 to execute the aforementioned method embodiment of the present application.

[0199] Corresponding to the device (or node, node device, service device) in the game server system provided in the embodiment of the present application, the embodiment of the present application also provides a computer-readable storage medium, which stores a program of execution logic corresponding to the device in the game server system. The program is run by the corresponding device and can execute the execution logic involved in the above-mentioned embodiments of the device.

[0200] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0201] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0202] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0203] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), random access memory (RAM) of other attributes, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage media or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0204] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

Claims

1. A back recognition method based on human body images, characterized in that: The method comprises: Determine the key points of a human body in a human body image, and determine the human body region in the human body image according to the key points of the human body; the key points of the human body include: shoulder key points, neck key points, hip joint key points, elbow key points and hip bone key points; Determine a shoulder boundary point according to the shoulder key point and a first connecting line between the neck key point and the hip joint key point; Determine the armpit point according to the angle formed by the connecting lines of the shoulder key point, the elbow key point and the hip key point, and the distance between the shoulder key point and the hip key point; Determine a chest point and a hip point according to the shoulder key point, the hip key point, the armpit point, and the first connecting line; A back region is determined in the human body region based on the shoulder boundary point, the armpit point, the thorax point, and the hip point.

2. The back recognition method based on human body image according to claim 1, characterized in that: Determining the shoulder boundary point according to the shoulder key point and the first connecting line between the neck key point and the hip joint key point includes: generating a first ray extending from the shoulder key point along a specified direction and parallel to the first connecting line; Determining a first intersection point in the human body region according to an intersection point between the first ray and the human body region; The first intersection point is determined as the shoulder boundary point.

3. The back recognition method based on human body image according to claim 1, characterized in that: Determining the shoulder boundary point according to the shoulder key point and the first connecting line between the neck key point and the hip joint key point includes: Obtain a shoulder key point connection line of the shoulder key point and a hip key point connection line of the hip key point; Determine the intersection of the shoulder key point connection line and the first connection line as the second intersection point; Generate a first parallel line extending from the second intersection point and parallel to the line connecting the hip bone key points; Determine the intersection point of the first parallel line and the human body region as a boundary intersection point; In response to the distance value of the line corresponding to the boundary intersection meeting the preset distance condition, the boundary intersection is determined as the first boundary point and the second boundary point; in response to the distance value of the line corresponding to the boundary intersection not meeting the preset distance condition, the second intersection is moved along a predetermined direction of the first line, and the boundary intersection follows the movement of the second intersection until the distance value of the line corresponding to the boundary intersection meets the preset distance condition; The first boundary point and the second boundary point satisfying a preset distance condition are determined as the shoulder boundary points.

4. The back recognition method based on human body image according to claim 3, characterized in that: In response to the distance value of the line corresponding to the boundary intersection point satisfying a preset distance condition, determining the boundary intersection point as a first boundary point and a second boundary point includes: Obtain the distance value of the shoulder key point connection line; In response to the fact that the distance value of the line corresponding to the boundary intersection point does not exceed the distance value of the line corresponding to the shoulder key points, the boundary intersection point is determined as the first boundary point and the second boundary point.

5. The back recognition method based on human body image according to claim 1, characterized in that: Determining the axillary point according to the angle formed by the connecting lines of the shoulder key point, the elbow key point and the hip key point, and the distance between the shoulder key point and the hip key point, comprises: generating a first vector extending from the shoulder key point to the elbow key point; generating a second vector extending from the shoulder key point to the hip key point; Generate a vector angle corresponding to the shoulder key point according to the first vector and the second vector; Determine a second ray within the vector angle according to the angle of the vector angle; The axillary point is determined on the second ray according to the distance between the shoulder key point and the hip key point.

6. The back recognition method based on human body image according to claim 5, characterized in that: The determining a second ray within the vector angle according to the angle of the vector angle comprises: Determine the proportional value corresponding to the vector angle; According to the ratio value, a second ray with the shoulder key point as a vertex is determined on a side close to the second vector within the vector angle.

7. The back recognition method based on human body image according to claim 1, characterized in that: Determining the chest point and the hip point according to the shoulder key point, the hip key point, the armpit point, and the first connecting line includes: generating a third ray extending from the hip key point along a specified direction and parallel to the first connecting line; Determining a first intermediate point on the third ray according to the distance value of the first connecting line; Generate a vertical line extending from the armpit point along a second specified direction and perpendicular to a line corresponding to the shoulder key point and the first middle point; Determine a second middle point according to an intersection of the vertical line and a line corresponding to the shoulder key point and the first middle point; Construct a parallelogram by using the first middle point, the second middle point and the axillary point as vertices of the parallelogram to determine the thoracic point; A parallelogram is constructed by using the hip bone key point, the first middle point and the thoracic cavity point as vertices of the parallelogram to determine the hip point.

8. The back recognition method based on human body image according to claim 7, characterized in that: The step of determining a first intermediate point on the third ray according to the distance value of the first connecting line includes: Determine, on the first connecting line, a distance value of a midpoint of the first connecting line according to the distance value of the first connecting line; The first middle point is determined on the third ray according to the distance value of the midpoints of the first connecting line.

9. The back recognition method based on human body image according to claim 1, characterized in that: The determining of the human body region in the human body image according to the human body key points comprises: Generating a human body preprocessing image in the human body image according to the human body key points; Performing plane detection on the preprocessed image to determine plane depth data; A human body region in the human body image is determined according to the depth data of the human body image and the planar depth data.

10. The back recognition method based on human body image according to claim 1, characterized in that: The step of determining key points of a human body in a human body image comprises: The human body image is input into a human body key point recognition model to determine the human body key points in the human body image.

11. A back recognition device based on human body image, characterized in that: The device comprises: A human body key point determination unit, used to determine the human body key points in the human body image, and determine the human body area in the human body image according to the human body key points; the human body key points include: shoulder key points, neck key points, hip joint key points, elbow key points and hip bone key points; A shoulder boundary point determination unit, configured to determine the shoulder boundary point according to the shoulder key point and a first connecting line between the neck key point and the hip joint key point; an axillary point determination unit, configured to determine the axillary point according to the angle formed by the connecting lines of the shoulder key point, the elbow key point and the hip key point, and the distance between the shoulder key point and the hip key point; A chest point and hip point determination unit, used to determine a chest point and a hip point according to the shoulder key point, the hip key point, the armpit point, and the first connecting line; A back region determining unit is used to determine a back region in the human body region according to the shoulder boundary point, the armpit point, the chest point and the hip point.

12. A massage robot, characterized in that: The massage robot includes the back recognition device based on human body image as claimed in claim 11.

13. An electronic device, characterized in that: include: A processor, a memory, and computer program instructions stored on the memory and executable on the processor; When the processor executes the computer program instructions, the back recognition method based on human body images as described in any one of claims 1 to 10 is implemented.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, they are used to implement the back recognition method based on human body images as described in any one of claims 1 to 10 above.

Images