A real-time detection and recognition method for human lying posture

Abstract

The present invention discloses a method for real-time detection and recognition of human lying posture. A depth camera is used for data collection. Under the premise of protecting the privacy of users to the maximum extent, the depth map collected by the depth camera is transmitted to the computer end, and the computer end captures and recognizes the subject in real time, generates the corresponding key nodes of the human skeleton, and displays the generated skeleton point matching picture on the display in real time, and encrypts and saves the picture in the storage end of the computer. In addition, the present invention also has a specific action recognition function, which can assist personnel to discover and deal with corresponding problems in time. The technology of the present invention has the advantages of low cost, non-contact, high precision, good privacy protection, etc., and meets the needs of human lying posture detection and real-time motion detection in multiple scenarios.

Description

Technical Field

[0001] The invention belongs to the technical field of machine vision, and in particular relates to a real-time detection and recognition method for a human lying posture. Background Art

[0002] With the development of technology, people have discovered that bed resting posture, which accounts for about one-third of life activities, has a profound impact on human health.

[0003] At present, there are several common methods in the field of lying posture recognition and human posture detection: wearable sensor method, camera monitoring method, pressure matrix method, etc. Although the wearable sensor data is accurate, it is less convenient. The camera monitoring method will cause a certain psychological burden on personnel. The pressure matrix method has problems such as high cost and poor reliability. Therefore, there is an urgent need for a low-cost, contactless and maximum privacy protection human lying posture real-time detection system. Summary of the invention

[0004] The purpose of the present invention is to provide a real-time detection and recognition method for human lying posture in view of the shortcomings of the current prior art. A depth camera is used for data collection. Under the premise of protecting the privacy of users to the maximum extent, the depth map collected by the depth camera is transmitted to the computer end, and the computer end captures and identifies the subject in real time, generates the corresponding key nodes of the human skeleton, and displays the generated skeleton point matching picture on the display in real time, and encrypts the picture and saves it in the storage end of the computer. In addition, the present invention also has a specific action recognition function, which can assist personnel to find and deal with corresponding problems in time. The technology of the present invention has the advantages of low cost, contactless, high precision, good privacy protection, etc., and meets the needs of human lying posture detection and real-time motion detection in multiple scenarios.

[0005] The present invention is achieved through the following technical solutions:

[0006] A method for real-time detection and recognition of a human body lying posture comprises the following steps:

[0007] S1: Arrange a depth camera, collect depth images of the lying subject in time series through the depth camera, and transmit the collected data to a computer for processing;

[0008] S2: Process the depth image data obtained from the depth camera, segment the background in the depth image, and mark the position of the person in the depth image;

[0009] S3: identifying key points of the human body, constructing a human posture model based on these key points of the human body, and displaying the image of the human posture model on the display in real time;

[0010] S4: Calculating the positions of the key points of the human body in the constructed human body posture model, and identifying the motion state of the subject according to the relative position changes of the key points of the human body, including the following steps:

[0011] S4.1: transform the coordinates of each key point data and map them to the global coordinate system;

[0012] S4.2: Use the principle of digital filter to filter, deal with the problems of overlap and confusion in the movement of key points of the human body in continuous frames, and pre-process the trajectory of the skeleton nodes to reduce the amplitude of node position jump;

[0013] S4.3: Clarify the joints in the human posture model. Each joint is composed of three adjacent human key points as joint points. Construct the structural vectors between the joint points corresponding to each joint. Calculate the angle of each joint based on the structural vectors between the joint points.

[0014] S4.4: Calculate the relative displacement of different joints in the same frame. Each joint in the same frame is represented by x i , the posture feature f of the current frame (c) cc , which can represent the coordinate difference of different joint points:

[0015] f cc ={x i -x j |i,j=1,2,…N;i≠j}

[0016] S4.5: Calculate the relative displacement of the same joint point in different frames. Relative to the previous frame (p), the offset of the current frame in the prone state movement can be calculated. The coordinate difference of each joint point between the current frame and the previous frame can be calculated to capture the joint movement characteristics, which is expressed as:

[0017]

[0018] S4.6: The offset of the current frame in motion relative to the initial frame (s). The coordinate difference of each joint point between the current frame and the initial frame is calculated as the offset feature, expressed as f cs :

[0019]

[0020] S4.7: For posture feature f cc , motion characteristics cp , offset feature f cs The combined form of each joint position information in each frame can be expressed as: c =[f cc ,f cp ,f cs ], for fc Perform dimensionality reduction and make judgment with the set value. If the angle change information exceeds the set threshold, the angle change information is discarded. If the angle change information falls within the set threshold, the angle change information is retained.

[0021] S4.8: For each joint angle information within a specific time window, the amount of information is represented by entropy, and the information entropy h(x) is expressed as:

[0022] h(x)=-∫f(x)logf(x)dx

[0023] The entropy of the random variable is proportional to the logarithm of its variance, so it is used to measure the joint angle information in each time period. The joint angle information is input into the behavior classifier of the support vector machine to identify the current motion state of the subject;

[0024] S5: Display the identified current motion state of the subject on a display.

[0025] In the above technical solution, step S2 includes the following steps:

[0026] S2.1: According to the relationship between focal length and object distance, a reference depth value s0 is set as the threshold for determining whether the image belongs to the foreground or the background;

[0027] S2.2: The depth image data collected by the depth camera is normalized to determine the scale of background segmentation. The normalization formula is:

[0028]

[0029] Where s is the original depth data of each point in the image, s max is the maximum original depth value s of the points contained in the image min is the minimum original depth value of the points contained in the image;

[0030] V represents the depth value normalized to the range of 0-255 for the original depth data of each point in the image;

[0031] According to the above formula, V0 is obtained, where V0 represents the depth value at the focus after s0 is normalized;

[0032] S2.3: Perform preliminary stratification on the depth image. When 0≤V≤V0, it is divided into the background layer; when V0≤V≤2V0, it is divided into the transition layer; when 2V0≤V≤255, it is divided into the foreground layer;

[0033] S2.4: Scale the depth information of each layer in the depth image, mark the foreground layer as 0, the transition layer as 1, and the background layer as 2; then perform edge smoothing preprocessing to maintain a natural transition of the image edge. The scale of each point in the smoothed image is between 0 and 2, and the blur scale of the blur processing of each point in the image is determined according to the scale label, where the blur radius R(i,j) of each point is determined according to the following formula:

[0034] R(i,j)=α(i,j)×k

[0035] S2.5: Calculate the Gaussian standard deviation and blur radius, and use the following formula to perform Gaussian blur processing of different scales on each point in the image to obtain a clear prone subject image:

[0036]

[0037] in is the convolution operation, and f(i,j,σ(i,j)) is the Gaussian distribution function.

[0038] In the above technical scheme, the key points of the human body include: head, nose, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hand, right hand, chest, abdomen, buttocks, left waist, right waist, left knee, right knee, left ankle, right ankle, left foot and right foot.

[0039] In the above technical solution, in step 3, a joint regression network is used to identify key points of the human body in the depth image processed by step S2, and the specific steps are:

[0040] S3.1: The joint regression network consists of a two-dimensional backbone network (i.e., ResNet-50) and three functional branches, namely, plane transfer branch, depth estimation branch, and anchor proposal branch;

[0041] S3.2: The plane offset branch and the depth estimation branch are used to predict the 3D position of the key points of the human body in the depth image. For a certain key point, the output of the aggregated anchor point is used for positioning;

[0042] S3.3: The anchor proposal branch searches for information anchor points of specific key points by weight specification. The output layer of this branch is set to 16 K channels.

[0043] S3.4: The backbone network is ResNet-50 pre-trained on ImageNet. The convolution stride in the fourth layer is set to 1. At the same time, the convolution operation of this layer is changed to dilated convolution with a dilation step of 2 to expand the receptive field.

[0044] S3.5: For the key point j in the depth image, in order to make Tij and the depth target T dj Comparable, the Ground Truth depth Z of the key point j Converts to:

[0045]

[0046] Among them, μ and θ are transformation parameters. For the requirement of lying posture recognition, μ is set to 1 and θ is set to 0;

[0047] S3.6: The A2J model is trained under the joint supervision of two loss functions: key point estimation loss and information anchor loss. The objective loss of key point loss is expressed as:

[0048]

[0049] where α is a factor that balances the smoothing offset and depth estimation tasks and is usually set to 0.5, and τ1 and τ2 are set to 1 and 3 to smooth the relative noise in the depth map. and is the in-plane and depth target position of key point j, and the smoothing loss function L τ The formula is:

[0050]

[0051] The surrounding information anchor loss is used to improve the generalization ability of the model so that key points can be observed from multiple angles. The information anchor point formula around the loss is:

[0052]

[0053] Therefore, the end-to-end learning process is determined by two loss functions, as follows:

[0054] loss=λloss1+loss2

[0055] λ is the loss weight factor, which is usually set to 3 to balance loss 1 and loss 2.

[0056] In the above technical solution, in step 3, the present invention takes into account that the contribution of each anchor point to the key point is not the same, so the plane position and depth value of the key point j can be used as the weighted average output of the anchor point, and the specific formula is as follows:

[0057]

[0058] In the formula, S j represents the plane position of key point j, D j represents the estimated depth of key point j; P j(a) is the normalized weight value of anchor point a through all anchor points toward key point j, and is obtained using SoftMax through the following formula:

[0059]

[0060] In the above technical solution, the images collected by the depth camera are encrypted and stored in a chaotic logistic manner to ensure the privacy of the person being detected; the specific steps are shown in S1-S4:

[0061] S1: Calculate the size of the original image for a frame to be saved and represent it in the matrix [M, N];

[0062] S2: Generate two Logistic chaotic sequences for the matrix [M, N], and transform these two Logistic sequences to obtain two y sequences;

[0063] S3: Encrypt the image by replacing the values ​​with the two sequences y1 and y2;

[0064] S4: Save y1, y2 sequences and encrypted pixels.

[0065] The advantages and beneficial effects of the present invention are:

[0066] The present invention can be applied to the detection of supine human body motion process in multiple scenarios. The subject can capture the posture information of the human body in the supine state more accurately without wearing any sensor equipment, generate a skeletal point diagram of the key node parts of the supine subject according to the human posture, adjust the subject's motion detection and judgment mechanism according to the subject's specific needs, and send the detection picture and judgment result to the display end to remind the monitoring personnel.

[0067] A depth camera is used to collect images in real time to avoid excessive psychological pressure on the subjects caused by the use of RGB cameras. Certain encryption processing is performed in the process of storing the images frame by frame to protect the privacy of the subjects to a certain extent.

[0068] Compared with several existing lying posture recognition methods, the present invention has the advantages of low cost, non-contact, high recognition accuracy, good privacy, etc., and has certain practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a flow chart of the real-time detection and recognition method of human lying posture of the present invention.

[0070] Figure 2 It is a schematic diagram of the composition structure of the skeleton points of the key parts of the human body in the present invention.

[0071] Figure 3 It is a display screen of the human body posture model detected and generated by the present invention.

[0072] For ordinary technicians in this field, other relevant drawings can be obtained based on the above drawings without any creative work. DETAILED DESCRIPTION

[0073] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention is further described below in conjunction with specific embodiments.

[0074] Embodiment 1

[0075] A method for real-time detection and recognition of a human body lying posture comprises the following steps:

[0076] S1: Arrange a depth camera, collect depth images of the lying subject in time series through the depth camera, and transmit the collected data to a computer for processing.

[0077] S2: Process the depth image data obtained from the depth camera, segment the background in the depth image, and mark the position of the person in the depth image. Specifically, the background in the depth image is segmented using the Gaussian depth layered processing method, and the steps are:

[0078] S2.1: According to the relationship between focal length and object distance, a reference depth value s0 is set as the threshold for determining whether the image belongs to the foreground or the background.

[0079] S2.2: The depth image data collected by the depth camera is normalized to determine the scale of background segmentation. The normalization formula is:

[0080]

[0081] Where s is the original depth data of each point in the image, s max is the maximum original depth value of the points contained in the image, s min is the minimum original depth value of the points contained in the image;

[0082] V represents the depth value normalized to the range of 0-255 for the original depth data of each point in the image;

[0083] According to the above formula, V0 is obtained, where V0 represents the depth value at the focus after s0 is normalized.

[0084] S2.3: Perform preliminary stratification on the depth image. When 0≤V≤V0, it is divided into the background layer; when V0≤V≤2V0, it is divided into the transition layer; when 2V0≤V≤255, it is divided into the foreground layer.

[0085] S2.4: Scale the depth information of each layer in the depth image, mark the foreground layer as 0, the transition layer as 1, and the background layer as 2; then perform edge smoothing preprocessing to maintain a natural transition of the image edge. The scale of each point in the smoothed image is between 0 and 2, and the blur scale of the blur processing of each point in the image is determined according to the scale label, where the blur radius R(i,j) of each point is determined according to the following formula:

[0086] R(i,j)=α(i,j)×k

[0087] S2.5: Calculate the Gaussian standard deviation and blur radius, and use the following formula to perform Gaussian blur processing of different scales on each point in the image to obtain a clear prone subject image:

[0088]

[0089] in is the convolution operation, and f(i,j,σ(i,j)) is the Gaussian distribution function.

[0090] S3: Continue to process the depth image with the person's position marked, identify the key bone points of the human body (hereinafter referred to as key points), and then construct a human posture model based on these key bone points of the human body, and display the human posture model in real time on the display (see Appendix Figure 3 );

[0091] In this embodiment, the key points of the human body include: head, nose, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hand, right hand, chest, abdomen, buttocks, left waist, right waist, left knee, right knee, left ankle, right ankle, left foot and right foot. Figure 2 .

[0092] Specifically, the anchored joint regression network A2J is used to identify the key points of the human body in the depth image processed by step S2. The specific steps are:

[0093] S3.1: A2J consists of a two-dimensional backbone network (i.e., ResNet-50) and three functional branches, namely, plane transfer branch, depth estimation branch, and anchor proposal branch.

[0094] S3.2: The plane offset branch and the depth estimation branch are used to predict the three-dimensional position of the key points of the human body in the depth image. For a certain key point, the output of the aggregated anchor point is used for positioning. The present invention takes into account that the contribution of each anchor point to the key point is not the same, so the plane position and depth value of the key point j can be used as the weighted average of the anchor point output. The specific formula is as follows:

[0095]

[0096] In the formula, S j represents the plane position of key point j, D j represents the estimated depth of key point j; P j (a) is the normalized weight value of anchor point a through all anchor points toward key point j, and is obtained using SoftMax through the following formula:

[0097]

[0098] S3.3: The anchor proposal branch searches for information anchor points of specific key points by weight specification, and the output layer of this branch is set to 16 K channels.

[0099] S3.4: The backbone network is ResNet-50 pre-trained on ImageNet. To meet the needs of lying posture recognition, the convolution stride in the fourth layer is set to 1. At the same time, the convolution operation of this layer is changed to dilated convolution with an expansion amplitude of 2 to expand the receptive field.

[0100] S3.5: For the key point j in the depth image, in order to make T il and the depth target T dj Comparable, the Ground Truth depth Z of the key point j Translates to:

[0101]

[0102] Among them, μ and θ are transformation parameters. For the requirement of lying posture recognition, μ is set to 1 and θ is set to 0;

[0103] S3.6: The A2J model is trained under the joint supervision of two loss functions: key point estimation loss and information anchor loss. The objective loss of key point loss is expressed as:

[0104]

[0105] where α is a factor that balances the smoothing offset and depth estimation tasks and is usually set to 0.5, and τ1 and τ2 are set to 1 and 3 to smooth the relative noise in the depth map. and is the in-plane and depth target position of key point j, and the smoothing loss function L τ The formula is:

[0106]

[0107] The surrounding information anchor loss is used to improve the generalization ability of the model so that key points can be observed from multiple angles. The information anchor point formula around the loss is:

[0108]

[0109] Therefore, the end-to-end learning process is determined by two loss functions, as follows:

[0110] loss=λloss1+loss2

[0111] λ is the loss weight factor, which is usually set to 3 to balance loss 1 and loss 2.

[0112] S4: Calculate the position of each key point in the constructed human posture model, and identify the motion state of the lying subject according to the relative position change of the key points. It includes the following steps:

[0113] S4.1: Perform coordinate transformation on the data of each key point and map them to the global coordinate system.

[0114] S4.2: Use the principle of digital filter to filter, deal with the problems of overlap and confusion in the movement of key points of the human body in continuous frames, and pre-process the trajectory of the skeleton nodes to reduce the amplitude of node position jumps.

[0115] S4.3: Clarify the joints in the human body posture model. Each joint is composed of three adjacent bone key points as joint points. Construct the structural vectors between the joint points corresponding to each joint. The angle of each joint can be calculated based on the structural vectors between the joints. Specifically, in this embodiment, see the attached Figure 2 The human body posture model includes 14 joints, and the angle θ of each joint is defined. i The angles of the joints of the trunk are θ1 and θ2, the angles of the joints of the upper arms are θ3, θ4, θ5, θ6, θ7, θ8, and the angles of the joints of the lower legs are θ9, θ10, θ11, θ12, θ13, θ14, θ15, θ16, θ17, θ18, and θ19. 10 ,θ 11 ,θ 12 ,θ 13 ,θ 14 .

[0116] Each joint point is represented by a three-dimensional coordinate, and then the structural vector between the joint points is defined, for example: Figure 2 In the figure, the right shoulder joint is represented by S (sx, sy, sz), the right elbow joint is represented by E (ex, ey, ez), and the right wrist joint is represented by W (wx, wy, wz). The inter-joint structure vector is constructed, and the vector from the right elbow to the right shoulder is The vector from the right elbow to the right wrist is The vector from the right shoulder to the right wrist is Then the angle θ7 can be expressed by the vector According to the inverse cosine theorem, the formula is:

[0117]

[0118] The other joint angles can be calculated in the same way.

[0119] S4.4: Calculate the relative displacement of different joints in the same frame. Each joint in the same frame is represented by x i , the posture feature f of the current frame (c) cc , which can represent the coordinate difference of different joint points:

[0120] f cc ={x i -x j |i,j=1,2,…N;i≠j}

[0121] S4.5: Calculate the relative displacement of the same joint point in different frames. Relative to the previous frame (p), the offset of the current frame in the prone state movement can be calculated. The coordinate difference of each joint point between the current frame and the previous frame can be calculated to capture the joint movement characteristics, expressed as f cp :

[0122]

[0123] S4.6: The offset of the current frame in motion relative to the initial frame (s). The coordinate difference of each joint point between the current frame and the initial frame is calculated as the offset feature, expressed as f cs :

[0124]

[0125] S4.7: For posture feature f cc , motion characteristics cp , offset feature f cs The combined form of each joint position information in each frame can be expressed as: c =[f cc ,f cp ,f cs ], for f c Dimensionality reduction is performed and judged against the set value. The angle change information is discarded if it exceeds the set threshold, and retained if it falls within the set threshold.

[0126] S4.8: For each joint angle information within a specific time window, the amount of information is represented by entropy, and the information entropy h(x) is expressed as:

[0127] h(x)=-∫f(x)logf(x)dx

[0128] The entropy of the random variable is proportional to the logarithm of its variance, so it is used to measure the joint angle information in each time period. The joint angle information is input into the behavior classifier of the support vector machine (SVM) to identify the current movement state of the lying subject.

[0129] S5: Display the identified current motion state of the subject in the prone position on a display so that medical staff can know it and remind the monitoring staff.

[0130] Embodiment 2

[0131] Furthermore, the present invention encrypts and stores the real-time lying posture detection image in a chaotic logistic manner, thereby ensuring the privacy of the person being detected. The specific steps are shown in S1-S4:

[0132] S1: Calculate the size of the original image for a frame to be saved and represent it in the matrix [M, N];

[0133] S2: Generate two Logistic chaotic sequences for the matrix [M, N], and transform these two Logistic sequences to obtain two y sequences;

[0134] S3: Encrypt the image by replacing the values ​​with the two sequences y1 and y2;

[0135] S4: Save y1, y2 sequences and encrypted pixels.

[0136] Embodiment 3

[0137] The effect of real-time motion state detection of the human body in a supine position is related to the angle between the depth camera and the subject's spatial position. The observation range of the depth camera is 0.25m-2.88m, and the measurement angle is a fan-shaped sphere of 120°x120°. This example takes a standard nursing bed with a length of 2200mm, a width of 900mm, and a height of 500-700mm as an example.

[0138] When a single camera is used, the depth camera is suspended above the central axis of the bed and at the top of the room xmm away from the end of the bed. The suspension angle is α. When the working mode is NFOV, the formula is satisfied:

[0139]

[0140] √[(h 2 +(2200-x) 2 ]<3860

[0141] The vertical distance between the depth camera and the bed satisfies:

[0142] 2300<h<2500

[0143] In this example, when the depth camera is suspended 245 mm from the end of the bed, the optimal angle α should be less than 6.17°.

[0144] When two cameras are used, they are hung diagonally at the head and foot of the bed, with hanging angles of α and β respectively. When the working mode is NFOV, the formula is satisfied:

[0145] h(tanα+tanβ)<2376.9

[0146] h*cosα<2880

[0147] h*cosβ<2880

[0148] The vertical distance between the camera and the bed satisfies:

[0149] 2300<h<2500

[0150] In this example, when the camera is hung at the head or foot of the bed, when the α angle is selected to be 15°, the β angle should be less than 34.33°.

[0151] The present invention is described above by way of example. It should be noted that, without departing from the core of the present invention, any simple deformation, modification or other equivalent replacement that can be made by those skilled in the art without inventive effort falls within the protection scope of the present invention.

Claims

1. A method for real-time detection and recognition of a human body lying posture, characterized in that: The following steps are involved: S1: Arrange a depth camera, collect depth images of the lying subject in time series through the depth camera, and transmit the collected data to a computer for processing; S2: Process the depth image data obtained from the depth camera, segment the background in the depth image, and mark the position of the person in the depth image; S3: identifying key points of the human body, constructing a human posture model based on these key points of the human body, and displaying the image of the human posture model on the display in real time; S4: Calculating the positions of the key points of the human body in the constructed human body posture model, and identifying the motion state of the subject according to the relative position changes of the key points of the human body, including the following steps: S4.1: transform the coordinates of each key point data and map them to the global coordinate system; S4.2: Use the principle of digital filter to filter, deal with the overlapping and confusion problems in the movement of human key points in continuous frames, and pre-process the skeletal node trajectory to reduce the amplitude of node position jump; S4.3: Define the joints in the human posture model. Each joint is composed of three adjacent human key points as joint points. Construct the structural vectors between the joint points corresponding to each joint. Calculate the angle of each joint based on the structural vectors between the joint points. S4.4: Calculate the relative displacement of different joints in the same frame. Each joint in the same frame is represented by x i , the posture feature f of the current frame (c) cc , indicating that the coordinate difference of different joint points is: f cc =x i -x j |i,j=1,2,…N;i≠j} S4.5: Calculate the relative displacement of the same joint point in different frames, the offset of the current frame in the prone state relative to the previous frame (p), and calculate the coordinate difference of each joint point between the current frame and the previous frame to capture the joint motion characteristics, expressed as: S4.6: The offset of the current frame in motion relative to the initial frame (s). The coordinate difference of each joint point between the current frame and the initial frame is calculated as the offset feature, expressed as f cs : S4.7: For posture feature f cc , motion characteristics cp , offset feature f cs The combined form of each joint position information in each frame can be expressed as: c =f cc ,f cp ,f cs ], for f c Perform dimensionality reduction and make judgment with the set value. If the angle change information exceeds the set threshold, the angle change information is discarded. If the angle change information falls within the set threshold, the angle change information is retained. S4.8: For each joint angle information within a specific time window, the amount of information is represented by entropy, and the information entropy h(x) is expressed as: h(x)=-∫f(x)logf(x)dx The entropy of the random variable is proportional to the logarithm of its variance, which is used to measure the joint angle information in each time period. The joint angle information is input into the behavior classifier of the support vector machine to identify the current movement state of the subject; S5: Display the identified current motion state of the subject on a display.

2. The method for real-time detection and recognition of a human body lying posture according to claim 1, characterized in that: Step S2 includes the following steps: S2.1: According to the relationship between focal length and object distance, a reference depth value s0 is set as the threshold for determining whether the image belongs to the foreground or the background; S2.2: The depth image data collected by the depth camera is normalized to determine the scale of background segmentation. The normalization formula is: Where, is the original depth data of each point in the image, s max is the maximum original depth value of the points contained in the image, s min is the minimum original depth value of the points contained in the image; V represents the depth value normalized to the range of 0-255 for the original depth data of each point in the image; According to the above formula, V0 is obtained, where V0 represents the depth value at the focus after s0 is normalized; S2.3: Perform preliminary stratification on the depth image. When 0≤V≤V0, it is divided into the background layer; when V0≤V≤2V0, it is divided into the transition layer; when 20≤V≤255, it is divided into the foreground layer; S2.4: Scale the depth information of each layer in the depth image, mark the foreground layer as 0, the transition layer as 1, and the background layer as 2; then perform edge smoothing preprocessing to maintain a natural transition of the image edge; the scale label of each point in the smoothed image is between 0 and 2, and the blur scale of the blur processing of each point in the image is determined according to the scale label, where the blur radius R(i,) of each point is determined according to the following formula: R(i,)=(i,j)×k S2.5: Calculate the Gaussian standard deviation and blur radius, and use the following formula to perform Gaussian blur processing of different scales on each point in the image to obtain a clear prone subject image: in is the convolution operation, and f(i,j,σ(i,j)) is the Gaussian distribution function.

3. The method for real-time detection and recognition of a human body lying posture according to claim 1, characterized in that: The key points of the human body include: head, nose, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hand, right hand, chest, abdomen, buttocks, left waist, right waist, left knee, right knee, left ankle, right ankle, left foot and right foot.

4. The method for real-time detection and recognition of a human body lying posture according to claim 1, characterized in that: In step 3, a joint regression network is used to identify key points of the human body in the depth image processed in step S2. The specific steps are: S3.1: The joint regression network consists of a two-dimensional backbone network and three functional branches: plane transfer branch, depth estimation branch, and anchor proposal branch. S3.2: The plane offset branch and the depth estimation branch are used to predict the 3D position of the key points of the human body in the depth image. For a certain key point, the output of the aggregated anchor point is used to locate it. S3.3: The anchor proposal branch searches for information anchor points of specific key points by weight specification. The output layer of this branch is set to 16 K channels. S3.4: The backbone network is ResNet-50 pre-trained on ImageNet. The convolution stride in the fourth layer is set to 1. At the same time, the convolution operation of this layer is changed to dilated convolution with a dilation step of 2 to expand the receptive field. S3.5: For the key point j in the depth image, in order to make T ij and the depth target T dj Comparable, the Ground Truth depth Z of the key point j Translates to: T j d =μ(Z j -i) Among them, μ and θ are transformation parameters. For the requirement of lying posture recognition, μ is set to 1 and θ is set to 0; S3.6: The A2J model is trained under the joint supervision of two loss functions: key point estimation loss and information anchor loss. The objective loss of key point loss is expressed as: where α is a factor that balances the task of smoothing the offset and depth estimation and is set to 0.5, τ1 and τ2 are set to 1 and 3 to smooth the relative noise in the depth map, and T j i and T j d is the in-plane and depth target position of key point j, and the smoothing loss function L τ The formula is: The surrounding information anchor loss is used to improve the generalization ability of the model so that key points can be observed from multiple angles. The information anchor point formula around the loss is: Therefore, the end-to-end learning process is determined by two loss functions, as follows: loss=λloss1+loss2 λ is the loss weight factor, which is set to 3 to balance loss 1 and loss 2.

5. The method for real-time detection and recognition of a human body lying posture according to claim 1, characterized in that: In step 3, considering that the contribution of each anchor point to the key point is not the same, the plane position and depth value of the key point j can be used as the weighted average of the anchor point output. The specific formula is as follows: In the formula, S j represents the plane position of key point j, D j represents the estimated depth of key point j; P j (a) is the normalized weight value of anchor point a through all anchor points toward key point j, and is obtained using SoftMax through the following formula:

6. The method for real-time detection and recognition of a human body lying posture according to claim 1, characterized in that: The images collected by the depth camera are encrypted and stored in a chaotic logistic manner to ensure the privacy of the person being detected; the specific steps are shown in S1.1-S1.4: S1.1: Calculate the size of the original image for a frame to be saved and represent it in the matrix [M, N]; S1.2: For the matrix [M, N], generate two Logistic chaotic sequences, transform these two Logistic sequences, and obtain two y sequences; S1.3: Encrypt the image by replacing the values ​​with the two sequences y1 and y2; S1.4: Save y1, y2 sequences and encrypted pixels.

Images