Human body data measurement method and device, electronic equipment and storage medium

Abstract

The invention discloses a human body data measurement method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a source point cloud of a target scene, a depth image of a target object in the target scene and a corresponding target point cloud through camera equipment; performing data comparison on the source point cloud and the target point cloud to obtain a difference point cloud; preprocessing the difference point cloud to obtain human body point cloud data; obtaining a human body pixel value of the target object in the depth image based on the human body point cloud data; and obtaining human body data of the target object based on the human body pixel value and a preset calibration value. Through efficient data acquisition, processing and analysis, accurate acquisition of the human body data is realized, the accuracy and reliability of the data are improved, the applicability and safety of the technology are enhanced, powerful technical support is provided for development of related fields, and the method can be widely applied to the technical field of data processing.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a human body data measurement method, device, electronic equipment and storage medium. Background Art

[0002] With the continuous development of smart home and health monitoring technology, people's demand for smart mattresses is increasing. Traditional smart mattress systems collect users' body data through built-in pressure sensors. However, this method has some shortcomings, especially during the store experience.

[0003] Currently, stores usually use manual measurement or standing measurement with a depth camera to obtain customers' body data before they can experience the mattress. Manual measurement is not only cumbersome, but also prone to errors. Although the traditional standing measurement is more accurate, the whole process is longer, which affects the customer experience. In addition, users need to manually enter their body data after measurement, which increases the complexity and time cost of the operation. Summary of the invention

[0004] The main purpose of the embodiments of the present invention is to provide a method, device, electronic device and storage medium for measuring human body data, in order to solve at least one problem of the prior art. The present invention can efficiently measure human body data.

[0005] To achieve the above object, an embodiment of the present invention provides a method for measuring human body data, the method comprising:

[0006] Collecting a source point cloud of a target scene and a depth image of a target object in the target scene and its corresponding target point cloud by a camera device;

[0007] Compare the source point cloud and the target point cloud to obtain the difference point cloud;

[0008] Preprocess the difference point cloud to obtain human body point cloud data;

[0009] Based on the human point cloud data, obtain the human pixel value of the target object in the depth image;

[0010] The human body data of the target object is obtained based on the human body pixel value combined with the preset calibration value.

[0011] In some embodiments, before the step of collecting the target point cloud of the target object in the target scene by the camera device, the method further includes the following steps:

[0012] Collecting posture data of the target object in the target scene through a camera device;

[0013] Extract multiple key points of the target object based on the posture data, and perform posture detection according to the spatial relationship between the key points;

[0014] When the result of the posture detection is failed, an adjustment reminder is issued to the target object, and after a preset period, the step of collecting posture data of the target object in the target scene through the camera device is returned to execute until the result of the posture detection is passed, and a maintenance reminder is issued to the target object.

[0015] In some embodiments, performing data comparison on the source point cloud and the target point cloud to obtain a difference point cloud includes the following steps:

[0016] Use Euclidean distance to detect and compare the source point cloud and the target point cloud to obtain the change points;

[0017] The difference point cloud is obtained by sorting all the change points between the source point cloud and the target point cloud.

[0018] In some embodiments, preprocessing the difference point cloud includes at least one of the following steps:

[0019] Use straight-through filtering to denoise the difference point cloud;

[0020] The RGB image of the target object in the target scene is collected by a camera device, and the point cloud data is compensated for the difference point cloud based on the RGB image.

[0021] In some embodiments, performing point cloud data compensation on the difference point cloud based on the RGB image includes the following steps:

[0022] The target color pixels in the RGB image are detected and marked by the color threshold segmentation method to obtain a binary mask image;

[0023] Based on the binary mask image and the difference point cloud, the iterative closest point algorithm is used for registration and completion.

[0024] In some embodiments, obtaining a human body pixel value of a target object in a depth image based on human body point cloud data includes the following steps:

[0025] Perform human posture recognition based on depth images to obtain the initial points of each part of the target object;

[0026] Based on the initial points, the boundary points of the target object are verified by combining the human body point cloud data; the boundary points include the head vertex, the sole position, and the left and right boundary points;

[0027] Based on the initial point position and the boundary point position, the human body pixel values ​​of each body range of the target object are obtained.

[0028] In some embodiments, obtaining the human body pixel values ​​of each body range of the target object based on the initial point and the boundary point includes the following steps:

[0029] Based on the camera intrinsic parameters of the camera device, the pixel width value and the pixel height value of the depth image are obtained in sequence in combination with the depth imaging width and the depth imaging height of the camera device;

[0030] Based on the initial point and the boundary point, the point distance corresponding to each body range of the target object is obtained;

[0031] Based on the point distance, combined with the pixel width value or the pixel height value, the human body pixel value of each body range of the target object is obtained.

[0032] To achieve the above object, another aspect of an embodiment of the present invention provides a human body data measurement device, the device comprising:

[0033] The first module is used to collect a source point cloud of a target scene and a depth image of a target object in the target scene and its corresponding target point cloud through a camera device;

[0034] The second module is used to compare the source point cloud and the target point cloud to obtain the difference point cloud;

[0035] The third module is used to pre-process the difference point cloud to obtain human body point cloud data;

[0036] The fourth module is used to obtain the human body pixel value of the target object in the depth image based on the human body point cloud data;

[0037] The fifth module is used to obtain the human body data of the target object based on the human body pixel value combined with the preset calibration value.

[0038] In some embodiments, the apparatus further includes the following module before the step of collecting a depth image of the target object in the target scene and its corresponding target point cloud by a camera device in the first module:

[0039] The sixth module is used to collect the posture data of the target object in the target scene through a camera device;

[0040] The seventh module is used to extract multiple key points of the target object based on the posture data, and perform posture detection according to the spatial relationship between the key points;

[0041] The eighth module is used to send an adjustment reminder to the target object when the result of the posture detection is failed, and return to the step of collecting the posture data of the target object in the target scene through the camera device after a preset period, until the result of the posture detection is passed, and send a maintenance reminder to the target object.

[0042] To achieve the above objective, another aspect of an embodiment of the present invention provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above method when executing the computer program.

[0043] To achieve the above objective, another aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above method is implemented.

[0044] The present invention provides a human body data measurement method, device, electronic device and storage medium. The scheme uses a camera to collect a source point cloud of a target scene and a depth image of a target object in the target scene and its corresponding target point cloud; compares the source point cloud and the target point cloud to obtain a difference point cloud; pre-processes the difference point cloud to obtain human body point cloud data; obtains the human body pixel value of the target object in the depth image based on the human body point cloud data; obtains the human body data of the target object based on the human body pixel value combined with a preset calibration value. The embodiments of the present invention include at least the following beneficial effects:

[0045] Efficient and accurate human data acquisition: The camera equipment collects the source point cloud of the target scene and the depth image and target point cloud of the target object, realizing comprehensive and high-precision data capture of the scene and the target object. Data comparison generates a difference point cloud, further focusing on human-related data, effectively filtering out background interference, and improving the accuracy and efficiency of human data extraction.

[0046] Optimize the data preprocessing process: Preprocess the difference point cloud to obtain pure human body point cloud data. This process reduces noise and redundant information, laying a solid foundation for subsequent human body data analysis. The preprocessing step helps to improve data quality and make the final human body data more reliable.

[0047] Accurate conversion of human pixels to human data: Based on human point cloud data, the human pixel values ​​of the target object in the depth image can be accurately obtained. This conversion process utilizes advanced algorithms and technologies to ensure accurate mapping from pixel level to actual human size, shape and other data, providing strong support for applications such as human body measurement and posture analysis.

[0048] Non-contact measurement, high safety: The entire technical solution is based on data collection by camera equipment, without the need for direct contact with the target object, avoiding the inconvenience or safety risks that may be caused by traditional measurement methods. It is particularly suitable for occasions where social distance needs to be maintained or physical contact needs to be avoided.

[0049] In summary, the present invention achieves accurate acquisition of human body data through efficient data collection, processing and analysis, which not only improves the accuracy and reliability of the data, but also enhances the applicability and safety of the technology, providing strong technical support for the development of related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flow chart of a human body data measurement method provided by an embodiment of the present invention;

[0051] Figure 2 It is a schematic diagram and a scene rendering diagram of an example of a bird's-eye view scene of human body data measurement provided by an embodiment of the present invention;

[0052] Figure 3 is a schematic diagram of an example of a side-view scene of human body data measurement provided by an embodiment of the present invention;

[0053] Figure 4 is a schematic diagram of an example of a PC terminal operation interface provided by an embodiment of the present invention;

[0054] Figure 5 is a schematic diagram of the overall business process of the human body data measurement method provided by an embodiment of the present invention;

[0055] Figure 6 is a schematic structural diagram of a human body data measuring device provided by an embodiment of the present invention;

[0056] Figure 7 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present invention. They are only examples of devices and methods consistent with some aspects of the embodiments of the present invention as detailed in the attached claims.

[0058] It is understood that the terms "first", "second", etc. used in the present invention may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0059] The terms "at least one", "multiple", "each", "any", etc. used in the present invention, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0061] The human body data measurement method provided by the embodiment of the present invention relates to the field of data processing technology. The human body data measurement method provided by the embodiment of the present invention can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or it can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms and other basic cloud computing services. The cloud server, the server can also be a node server in the blockchain network; the software can be an application that implements the human body data measurement method, etc., but is not limited to the above forms.

[0062] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0063] Figure 1 is an optional flow chart of a human body data measurement method provided by an embodiment of the present invention, Figure 1 The method may include but is not limited to steps S100 to S500.

[0064] S100, collecting a source point cloud of a target scene and a depth image of a target object in the target scene and its corresponding target point cloud by a camera device;

[0065] For example, in some specific implementations, the technology of the present invention can measure human body data in the application scenario when the user is lying down, and the camera device may include a depth camera and a conventional color camera. Specifically, taking the human body data measurement in the scenario of the user lying down on the bed as an example, the collection process of step S100 can be implemented as follows:

[0066] like Figure 2 (The left side is a schematic diagram of an example of a bird's-eye view of the scene, and the right side is a rendering of the actual application scene) and Figure 3 As shown in (Schematic diagram of side view scene example), the depth camera can be placed 2 to 2.5 meters above the bed and about 85 to 90 cm away from the bed screen, so that the field of view of the depth camera can cover the entire mattress area, thereby obtaining the point cloud data of the depth camera:

[0067] Specifically, a depth camera based on 3D structured light imaging technology can be used to obtain a depth image of the object, and a color camera can be used to collect a color image of the object. Then, the SDK corresponding to the depth camera can be applied to obtain the depth point cloud data, the camera's intrinsic parameters fx, fy and distortion factor x, and the width (w) and height (h) of the camera's actual pixel points (the calculation of the depth camera's properties will be described in detail later).

[0068] Finally, cache the point cloud data:

[0069] The point cloud data obtained by the depth camera can be stored in the computer data background, and the cache export of the point cloud data can be realized through the corresponding cache instructions of the administrator (it can also be realized through an automated process), such as Figure 4 As shown, by clicking the reset button of the relevant software on the PC, the deep point cloud data model deepCatch of the smart bed and its surroundings (marked as source point cloud, used as data) and the deep point cloud data (bodyDeepCatchArray) of the user lying in the corresponding scene can be cached.

[0070] Among them, 10 copies (the configuration can be adjusted based on actual needs) of deep point cloud data (bodyDeepCatchArray) can also be obtained, and the point cloud data bodyDeepCatch (the target point cloud obtained) can be obtained after averaging and processing the data.

[0071] Among them, in some embodiments, before the step of collecting the target point cloud of the target object in the target scene through the camera device, the method may also include the following steps: collecting the posture data of the target object in the target scene through the camera device; extracting multiple key points of the target object based on the posture data, and performing posture detection according to the spatial relationship between the key points; when the result of the posture detection is failed, issuing an adjustment reminder to the target object, and returning to the step of collecting the posture data of the target object in the target scene through the camera device after a preset period, until the result of the posture detection is passed, and issuing a maintenance reminder to the target object.

[0072] For example, in some specific implementations, the implementation process of posture detection may be as follows:

[0073] 1. Test of the smart mattress with the test subject lying down:

[0074] The test subject lies down naturally with his hands flat on his abdomen, and those around him who may affect him should stay away from the test area.

[0075] 2. Segmentation of the depth point cloud data of the tested person:

[0076] 1. Obtain the human body posture of the person being tested: The depth camera obtains the user's lying posture data.

[0077] 2. Posture detection and prompts: The system detects the user's posture. If the posture fails the detection, the system will prompt the user to lie down again in the correct posture to ensure the reliability of the data.

[0078] Among them, posture detection supplements:

[0079] The key points of the human body can be obtained in real time through the human body recognition algorithm or the camera integrated with the human body recognition algorithm. The specific posture detection logic is as follows: take the two shoulders as a straight line a, the shoulders and elbows as a straight line b, and the elbows and wrists as a straight line c. The angle between ab and bc is recorded as ∠ab, ∠bc. The value of the angle ∠ab is in the range of {35,55}, and the value of the angle ∠ab is in the range of {100,80}. Similarly, the angle between the three points of the legs is kept in a certain range of {170,190}. The angle range of the arms and legs forms our fixed posture. Use voice prompts during the detection process (keep reminder: correct posture, please keep it for 5 seconds. Adjustment reminder: wrong posture, please try again).

[0080] S200, performing data comparison on the source point cloud and the target point cloud to obtain a difference point cloud;

[0081] It should be noted that, in some embodiments, performing data comparison on the source point cloud and the target point cloud to obtain a difference point cloud may include the following steps: detecting and comparing the source point cloud and the target point cloud using Euclidean distance to obtain change points; and obtaining a difference point cloud based on all change points between the source point cloud and the target point cloud.

[0082] For example, in some specific implementations, the Euclidean distance can be used to check and calculate the change of each point between the two point clouds. If a point in the target point cloud (bodyDeepCatch) does not find a corresponding neighbor point in the source point cloud (deepCatch), then the point is considered a change point. Save all change points as a difference point cloud (bodyDeepBefore)

[0083] S300, pre-processing the difference point cloud to obtain human body point cloud data;

[0084] It should be noted that, in some embodiments, preprocessing the difference point cloud may include at least one of the following steps:

[0085] Use straight-through filtering to denoise the difference point cloud;

[0086] Exemplarily, in some specific implementations, the difference point cloud data (bodyDeepBefore) is denoised using straight-through filtering to remove noise interference, improve data quality, and obtain purified point cloud data (bodyDeep).

[0087] The RGB image of the target object in the target scene is collected by the camera device, and the point cloud data is compensated for the difference point cloud based on the RGB image.

[0088] It should be understood that point cloud data compensation may also be performed on the basis of point cloud data after denoising.

[0089] Among them, in some embodiments, point cloud data compensation is performed on the difference point cloud based on the RGB image, including the following steps: detecting and marking the target color pixels in the RGB image by a color threshold segmentation method to obtain a binary mask image; combining the binary mask image with the difference point cloud, and performing alignment and completion using an iterative nearest point algorithm.

[0090] For example, in an actual scene, the person being tested will wear black shoes, pants and other clothing. Under the depth camera, black objects are often difficult to detect due to their low reflectivity, resulting in loss of point cloud data. Therefore, it is necessary to compensate the point cloud data (bodyDeep). In some specific implementations, taking the point cloud data compensation based on the point cloud data after denoising as an example, the specific method of point cloud data compensation is as follows:

[0091] Black clothing detection: Use RGB images to detect the pixel position of black objects through color threshold segmentation. The specific steps include:

[0092] 1. Convert the RGB image to HSV color space (HSV color space is more easily distinguishable from black and is a commonly used method).

[0093] 2. Set the HSV range threshold for black, for example [0,0,0] to [180,255,30].

[0094] 3. Use the threshold segmentation method to generate a binary mask image, in which the black object pixels are set to white (255) and other pixels are set to black (0).

[0095] Point cloud data processing:

[0096] (a) Traverse each pixel of the depth image corresponding to the point cloud data and use the binary mask image to determine whether the corresponding pixel belongs to a black object. If the value of the pixel in the mask image is 255, it means that the pixel belongs to a black object.

[0097] (b) For pixels belonging to black objects, their depth values ​​are extracted from the point cloud data (bodyDeep) and combined with the color information of the RGB image to generate point cloud data points. Each point includes X, Y, Z coordinates and RGB color values.

[0098] Point cloud data completion: RGB-D camera and point cloud data (bodyDeep) are registered and fused to generate complete black clothing point cloud data. Use iterative closest point (ICP algorithm: a commonly used point cloud registration algorithm used to align the coordinate systems of two or more point clouds. The principle is to optimize the matching error between two point clouds through iteration until the error is minimized) to register and merge data from multiple perspectives to obtain the final human point cloud data (realBodyDeep) as human posture data.

[0099] S400, obtaining a human body pixel value of a target object in a depth image based on human body point cloud data;

[0100] It should be noted that, in some embodiments, obtaining the human body pixel values ​​of the target object in the depth image based on the human body point cloud data may include the following steps: performing human body posture recognition based on the depth image to obtain the initial point positions of various parts of the target object; based on the initial point positions, combining the human body point cloud data to verify and obtain the boundary point positions of the target object; the boundary points include the vertex of the head, the soles of the feet, and the left and right boundary points; based on the initial point positions and the boundary points, obtaining the human body pixel values ​​of each body range of the target object.

[0101] Among them, in some embodiments, obtaining the human pixel values ​​of each body range of the target object based on the initial point position and the boundary point position may include the following steps: based on the camera intrinsic parameters of the camera device, combining the depth imaging width and the depth imaging height of the camera device to obtain the pixel width value and the pixel height value of the depth image in turn; based on the initial point position and the boundary point position, obtaining the point distance corresponding to each body range of the target object; based on the point distance, combining the pixel width value or the pixel height value to obtain the human pixel values ​​of each body range of the target object.

[0102] Exemplarily, in some specific implementations, the height, shoulder width, hip width and torso positions (such as the top position of the head, the position of the feet, the position of the shoulders, the left and right positions of the hips, and the upper and lower positions of the torso) can be determined based on the human posture data output by the depth camera to ensure accurate acquisition of key data.

[0103] Among them, the location information is supplemented as follows:

[0104] Head vertices: Usually located on the center line between the eyes, which makes it possible to more accurately infer the position of the head vertices through the eye position. You can first perform preliminary human body recognition on the depth image to obtain the position information of both eyes, and then get the head vertices based on the above rules combined with the point cloud data (realBodyDeep).

[0105] Foot position: You can first perform preliminary human body recognition on the depth image to obtain the position information of both heels, and then use the point cloud data (realBodyDeep) to search downward until the lowest point is obtained to obtain the position of the sole of the foot.

[0106] Shoulder position: The depth image can be directly used for human body recognition to obtain the position of the shoulders.

[0107] Hip position: You can first perform preliminary human body recognition on the depth image to obtain the points of the left and right hips, and then use the point cloud data (realBodyDeep) to find the farthest points to the left and right.

[0108] Torso position: Human body recognition can be performed directly on the depth image to obtain the key points of the neck and waist.

[0109] Based on the above identified points, the following can be achieved:

[0110] 1. Calculate the point distance (dw) between the height (head vertex and foot sole) / torso (neck and waist) on the X-axis of the depth image;

[0111] 2. Calculate the distance (dh) between the shoulder width (both shoulders) and hip width (the farthest points on the left and right hips) on the Y axis of the depth image;

[0112] 3. The formula for obtaining pixel width is:

[0113] 4. The formula for obtaining pixel height value is:

[0114] 5. The formula for obtaining height and torso pixel length is plx = dw pw;

[0115] 6. The pixel length formula of shoulder width and hip width is ply=dh·ph;

[0116] Among them, explanation of the formula symbols: fx, fy, x are the intrinsic parameters of the depth camera; w and h are the width and height of the depth imaging; dw represents the point distance obtained on the x-axis, such as height and torso; dh represents the point distance obtained on the y-axis, such as shoulder width and hip width; plx and ply are the pixel lengths of height, torso, shoulder width, and hip width.

[0117] S500: Obtaining human body data of the target object based on human body pixel values ​​combined with preset calibration values.

[0118] For example, in some specific implementations, a calibration value (calibration) from a depth camera to a smart mattress is obtained, and then:

[0119] The formula for obtaining the real data of height and trunk through calibration value is bodyData = calibration·plx;

[0120] The actual data formula for obtaining shoulder width and hip width through calibration values ​​is bodyData = calibration·ply;

[0121] Among them, in some specific application scenarios, the technical solution of the present invention visualizes the measured human body data (for example, organizes and sends it to a PC for display). Specifically, in an application scenario where the user is lying on the bed, when the mattress on which the user is lying is a smart mattress, the sensor data of the sensors provided in the smart mattress (for example, real-time pressure data when the user is lying) can also be visualized.

[0122] In order to explain the principle of the technical solution of the present invention in detail, the overall process of the present invention is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and cannot be regarded as a limitation of the present invention.

[0123] The human body data measurement method provided in the embodiment of the present invention can be specifically implemented through the following process:

[0124] First of all, it should be noted that existing smart mattresses mainly rely on pressure sensors to detect the pressure distribution of users on the mattress, and adjust it by controlling the inflation and deflation of air bags. However, this method can only provide local pressure data, which cannot fully reflect the user's physical characteristics and needs, and it is difficult to achieve personalized comfort adjustment.

[0125] In actual scenarios, the bed may be affected by pillows, foot pads, sheets, etc., and the plane segmentation algorithm cannot accurately segment the human body point cloud data. The person lying down for measurement may wear black shoes, pants, and other clothing. Under the depth camera, black objects are often difficult to detect due to their low reflectivity, resulting in the loss of point cloud data.

[0126] In order to optimize this process and improve the customer experience, the present invention proposes a method for measuring the body data of a lying human body based on a depth camera. The method can quickly and accurately obtain the user's body data when the user lies on the mattress, and automatically input the data into the smart mattress system, so as to achieve instant and personalized adjustment and improve the customer's experience.

[0127] like Figure 5 As shown, the overall process of the present invention can be implemented as follows:

[0128] 1. Depth camera position:

[0129] like Figure 2 and Figure 3As shown, the depth camera can be placed 2 to 2.5 meters above the bed and about 85 to 90 cm away from the bed screen, so that the field of view of the depth camera can cover the entire mattress area.

[0130] 2. Get the point cloud data of the depth camera:

[0131] A depth camera based on 3D structured light imaging technology is used to obtain the depth image of the object, and a color camera is used to collect the color image of the object. Then, the SDK corresponding to the depth camera is applied to obtain the depth point cloud data, the camera's internal parameters fx, fy and distortion factor x, and the width (w) and height (h) of the actual pixel points of the camera (the calculation of the properties of the depth camera will be described in detail later).

[0132] 3. Cache point cloud data:

[0133] The point cloud data obtained by the depth camera can be stored in the computer data background, and the cache export of the point cloud data can be realized through the corresponding cache instructions of the administrator (it can also be realized through an automated process), such as Figure 4 As shown, you can click the reset button of the relevant software on the PC to cache the deep point cloud data model deepCatch (marked as source point cloud, used as data).

[0134] The point cloud data cached here includes the point cloud data of pillows and sheets.

[0135] 4. Test of the smart mattress with the subject lying down:

[0136] The test subject lies down naturally with his hands flat on his abdomen, and those around him who may affect him should stay away from the test area.

[0137] 5. Segmentation of the depth point cloud data of the tested person:

[0138] 1. Obtain the human body posture of the person being tested: The depth camera obtains the user's lying posture data.

[0139] 2. Posture detection and prompts: The system detects the user's posture. If the posture fails the detection, the system will prompt the user to lie down again in the correct posture to ensure the reliability of the data.

[0140] Among them, posture detection supplements:

[0141] The key points of the human body can be obtained in real time through the human body recognition algorithm or the camera integrated with the human body recognition algorithm. The specific posture detection logic is as follows: take the two shoulders as a straight line a, the shoulders and elbows as a straight line b, and the elbows and wrists as a straight line c. The angle between ab and bc is recorded as ∠ab, ∠bc. The value of the angle ∠ab is in the range of {35,55}, and the value of the angle ∠ab is in the range of {100,80}. Similarly, the angle between the three points of the legs is kept in a certain range of {170,190}. The angle range of the arms and legs forms our fixed posture. Use voice prompts during the detection process (keep reminder: correct posture, please keep it for 5 seconds. Adjustment reminder: wrong posture, please try again).

[0142] 3. Posture detection passed: After the posture detection is passed, the system starts a 5-second countdown, during which 10 deep point cloud data (bodyDeepCatchArray) are obtained, and the point cloud data bodyDeepCatch (the target point cloud) is obtained after averaging and processing the data.

[0143] 4. Data comparison: Use Euclidean distance to check and calculate the change of each point between the two point clouds. If a point in the target point cloud (bodyDeepCatch, the point cloud data obtained in the previous step) does not find a corresponding neighbor point in the source point cloud (deepCatch, the source of the cached point cloud data obtained in the third step), then the point is considered a change point. Save all change points as a difference point cloud (bodyDeepBefore).

[0144] 5. Denoising: Use straight-through filtering to denoise the difference point cloud data (bodyDeepBefore) to remove noise interference, improve data quality, and obtain the purified point cloud data (bodyDeep).

[0145] 6. Point cloud data compensation: In actual scenarios, the subjects will wear black shoes, pants and other clothing. Under the depth camera, black objects are often difficult to detect due to their low reflectivity, resulting in point cloud data loss. Therefore, it is necessary to compensate the point cloud data (bodyDeep). The specific method is as follows:

[0146] Black clothing detection: Use RGB images to detect the pixel position of black objects through color threshold segmentation. The specific steps include:

[0147] (a) Convert the RGB image to the HSV color space (HSV color space is more easily able to distinguish black and is a commonly used method).

[0148] (b) Set the HSV range threshold for black, for example, [0,0,0] to [180,255,30].

[0149] (c) The threshold segmentation method is used to generate a binary mask image, where the black object pixels are set to white (255) and other pixels are set to black (0).

[0150] Point cloud data processing:

[0151] (a) Traverse each pixel of the depth image corresponding to the point cloud data and use the binary mask image to determine whether the corresponding pixel belongs to a black object. If the value of the pixel in the mask image is 255, it means that the pixel belongs to a black object.

[0152] (b) For pixels belonging to black objects, their depth values ​​are extracted from the point cloud data (bodyDeep) and combined with the color information of the RGB image to generate point cloud data points. Each point includes X, Y, Z coordinates and RGB color values.

[0153] Point cloud data completion: RGB-D camera and point cloud data (bodyDeep) are registered and fused to generate complete black clothing point cloud data. Use iterative closest point (ICP algorithm: a commonly used point cloud registration algorithm used to align the coordinate systems of two or more point clouds. The principle is to optimize the matching error between two point clouds through iteration until the error is minimized) to register and merge data from multiple perspectives to obtain the final human point cloud data (realBodyDeep) as human posture data.

[0154] 6. Calculate the pixel length of body data

[0155] 1. According to the human posture data output by the depth camera, determine the height, shoulder width, hip width and torso points (such as the top position of the head, the position of the feet, the position of the shoulders, the left and right positions of the hips, and the upper and lower positions of the torso) to ensure the accurate acquisition of key data.

[0156] Additional information about the location:

[0157] Head vertices: Usually located on the center line between the eyes, which makes it possible to more accurately infer the position of the head vertices through the eye position. You can first perform preliminary human body recognition on the depth image to obtain the position information of both eyes, and then get the head vertices based on the above rules combined with the point cloud data (realBodyDeep).

[0158] Foot position: You can first perform preliminary human body recognition on the depth image to obtain the position information of both heels, and then use the point cloud data (realBodyDeep) to search downward until the lowest point is obtained to obtain the position of the sole of the foot.

[0159] Shoulder position: The depth image can be directly used for human body recognition to obtain the position of the shoulders.

[0160] Hip position: You can first perform preliminary human body recognition on the depth image to obtain the points of the left and right hips, and then use the point cloud data (realBodyDeep) to find the farthest points to the left and right.

[0161] Torso position: Human body recognition can be performed directly on the depth image to obtain the key points of the neck and waist.

[0162] 2. Calculate the point distance (dw) between the height (top of the head and the bottom of the foot) / torso (neck and waist) on the X-axis of the depth image;

[0163] 3. Calculate the distance (dh) between the shoulder width (both shoulders) and hip width (the farthest points on the left and right hips) on the Y axis of the depth image;

[0164] 4. The formula for obtaining pixel width is:

[0165] 5. The formula for obtaining pixel height value is:

[0166] 6. The formula for obtaining height and torso pixel length is plx = dw pw;

[0167] 7. The pixel length formula of shoulder width and hip width is ply=dh·ph;

[0168] Among them, explanation of the formula symbols: fx, fy, x are the intrinsic parameters of the depth camera; w and h are the width and height of the depth imaging; dw represents the point distance obtained on the x-axis, such as height and torso; dh represents the point distance obtained on the y-axis, such as shoulder width and hip width; plx and ply are the pixel lengths of height, torso, shoulder width, and hip width.

[0169] 7. Calculate real body data:

[0170] 1. Get the calibration value from the depth camera to the smart mattress;

[0171] 2. The formula for obtaining the real data of height and torso through calibration value is bodyData = calibration·plx;

[0172] 3. The formula for obtaining the real data of shoulder width and hip width through calibration value is bodyData = calibration·ply;

[0173] 8. Output body data bodyData to the smart mattress;

[0174] 9. The PC terminal that obtains the adjustment gear, real-time pressure value and recommended mattress display through the algorithm of the smart mattress;

[0175] 10. The test (measurement) is completed.

[0176] In summary, the present invention provides a method for measuring the body data of a lying person based on a depth camera, which can quickly and accurately obtain the height, shoulder width, hip width, and torso of the person being measured, and present the body data information of the person being measured on a terminal device, thereby improving the customer interaction experience. Compared with the prior art, the present invention has at least the following beneficial effects:

[0177] While experiencing the mattress, customers can use a simple, contactless, high-precision and real-time body data measurement method, which provides powerful data support and automatic adjustment capabilities for the smart mattress system, significantly improving customers' sleep quality and health management level.

[0178] In the terminal stores, in order to better demonstrate the advantages of smart mattresses, a test system is formed with a depth camera with a smart mattress as the core, and combined with the Demoiselle Tidal Algorithm, a "Customer Mattress Demand Report" is output in real time. The depth camera accurately collects the user's body data in a contactless manner, including body shape, posture and physiological parameters. Based on this data, the Demoiselle Tidal Algorithm calculates the best mattress adjustment plan and displays the adjustment process of the smart mattress in real time. This system can not only help customers intuitively experience the functions of smart mattresses, but also provide personalized mattress adjustment suggestions to optimize customers' purchasing decisions and user experience.

[0179] In this way, consumers can experience the automatic adjustment effect of smart mattresses on site and understand how smart mattresses provide customized comfort based on personal data, so that they can have more trust and recognition of the innovative technology and practical value of smart mattresses.

[0180] See also Figure 6 The embodiment of the present invention further provides a human body data measurement device 900, which can implement the above human body data measurement method, and the device includes:

[0181] The first module 901 is used to collect a source point cloud of a target scene and a depth image of a target object in the target scene and its corresponding target point cloud through a camera device;

[0182] The second module 902 is used to compare the source point cloud and the target point cloud to obtain a difference point cloud;

[0183] The third module 903 is used to pre-process the difference point cloud to obtain human body point cloud data;

[0184] The fourth module 904 is used to obtain the human body pixel value of the target object in the depth image based on the human body point cloud data;

[0185] The fifth module 905 is used to obtain the human body data of the target object based on the human body pixel value combined with the preset calibration value.

[0186] In some embodiments, the apparatus may further include the following module before the step of acquiring a depth image of the target object in the target scene and its corresponding target point cloud by a camera device in the first module:

[0187] The sixth module is used to collect the posture data of the target object in the target scene through a camera device;

[0188] The seventh module is used to extract multiple key points of the target object based on the posture data, and perform posture detection according to the spatial relationship between the key points;

[0189] The eighth module is used to send an adjustment reminder to the target object when the result of the posture detection is failed, and return to the step of collecting the posture data of the target object in the target scene through the camera device after a preset period, until the result of the posture detection is passed, and send a maintenance reminder to the target object.

[0190] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0191] The embodiment of the present invention further provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned human body data measurement method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0192] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0193] See also Figure 7 , Figure 7 The hardware structure of an electronic device 1000 of another embodiment is illustrated. The electronic device includes:

[0194] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0195] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 is called to execute the human body data measurement method of the embodiment of the present invention;

[0196] Input / output interface 1003, used to implement information input and output;

[0197] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0198] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0199] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0200] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned human body data measurement method is implemented.

[0201] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0202] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0203] The human body data measurement method, human body data measurement device, electronic device and storage medium provided by the embodiment of the present invention obtain the test instruction of the target object; the test instruction includes the device model and the upgrade number; according to the device model, the target firmware package is downloaded; based on the device model, the communication module of the target mattress is scanned and processed; when the result of the scanning process is successful, the firmware version of the target mattress is obtained through the communication module; when the firmware version is less than or equal to the version information of the target firmware package, the upgrade protocol instruction is sent to the target mattress and the confirmation information replied by the target mattress is obtained; based on the confirmation information, the target firmware package is sub-packaged and sent to the target mattress, and the result of each sub-package sending is verified; when all sub-packages are sent, the upgrade protocol instruction is sent to the target mattress, and the confirmation information of ... If the results are all passed, a firmware verification instruction is sent to the target mattress, so that the target mattress performs a firmware upgrade according to the target firmware package based on the verification instruction; the upgrade result of the target mattress is obtained based on the upgrade reply value replied by the target mattress, and the number of tests is increased by 1; wherein the number of tests is initially 0; the upgrade result includes the upgrade test result, and the upgrade test result is an upgrade success or an upgrade failure; when the upgrade test result is an upgrade failure, the upgrade result also includes the failure reason; when the number of tests is less than the upgrade number, then return to execute the steps of sending the upgrade protocol instruction to the target mattress and obtaining the confirmation information replied by the target mattress, until the number of tests is equal to the upgrade number, and the test report is obtained according to all the upgrade results. The embodiment of the present invention greatly reduces the tediousness and error rate of manual operation by automatically obtaining the firmware version of the target mattress, sending the upgrade protocol instruction, sending the target firmware package in a sub-package, and performing firmware verification, thereby improving the efficiency and accuracy of the firmware upgrade; and through multiple upgrade tests (when the number of tests is less than the number of upgrades), the stability and reliability of the device system in various scenarios can be ensured, thereby enhancing the robustness of the entire device system. The embodiment of the present invention can efficiently perform human body data measurement.

[0204] The embodiments described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art can appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0205] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present invention and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0206] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0207] Those skilled in the art will appreciate that all or some of the steps, devices, and functional modules / units in the above disclosed methods may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0208] The terms "first", "second", "third", "fourth", etc. (if any) in the description of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0209] It should be understood that in the present invention, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can represent: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0210] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0211] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0212] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0213] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store programs.

[0214] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the embodiments of the present invention is not limited thereby. Any modification, equivalent substitution and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present invention shall be within the scope of the rights of the embodiments of the present invention.

Claims

1. A method for measuring human body data, characterized in that: The method comprises the following steps: Collecting a source point cloud of a target scene and a depth image of a target object in the target scene and its corresponding target point cloud by a camera device; Performing data comparison on the source point cloud and the target point cloud to obtain a difference point cloud; Preprocessing the difference point cloud to obtain human body point cloud data; Based on the human body point cloud data, obtaining a human body pixel value of the target object in the depth image; The human body data of the target object is obtained based on the human body pixel value combined with a preset calibration value.

2. The method according to claim 1, characterized in that Before the step of collecting the target point cloud of the target object in the target scene by means of a camera, the method further comprises the following steps: Collecting posture data of the target object in the target scene by the camera device; Extracting a plurality of key points of the target object based on the posture data, and performing posture detection according to the spatial relationship between the key points; When the result of the posture detection is failure, an adjustment reminder is issued to the target object, and after a preset period, the step of collecting the posture data of the target object in the target scene by the camera device is returned to execution until the result of the posture detection is passed, and a maintenance reminder is issued to the target object.

3. The method according to claim 1, characterized in that The step of performing data comparison on the source point cloud and the target point cloud to obtain a difference point cloud comprises the following steps: Detecting and comparing the source point cloud and the target point cloud using Euclidean distance to obtain change points; The difference point cloud is obtained by sorting all the change points between the source point cloud and the target point cloud.

4. The method according to claim 1, characterized in that The preprocessing of the difference point cloud comprises at least one of the following steps: De-noising the difference point cloud by using a straight-through filter; An RGB image of the target object in the target scene is captured by the camera device, and point cloud data compensation is performed on the difference point cloud based on the RGB image.

5. The method according to claim 4, characterized in that The step of performing point cloud data compensation on the difference point cloud based on the RGB image comprises the following steps: Detecting and marking target color pixels in the RGB image using a color threshold segmentation method to obtain a binary mask image; According to the binary mask image and the difference point cloud, an iterative closest point algorithm is used to perform registration and completion.

6. The method according to claim 1, characterized in that The method of obtaining a human body pixel value of the target object in the depth image based on the human body point cloud data comprises the following steps: Performing human posture recognition based on the depth image to obtain initial point positions of various parts of the target object; Based on the initial point position, the boundary point position of the target object is verified in combination with the human body point cloud data; the boundary point position includes the head vertex, the sole position and the left and right side boundary points; The human body pixel values ​​of each body range of the target object are obtained based on the initial point position and the boundary point position.

7. The method according to claim 6, characterized in that The step of obtaining the human body pixel values ​​of each body range of the target object based on the initial point and the boundary point comprises the following steps: Based on the camera intrinsic parameters of the camera device, in combination with the depth imaging width and the depth imaging height of the camera device, the pixel width value and the pixel height value of the depth image are obtained in sequence; Based on the initial point and the boundary point, obtaining the point distance corresponding to each body range of the target object; Based on the point distance and in combination with the pixel width value or the pixel height value, the human body pixel value of each body range of the target object is obtained.

8. A human body data measuring device, characterized in that: The device comprises: The first module is used to collect a source point cloud of a target scene and a depth image of a target object in the target scene and its corresponding target point cloud through a camera device; The second module is used to compare the source point cloud and the target point cloud to obtain a difference point cloud; The third module is used to pre-process the difference point cloud to obtain human body point cloud data; A fourth module is used to obtain a human pixel value of the target object in the depth image based on the human point cloud data; The fifth module is used to obtain the human body data of the target object based on the human body pixel value combined with a preset calibration value.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images