Background Segmentation Method and Device for Three-Dimensional Human Rotation Scanning Based on Depth Camera

Abstract

The present invention belongs to the technical field of digital information processing, and provides a background segmentation method and device based on three-dimensional human body rotation scanning by a depth camera. By obtaining a two-dimensional depth image generated by the depth camera when photographing a human body, the user stands still in a prescribed posture on a turntable during photographing, and the turntable drives the user to rotate. The collected depth image is pre-segmented by using two-dimensional image background segmentation technology, and according to the internal parameter calibration result of the depth camera, the pre-segmented depth image is converted into a three-dimensional point cloud. Based on the generated three-dimensional point cloud, a cutting cylinder is calculated, and the three-dimensional point cloud background segmentation is performed by using the cutting cylinder to generate a human body point cloud after background segmentation, so as to realize the elimination of background interference information in the human body depth image, improve the accuracy and integrity of background segmentation, reduce the segmentation difficulty and cost, and at the same time protect the privacy of the user.

Description

Technical Field

[0001] The present invention relates to the technical field of digital information processing, and more specifically, to a background segmentation method and device based on three-dimensional human body rotation scanning by a depth camera. Background Art

[0002] By using a three-dimensional human body scanning method based on a consumer depth camera, a three-dimensional digital model of the human body can be quickly reconstructed, which is widely used in fields such as clothing customization, cultural and creative industries, medical beauty, and film and television animation. In order to reconstruct a complete three-dimensional digital model of the human body, multiple depth cameras can be used to capture images of the human body simultaneously for reconstruction, or two depth cameras can be used, and the turntable is used to drive the human body to rotate to achieve 360° scanning. However, no matter which scanning method is adopted, there are more or less interference objects in the scene where the scanning device is used (such as other sundries near the scanning device, other people passing by during the scanning process, etc.). The existence of these interference objects will cause that in the two-dimensional images captured by the depth camera, in addition to the human body target object, there is also a part of background interference information. If these interference information cannot be effectively segmented from the images, they will affect the subsequent two-dimensional image and three-dimensional point cloud processing and calculation, and ultimately lead to the failure or abnormality of the reconstructed human body model. Therefore, it is necessary to perform background segmentation processing on the collected data to remove the background interference information other than the human body. The existing three-dimensional human body scanning background segmentation methods mainly adopt two-dimensional image segmentation techniques, and use techniques such as region growing and connected component segmentation to remove the background interference information other than the human from the captured original images. By using the image-based background segmentation technology, only the neighborhood information of the image pixels is utilized, and at the same time, it is relatively sensitive to the setting of segmentation parameters. For input images with complex backgrounds, problems such as segmentation errors or incomplete segmentation will occur. In addition, in the prior art, there are also cases of using deep learning segmentation methods to segment background interference information. The deep learning segmentation method mainly uses color images as input for background segmentation, usually requires a large number of samples for model training, and the acquisition of training samples is a challenging task. At the same time, since color information is required, this requires the depth camera to have a color information acquisition module, which increases the hardware cost of the depth camera, and since deep learning has a minimum requirement for hardware configuration, it will also increase the hardware cost of the scanning system. In particular, the deep learning segmentation method collects the facial color information of the human body, which is not conducive to the privacy protection of the scanned users.

[0003] In summary, the existing image background segmentation technologies have technical problems such as easy segmentation errors, incomplete segmentation, large segmentation challenges, high hardware costs, and being not conducive to the privacy protection of the scanned users. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a background segmentation method and device based on three-dimensional human body rotation scanning by a depth camera, aiming at the deficiencies existing in the above technical solutions, so as to improve the accuracy and integrity of background segmentation, and reduce the segmentation difficulty and cost.

[0005] In a first aspect, the present invention provides a background segmentation method based on three-dimensional human body rotation scanning by a depth camera. The background segmentation method based on three-dimensional human body rotation scanning by a depth camera includes the following steps:

[0006] Obtain a two-dimensional depth image generated by the depth camera when photographing a human body. When photographing, the user stands still in a specified posture on a turntable, and the turntable drives the user to rotate.

[0007] Adopt two-dimensional image background segmentation technology to perform pre-segmentation processing on the depth image, and convert the pre-segmented depth image into a three-dimensional point cloud according to the internal parameter calibration result of the depth camera.

[0008] Based on the generated three-dimensional point cloud, calculate a cutting cylinder, and use the cutting cylinder to perform background segmentation of the three-dimensional point cloud to generate a human body point cloud after background segmentation.

[0009] Further, the specified posture is the A-shaped posture; in the A-shaped posture, the included angle between a single arm of the user and the inner side of the body is 45°.

[0010] Further, the depth camera includes an upper depth camera and a lower depth camera; the upper depth camera and the lower depth camera synchronously collect the upper and lower body depth images of the human body to generate a set of the depth images, and multiple sets of the depth images are collected as the turntable drives the user to rotate.

[0011] Further, the step of adopting two-dimensional image background segmentation technology to perform pre-segmentation processing on the depth image and converting the pre-segmented depth image into a three-dimensional point cloud according to the internal parameter calibration result of the depth camera includes:

[0012] Adopt two-dimensional image background segmentation technology to perform pre-segmentation processing on multiple sets of the depth images in sequence.

[0013] Convert multiple sets of the pre-segmented depth images into multiple sets of three-dimensional point clouds according to the internal parameter calibration results of the upper depth camera and the lower depth camera.

[0014] Further, the step of calculating a cutting cylinder based on the generated three-dimensional point cloud and using the cutting cylinder to perform background segmentation of the three-dimensional point cloud to generate a human body point cloud after background segmentation includes:

[0015] Calculate a cutting cylinder based on the generated three-dimensional point cloud.

[0016] Using the cutting cylinder to perform 3D point cloud background segmentation on the multiple groups of 3D point clouds to generate human body point clouds after background segmentation.

[0017] Furthermore, adopting the 2D image background segmentation technology to perform pre-segmentation processing on the depth image, and converting the pre-segmented depth image into a 3D point cloud according to the internal parameter calibration result of the depth camera, including:

[0018] Primarily removing the background interference information in the non-human body region of the depth image through a depth threshold and a pixel valid region to obtain a depth value removed image;

[0019] Growing adjacent pixel points satisfying a certain depth threshold condition in the depth value removed image into one region to obtain a region growing image;

[0020] Performing contour detection on the region growing image to obtain a depth image of the initial human body region;

[0021] Filtering the edge information of the depth image of the initial human body region to obtain a filtered depth image;

[0022] Converting the filtered depth image into a 3D point cloud.

[0023] Furthermore, the primarily removing the background interference information in the non-human body region of the depth image through a depth threshold and a pixel valid region to obtain a depth value removed image includes:

[0024] Determining a maximum depth threshold d max and a minimum depth threshold d min ;

[0025] For the depth value d (u,v) of any pixel point (u, v) in the depth image, performing depth removal according to Formula 1, and the Formula 1 is:

[0026]

[0027] In the width direction of the depth image, setting the depth values of the pixel points with pixel coordinates in the u direction located in [0, u ths and [u wide - uths , u wide to 0, so as to recognize the depth information within a certain range on both the left and right sides of the depth image as background interference information and remove it, where u ths is set according to the resolution of the depth image;

[0028] The depth value removal image after removing the pixel valid area is denoted as I1.

[0029] Further, growing adjacent pixel points that meet certain depth threshold conditions in the depth value removal image into a region to obtain a region growing image includes:

[0030] For the seed point p in the depth value removal image I1 s , find its 8 neighborhood pixel points {p0,..., p7} according to the 8 - connectivity rule;

[0031] If the depth value difference between any neighborhood point p i and the seed point p s is less than a certain threshold d ths1 , satisfying {(dp i -dp s ) < d ths1 , i = 0,..., 7}, then it is determined that this neighborhood point and the seed point are a generating region;

[0032] In the same way, perform region growing judgment on each pixel point in the depth value removal image I1 to obtain k growing regions {R i , i = 0,..., k} of the depth value removal image I1, and further obtain the region growing image.

[0033] Further, performing contour detection on the region growing image to obtain the depth image of the initial human body region includes:

[0034] Calculate the rectangular outer contour {P i , i = 0,..., k} of the growing regions {R i , i = 0,..., k};

[0035] Based on the size and position of the outer contour {P i , i = 0,..., k}, screen out the growing region R p belonging to the user to be measured to obtain the depth image I2 of the initial human body region.

[0036] Further, filtering the edge information of the depth image of the initial human body region to obtain the filtered depth image includes:

[0037] Perform erosion and dilation operations on the edge information of the depth image I2 of the initial human body region to obtain the filtered depth image I3.

[0038] In a second aspect, the present invention provides a background segmentation device for three - dimensional human body rotation scanning based on a depth camera, including:

[0039] A two-dimensional image acquisition module, which is used to acquire a two-dimensional depth image generated by a depth camera photographing a human body. When photographing, the user stands still on a turntable in a specified posture, and the turntable drives the user to rotate;

[0040] A two-dimensional image background segmentation module, which is used to perform pre-segmentation processing on the depth image by using two-dimensional image background segmentation technology, and convert the pre-segmented depth image into a three-dimensional point cloud according to the internal parameter calibration result of the depth camera;

[0041] A three-dimensional point cloud background segmentation module, which is used to calculate a cutting cylinder based on the generated three-dimensional point cloud, and use the cutting cylinder to perform three-dimensional point cloud background segmentation to generate a human body point cloud after background segmentation.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0043] The present invention provides a background segmentation method and device for three-dimensional human body rotation scanning based on a depth camera. By acquiring a two-dimensional depth image generated by a depth camera photographing a human body, when photographing, the user stands still on a turntable in a specified posture, the turntable drives the user to rotate, perform pre-segmentation processing on the acquired depth image by using two-dimensional image background segmentation technology, and convert the pre-segmented depth image into a three-dimensional point cloud according to the internal parameter calibration result of the depth camera. Based on the generated three-dimensional point cloud, calculate a cutting cylinder, and use the cutting cylinder to perform three-dimensional point cloud background segmentation to generate a human body point cloud after background segmentation, so as to realize the elimination of background interference information in the human body depth image, improve the accuracy and integrity of background segmentation, reduce the segmentation difficulty and cost, and protect the privacy of users at the same time. Description of the Drawings

[0044] Figure 1 is a schematic flow chart of a background segmentation method for three-dimensional human body rotation scanning based on a depth camera of the present invention;

[0045] Figure 2 is a schematic architecture diagram of a data acquisition device of the present invention;

[0046] Figure 3 is a schematic flow chart of two-dimensional image background segmentation of the present invention;

[0047] Figure 4 is a schematic topological structure diagram of region growing of the present invention;

[0048] Figure 5 is a schematic flow chart of three-dimensional point cloud background segmentation of the present invention;

[0049] Figure 6 is a schematic flow chart of Euclidean distance clustering calculation of the present invention;

[0050] Figure 7 It is a schematic structural diagram of a background segmentation device based on three-dimensional human body rotation scanning by a depth camera according to the present invention;

[0051] Figure 8 It is a schematic structural diagram of a user terminal according to the present invention. Detailed implementation manners

[0052] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0053] It should be noted that if terms such as "first" and "second" appear in the description and claims of the present invention and the above-mentioned drawings, their purposes are to distinguish similar objects, and it is not necessary to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0054] Embodiment 1

[0055] Refer to Figures 1 to 7 , Embodiment 1 provides a background segmentation method based on three-dimensional human body rotation scanning by a depth camera, including step S1, step S2 and step S3. This embodiment provides a background segmentation method based on three-dimensional human body rotation scanning by a depth camera. By acquiring a two-dimensional depth image generated by the depth camera when photographing a human body, when photographing, the user stands still on the turntable in a specified posture, and the turntable drives the user to rotate. The collected depth image is pre-segmented by using two-dimensional image background segmentation technology, and according to the internal parameter calibration result of the depth camera, the pre-segmented depth image is converted into a three-dimensional point cloud. Based on the generated three-dimensional point cloud, a cutting cylinder is calculated, and the three-dimensional point cloud background segmentation is performed by using the cutting cylinder to generate a human body point cloud after background segmentation, so as to remove the background interference information of the human body depth image, improve the accuracy and integrity of background segmentation, reduce the segmentation difficulty and cost, and at the same time protect the privacy of the user.

[0056] It should be noted that the background segmentation method based on three-dimensional human body rotation scanning by a depth camera provided in this embodiment can run on the user side. The user side serves as the execution entity for all or part of the steps in the background segmentation method based on three-dimensional human body rotation scanning by a depth camera. In addition to being able to execute steps S1, S2, and S3 in this embodiment, it can also run part or all of the steps of the methods described below. Among them, the user side includes a memory, a processor, and a network interface that are communicatively connected to each other through a system bus. It should be pointed out that only some components of the user side are shown in the figure. However, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Among them, those skilled in the art of this technology can understand that the user side here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc. The user side can be a computing device such as a smart phone or a smart wearable device. The user side can interact with the user through a keyboard, a mouse, a remote control, a touchpad, or a voice control device, etc. The memory includes at least one type of readable storage medium. The readable storage medium includes flash memory, a hard disk, a multimedia card, a card-type memory (such as an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory can be an internal storage unit of the user side, such as the hard disk or memory of the user side. In other embodiments, the memory can also be an external storage device of the user side, such as a plug-in hard disk equipped on the user side, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Of course, the memory can also include both the internal storage unit and the external storage device of the user side.

[0057] See Figure 2, in this embodiment, the client further includes a data acquisition device, which includes: an upper depth camera (1), a lower depth camera (3), a fixing bracket (2), a floor mat (6), and a turntable (7). The upper depth camera (1) and the lower depth camera (3) are both monocular depth cameras, each including an infrared camera, a dot matrix projector, and a depth calculation processor. The depth cameras only collect depth information and do not collect color information. The field of view of the upper depth camera (1) is the first area (4), which can cover the upper body area of the human body (8) and is responsible for collecting the depth image of the upper body of the human body (8). The field of view of the lower depth camera (3) is the second area (5), which can cover the lower body area of the human body (8) and is responsible for collecting the depth image of the lower body of the human body (8). The fixing bracket (2) is used to fix the upper depth image camera (1) and the lower depth camera (3). A processor is installed inside the fixing bracket (2) for controlling the depth cameras to collect images, process data and transmit data, and for controlling the rotation and stop of the turntable. The left end of the floor mat (6) is aligned with the bottom of the fixing bracket (2) and is used to determine the fixed position of the turntable (7). A circular pattern for placing the turntable (7) is printed on the surface of the floor mat (6). By placing the turntable (7) within the circular pattern, the positional relationship of the turntable (7) relative to the fixing bracket (2) can be determined. The turntable (7) is used to drive the human body (8) to rotate clockwise around the axis of the turntable. During the rotation, the human body (8) keeps the hands in an A-shaped posture and the face faces the depth camera. During the rotation of the turntable (8), the upper depth camera (1) and the lower depth camera (3) alternately collect the depth images of the human body at different rotation positions.

[0058] It should be noted that since the upper depth camera (1) and the lower depth camera (3) are both monocular depth cameras, each including an infrared camera, a dot matrix projector, and a depth calculation processor, and the depth cameras only collect depth information and do not collect color information, the privacy of users can be protected.

[0059] Specifically in step S1, the client obtains the two-dimensional depth image generated by the depth camera photographing the human body. When photographing, the user stands still on the turntable in a specified posture, and the turntable drives the user to rotate.

[0060] It should be noted that the depth image captured by the depth camera for the human body is a two-dimensional image. The two-dimensional image is mainly captured by the upper depth camera (1) and the lower depth camera (3) alternately during the rotation of the turntable, and the depth image of the human body surface is obtained. The pixel value of the depth image represents the spatial distance between a certain position on the human body surface and the depth camera. The two-dimensional image acquisition process is as follows: 1. The human body (8) stands at the center position of the turntable, facing the upper and lower depth cameras, and keeps the hands in an A-shaped posture; 2. After receiving the first image acquisition instruction, the upper and lower depth cameras capture an upper and lower body depth image of the human body (8); 3. After receiving the start rotation instruction, the turntable (7) drives the human body to rotate clockwise around the axis of the turntable and stops after rotating 360°; 4. After the turntable (7) starts to rotate, the upper depth camera (1) receives the second image acquisition instruction and captures the second upper body depth image of the human body (8) by the upper depth camera (1); 5. After the second image acquisition instruction of the upper depth camera (1) is executed, the lower depth camera (3) receives the second image acquisition instruction and captures the second lower body depth image of the human body (8) by the lower depth camera (3). 6. Alternately repeat steps 4 and 5 until the turntable (7) stops, and the two-dimensional image acquisition of the upper and lower depth cameras is completed.

[0061] Specifically for step S2, the client uses two-dimensional image background segmentation technology to perform pre-segmentation processing on the depth image, and converts the pre-segmented depth image into a three-dimensional point cloud according to the internal parameter calibration result of the depth camera.

[0062] See Figure 3 , the two-dimensional image background segmentation shown includes depth value elimination, region growing, contour detection, edge erosion, and three-dimensional point cloud generation.

[0063] The depth value elimination is used to initially eliminate the background interference information in the non-human body area of the captured depth image through the depth threshold and the pixel valid area. The principle of depth value elimination is: according to the positional relationship between the turntable (7) and the fixed bracket (2), the maximum depth threshold d max and the minimum depth threshold d min are determined; for the depth value d (u,v) of any pixel point (u, v) in the depth image, depth elimination is performed according to Equation (1):

[0064]

[0065] The principle of pixel valid area elimination is: in the width direction of the depth image, for the pixel coordinates (u, v) in the u direction, the pixel coordinates are located in [0, u ths and [u wide - uths , u wide] is set to 0, that is, the depth information within a certain range on the left and right sides of the depth image is considered as background interference information and is removed. uths needs to be set according to the resolution of the depth image. For example, for a depth image with a resolution of 640×480, uths can be set to 120. The depth image after the effective pixel area is removed is recorded as I1.

[0066] As shown in Figure 4, the region growing is used to grow adjacent pixels that meet a certain depth threshold condition into a region. s , find its 8 neighboring pixels {p0,...,p7} according to the 8-connectivity rule; if any neighboring point p i The depth value of the seed point p s The depth difference is less than a certain threshold d ths1 , then the neighborhood point and the seed point are considered to be a generated area, that is, {(dp i -dp s )<d ths1 , i=0,...,7}. According to the same method, the region growth judgment is performed on each pixel point in the depth image I1, and then k growth regions {R i ,i=0,...,k}.

[0067] The contour detection is used to calculate the growth area {R i ,i=0,...,k}, the rectangular outer contour {P i , i=0,...,k}, based on the size and position of the outer contour, the growth area R belonging to the measured human body is screened out p . Taking the growth area R i Take the growth area R as an example to give the specific screening process: i For each pixel, calculate the maximum minimum point pass and Calculate the growth area R i The center point For a given boundary threshold [u dmin ,u dmax ] and [v dmin ,v dmax ],if and If the boundary threshold conditions are met in the u and v directions, the growth region R is determined. i The information representing the scanned human body (that is, the initial human body region segmented from the two-dimensional image) is I2.

[0068] The edge erosion is used to filter the edge information of I2. For the monocular depth camera used in the present invention, due to the limitation of the optical imaging principle, there are relatively large depth measurement errors in the effective depth edge region of the depth image. Therefore, it is necessary to remove the depth information in the edge region. In this embodiment, conventional edge erosion and dilation operations are used to erode the edge of I2 to obtain the processed depth image I3.

[0069] The three-dimensional point cloud generation is used to convert the depth image I3 into a three-dimensional point cloud PC1. The specific conversion process is as follows: Using the internal parameter calibration result (f x , f y , c x , c y ) of the depth camera, the three-dimensional point cloud PC1 is calculated using the formula shown in Equation (2). Most of the background interference information (non-human body area) in the point cloud PC1 has been removed, but there is still some background interference information that has not been removed. Therefore, "three-dimensional point cloud background segmentation" needs to be performed in the subsequent step S3.

[0070]

[0071] Among them, x represents the index value of the pixel point in the horizontal axis direction in the depth image I3, y represents the index value of the pixel point in the vertical axis direction in the depth image I3, and z represents the index value of the pixel point in the vertical direction in the depth image I3. c x represents the offset of the depth camera optical axis from the coordinate center of the projection plane in the x direction, and c y represents the offset of the depth camera optical axis from the coordinate center of the projection plane in the y direction, and f x represents the focal length of the depth camera in the x direction, and f y represents the focal length of the depth camera in the y direction.

[0072] Specifically in step S3, the client calculates a cutting cylinder based on the generated three-dimensional point cloud and uses the cutting cylinder for three-dimensional point cloud background segmentation to generate a human body point cloud after background segmentation.

[0073] As shown in FIG. 5, the three-dimensional point cloud background segmentation may include Euclidean distance clustering, segmentation parameter calculation, and background point cloud segmentation.

[0074] As Figure 6 shown, the calculation principle of the Euclidean distance clustering is specifically as follows: For the input three-dimensional point cloud PC1, first establish a kdtree of the point cloud, and select a seed point p s from it. According to the set threshold, find the k nearest neighbor points of the seed point p s ; then use the k nearest neighbor points as the initial cluster C'1, and then select from the initial cluster C'1 the points other than the seed point p sOther points outside are used as new seed points to perform k-nearest neighbor search again. If the threshold condition is met, the newly found k nearest neighbor points are added to the initial cluster C'1. If the number of points in the initial cluster C'1 no longer increases, it means that the growth of this cluster has stopped, and the final cluster C1 of the initial cluster C'1 is obtained. In the same way, other points outside the cluster C1 are selected from the three-dimensional point cloud PC1 as new seed points, and the above steps are repeated to obtain a set of clustering results {C i , i = 1,..., k}.

[0075] The above-mentioned segmentation parameter calculation takes the clustering results {C i , i = 1,..., k} as input, and calculates the number of points S i , the centroid O i , the maximum point p i and the minimum point p i_max of each cluster C i_min . Then, according to the following judgment rules, the clusters belonging to the human body point cloud are screened out:

[0076] If the number of points S i of the cluster C i is less than 2000, it is determined that this cluster is background interference information;

[0077] If the component value of the centroid O i of the cluster C i in the horizontal direction (Y direction, that is, the horizontal direction of the three-dimensional space where the three-dimensional point cloud is located) is greater than -0.45 m and less than 0.45 m, then this cluster belongs to the potential human body point cloud; further, if the maximum and minimum values of the cluster C i in the horizontal direction (Y direction) are located in [-0.45 m, 0.45 m], it is determined that the cluster C i is a human body point cloud cluster.

[0078] If the component value of the centroid O i of the cluster C i in the horizontal direction (Y direction) is greater than -0.45 m and less than 0.45 m, then this cluster belongs to the potential human body point cloud; further, if the minimum value of the cluster C i in the vertical direction (-X direction) is greater than -0.5 m, it is determined that the cluster C i is a human body point cloud cluster.

[0079] For each cluster in the cluster set {C i , i = 1,..., k}, it is judged according to the above judgment rules, and then the cluster point cloud C body belonging to the human body is obtained.

[0080] The calculation of the segmentation parameters includes calculating the axis of the cutting cylinder, calculating the center point of the cutting cylinder, and calculating the radius of the cutting cylinder.

[0081] According to Figure 2 the turntable (7) shown, using the lower depth camera (3), collect a turntable depth image and convert it into a turntable three-dimensional point cloud. Through plane fitting segmentation of the turntable three-dimensional point cloud, extract the three-dimensional point cloud of the turntable tabletop, and finally perform plane fitting on the turntable tabletop point cloud to obtain the normal vector n of the turntable tabletop. Table Take n Table as the axis direction of the cutting cylinder.

[0082] Using the human body clustering point cloud C body calculate its centroid O body as the center point of the cutting cylinder.

[0083] Using the human body clustering point cloud C body calculate its maximum and minimum values in the horizontal direction (Y direction). Based on the maximum and minimum values, calculate the arm width of the human body. Considering that the arm opening amplitudes of different users vary, add a margin (reference value 0.05 m) to the calculated arm width to obtain the final human body arm width W. body as the radius of the cutting cylinder. Considering that generally users open their hands in an A-shaped posture and the maximum arm width does not exceed 0.9 m, if the calculated user arm width exceeds 0.9 m, then by default, take 0.9 m.

[0084] For the background point cloud segmentation, mainly for each depth image collected during the rotary scanning process, respectively perform two-dimensional image background segmentation, three-dimensional point cloud generation, and cylinder background point cloud cutting to remove background interference information. For the two-dimensional image background segmentation, according to Figure 3 the "two-dimensional image background segmentation" method shown, perform background pre-segmentation on the depth image collected during the rotary scanning process, thereby removing most of the background interference information. The three-dimensional point cloud generation is used to calculate the three-dimensional point cloud from the depth image segmented in the previous step using the calibration result of the depth camera internal parameters. For the cylinder background point cloud cutting, using the three-dimensional point cloud obtained in the previous step as the input, based on the calculated cutting cylinder parameters (cylinder axis, cylinder center, cylinder radius), calculate the distance between each point in the input point cloud and the center of the cutting cylinder in the cutting cylinder radius direction. If the distance is greater than the cylinder radius, then determine this point as a background interference point. For each point in the input point cloud, perform distance judgment according to the above steps, and then screen out the background interference point set and delete them from the input point cloud. Finally, obtain the three-dimensional point cloud that only retains the human body, realize the background segmentation of the three-dimensional point cloud, and obtain the three-dimensional point cloud with only the human body part.

[0085] Embodiment 2

[0086] Refer to Figures 1 to 8 , Embodiment 2 provides a background segmentation device based on three-dimensional human body rotation scanning by a depth camera, including:

[0087] A two-dimensional image acquisition module, configured to acquire a two-dimensional depth image generated by the depth camera photographing a human body. When photographing, the user stands still in a specified posture on a turntable, and the turntable drives the user to rotate;

[0088] A two-dimensional image background segmentation module, configured to perform pre-segmentation processing on the depth image by using two-dimensional image background segmentation technology, and convert the pre-segmented depth image into a three-dimensional point cloud according to the internal parameter calibration result of the depth camera;

[0089] A three-dimensional point cloud background segmentation module, configured to calculate a cutting cylinder based on the generated three-dimensional point cloud, and use the cutting cylinder to perform three-dimensional point cloud background segmentation to generate a human body point cloud after background segmentation.

[0090] It should be noted that for the background segmentation device based on three-dimensional human body rotation scanning by a depth camera provided in this embodiment, by acquiring a two-dimensional depth image generated by the depth camera photographing a human body, when photographing, the user stands still in a specified posture on a turntable, the turntable drives the user to rotate, performing pre-segmentation processing on the collected depth image by using two-dimensional image background segmentation technology, and converting the pre-segmented depth image into a three-dimensional point cloud according to the internal parameter calibration result of the depth camera, calculating a cutting cylinder based on the generated three-dimensional point cloud, and using the cutting cylinder to perform three-dimensional point cloud background segmentation to generate a human body point cloud after background segmentation, so as to realize the elimination of background interference information in the human body depth image, improve the accuracy and integrity of background segmentation, reduce the segmentation difficulty and cost, and protect the privacy of the user at the same time.

[0091] It should be pointed out that the above embodiments are only preferred specific embodiments of the present invention, and the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. The protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A background segmentation method based on three-dimensional human body rotation scanning by a depth camera, characterized in that, The background segmentation method based on three-dimensional human body rotation scanning by a depth camera comprises the following steps: Obtain a two-dimensional depth image generated by the depth camera when photographing a human body. When photographing, the user stands still in a prescribed posture on a turntable, and the turntable drives the user to rotate; Adopt two-dimensional image background segmentation technology to perform pre-segmentation processing on the depth image, and convert the pre-segmented depth image into a three-dimensional point cloud according to the internal parameter calibration result of the depth camera; Based on the generated three-dimensional point cloud, calculate a cutting cylinder, and use the cutting cylinder to perform background segmentation of the three-dimensional point cloud to generate a human body point cloud after background segmentation; Using the two-dimensional image background segmentation technology, pre-segment the depth image, and convert the pre-segmented depth image into a three-dimensional point cloud according to the internal parameter calibration result of the depth camera, including: initially removing the background interference information in the non-human region of the depth image through a depth threshold and a pixel valid region to obtain a depth value removed image; growing adjacent pixel points that meet a certain depth threshold condition in the depth value removed image into a region to obtain a region growing image; performing contour detection on the region growing image to obtain a depth image of the initial human region; filtering the edge information of the depth image of the initial human region to obtain a filtered depth image; converting the filtered depth image into a three-dimensional point cloud; the initially removing the background interference information in the non-human region of the depth image through a depth threshold and a pixel valid region to obtain a depth value removed image includes: determining a maximum depth threshold and a minimum depth threshold ; for any pixel point in the depth image , perform depth removal according to Formula 1, and Formula 1 is: ； In the width direction of the depth image, pixel points In direction, set the depth value of pixel points whose pixel coordinates are located in and to 0, so as to identify the depth information within a certain range on both the left and right sides of the depth image as background interference information and eliminate it. Set according to the resolution of the depth image; Denote the depth value elimination image after eliminating the pixel effective area as ; Growing adjacent pixel points that meet certain depth threshold conditions in the depth value removed image into a region to obtain a region growing image includes: For the depth value removed image the seed points in , find its 8 neighborhood pixel points according to the 8-connectivity rule ; If the depth value difference between any neighborhood point and the seed point is less than a certain threshold , and meets , then it is determined that this neighborhood point and the seed point are a generated region; In the same way, perform region growth judgment on each pixel point in the depth value removed image to obtain the number of growth regions of the depth value removed image , and further obtain a region growing image; The contour detection of the region growing image to obtain the depth image of the initial human body region includes: calculating the rectangular outer contour of the growth region ; Based on the size and position of the outer contour , screen out the growth regions belonging to the user to be measured to obtain the depth image of the initial human body region ; The filtering of the edge information of the depth image of the initial human body region to obtain the filtered depth image includes: performing erosion and dilation operations on the edge information of the depth image of the initial human body region to obtain the filtered depth image .

2. The background segmentation method based on three-dimensional human body rotation scanning using a depth camera according to claim 1, wherein, The depth camera includes an upper depth camera and a lower depth camera; the upper depth camera and the lower depth camera synchronously collect the upper and lower body depth images of the human body to generate a set of the depth images, and multiple sets of the depth images are collected as the turntable drives the user to rotate.

3. The background segmentation method based on three-dimensional human body rotation scanning using a depth camera according to claim 2, characterized in that, The step of adopting two-dimensional image background segmentation technology to perform pre-segmentation processing on the depth image and converting the pre-segmented depth image into a three-dimensional point cloud according to the internal parameter calibration result of the depth camera includes: Adopt two-dimensional image background segmentation technology to sequentially perform pre-segmentation processing on multiple sets of the depth images; According to the internal parameter calibration results of the upper depth camera and the lower depth camera, convert multiple sets of the pre-segmented depth images into multiple sets of three-dimensional point clouds.

4. The background segmentation method based on three-dimensional human body rotation scanning using a depth camera according to claim 3, characterized in that, The step of calculating a cutting cylinder based on the generated three-dimensional point cloud and using the cutting cylinder to perform background segmentation of the three-dimensional point cloud to generate a human body point cloud after background segmentation includes: Calculate a cutting cylinder based on the generated three-dimensional point cloud; Use the cutting cylinder to perform background segmentation of the multiple sets of three-dimensional point clouds to generate a human body point cloud after background segmentation.

Images