Multi-target person recognition and localization method based on RGB-D image segmentation

Abstract

The present invention provides a multi-target personnel identification and positioning method based on RGB-D image segmentation, comprising an offline training phase and an online positioning phase. The offline training phase collects training data, then uses a Yo L OV3 network to establish a multi-target detection model, then establishes a target recognition model after classification learning, generates a color depth map for image segmentation, and then uses a convolutional neural network to obtain a position estimation model. In the online positioning phase, after collecting online data, personnel identification is completed using the multi-target detection model and the target recognition model; a color depth map is generated; and the position estimation model is used to complete the positioning of the personnel target. Compared with the prior art, the multi-target personnel identification and positioning method based on RGB-D image segmentation of the present invention solves the problems of insufficient flexibility and accuracy of existing multi-target personnel identification and positioning methods by generating a color depth map, performing image segmentation according to the Yo L OV3 network recognition results, and inputting the segmented image into the position estimation model to complete the positioning of the personnel target.

Description

Technical Field

[0001] The present invention relates to a multi-target personnel recognition and positioning method based on RGB-D image segmentation, and belongs to the field of deep learning. Background Art

[0002] The research on multi-target person recognition and localization technology has always been one of the most basic and challenging research topics in the field of Computer Vision (CV).

[0003] Commonly used personnel recognition technologies are categorized as radio frequency identification (RFID)-based and image-based. RFID-based personnel recognition involves individuals wearing RFID tags in the form of cards or bracelets transmitting radio signals within the coverage area. A receiver then uploads the received signals to a server for analysis, obtaining the individual's identity and location information. Image-based methods, on the other hand, do not require individuals to carry tags, are low-cost, simplify equipment deployment, avoid electromagnetic interference, and have a wider range of applications. Convolutional neural networks (CNNs) can directly use images as input and, through multiple convolution operations, obtain feature information for each target. The application of CNNs to target recognition has significantly improved the accuracy and speed of image-based target detection methods.

[0004] Image-based object detection algorithms are categorized into two-stage and single-stage algorithms. Two-stage algorithms involve two steps: the first step is to identify candidate regions from the image, and the second step is to perform classification and regression on each candidate region. Classic candidate region-based object detection algorithms include R-CNN, Fast R-CNN, and Faster R-CNN. Single-stage methods do not require prior identification of candidate regions and can directly output classification and regression results. These methods can achieve real-time processing. Classic networks include the YOLO series and SSD.

[0005] With the increasing popularity of smartphones and smart surveillance, the use of cameras for target positioning is gaining increasing attention. Vision-based positioning technology is typically divided into two phases: offline and online. The offline phase involves collecting image information and generating a database. The online phase extracts features from camera images and compares them with the image features in the database to estimate position. The larger the database collected during the offline phase, the higher the accuracy and robustness of visual positioning. This makes vision-based positioning less susceptible to environmental changes.

[0006] Visual positioning solutions use color cameras and RGB-D cameras. Color cameras only provide color images of the environment, while RGB-D cameras provide both color and depth images. Depth images reflect the relative position of the target to the camera, with larger depth values ​​indicating greater distance from the camera. Therefore, visual positioning systems based on RGB-D cameras have advantages.

[0007] In view of this, it is necessary to provide a multi-target personnel recognition and positioning method based on RGB-D image segmentation to solve the above problems. Summary of the Invention

[0008] The purpose of the present invention is to provide a multi-target personnel recognition and positioning method based on RGB-D image segmentation to solve the problems of insufficient prediction flexibility and accuracy of existing target positioning methods.

[0009] To achieve the above objectives, the present invention provides a multi-target personnel recognition and positioning method based on RGB-D image segmentation, which includes two stages: an offline training stage and an online positioning stage.

[0010] The offline training phase includes:

[0011] Step 1: Use an RGB-D camera to collect training data, which includes RGB color images and depth images;

[0012] Step 2: Use the yolov3 network to perform multi-target personnel target detection training on the RGB color map of step 1, obtain the training results, and establish a multi-target detection model;

[0013] Step 3: Based on the training results obtained in step 2, the RGB color map is segmented, the target face data is used for classification learning, and a target recognition model is established;

[0014] Step 4: Combine the RGB color map and depth image to generate a color depth map;

[0015] Step 5: Based on the training results obtained in step 2, the color depth map obtained in step 4 is segmented and then trained offline using a convolutional neural network to obtain a position estimation model.

[0016] The online orientation phase includes:

[0017] Step 6: Use the RGB-D camera to collect online data, which includes RGB color images and depth images;

[0018] Step 7: Use the multi-target detection model in step 2 and the target recognition model in step 3 to complete person recognition;

[0019] Step 8: Combine the RGB color map and depth image to generate a color depth map;

[0020] Step 9: Use the position estimation model in step 5 to complete the positioning of the human target.

[0021] As a further improvement of the present invention, the training result in step 2 includes the target rectangular box position information of the target in the RGB color map output by the yolov3 network.

[0022] As a further improvement of the present invention, step 3 specifically includes:

[0023] Step 31: Based on the training results of the multi-target personnel target detection training in step 2, the obtained target rectangular frame position information is used to segment the RGB color map to obtain an RGB color map segmentation map of each person target;

[0024] Step 32: Use the Haar cascade classifier to detect the RGB color map segmentation map and extract the target face data;

[0025] Step 33: Use the Eigen Face method to perform classification learning using the target face data extracted in step 32 to establish a target recognition model.

[0026] As a further improvement of the present invention, step 4 of generating a color depth map specifically includes:

[0027] Step 41: Use the coordinate system transformation method to align the pixels in the depth image with the RGB color map;

[0028] Step 42: Use a mapping method to convert the original grayscale depth image into a color depth image.

[0029] As a further improvement of the present invention, step 4 of generating a color depth map further includes:

[0030] Step 43: using a median smoothing filter to remove noise from the color depth image;

[0031] Step 44: Use bilateral filtering to repair color depth image holes.

[0032] As a further improvement of the present invention, step 5 specifically includes: according to the training results obtained in step 2, the color depth map is segmented, the depth image within the target rectangular frame is retained, and the depth value outside the area is set to 255; the segmented color depth map and the corresponding position information are sent to the convolutional neural network training position estimation model.

[0033] As a further improvement of the present invention, in step 1 and step 6, when the RGB-D camera collects online data, it will simultaneously generate an RGB color map and a depth image.

[0034] As a further improvement of the present invention, step 7 is specifically as follows:

[0035] Step 71: Send the RGB color map collected by the RGB-D camera in step 6 to the YOLOv3 network for multi-target detection of people;

[0036] Step 72: Extracting facial data of each target based on the multi-target detection results of step 71;

[0037] Step 73: The collected facial data is fed into the target recognition model in step 3 to complete person recognition.

[0038] As a further improvement of the present invention, the process of generating the color depth map in step 8 is the same as the process of generating the color depth map in step 4.

[0039] As a further improvement of the present invention, step 9 specifically includes:

[0040] Step 91: Use the target detection results output by the multi-target detection model in step 7 to perform image segmentation on the color depth map in step 8;

[0041] Step 92: Input the color depth map obtained after image segmentation in step 91 into the position estimation model in step 5 to obtain the position of the person, thereby completing the positioning of the person target.

[0042] The beneficial effects of the present invention are as follows: compared with the existing technology, the multi-target personnel recognition and positioning method based on RGB-D image segmentation of the present invention generates a color depth map and then performs image segmentation according to the YOLOv3 network recognition result, and inputs the segmented image into the position estimation model to complete the positioning of the personnel target, thereby solving the problems of insufficient flexibility and insufficient accuracy of the existing multi-target personnel recognition and positioning method based on RGB-D image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a step diagram of the multi-target personnel recognition and positioning method based on RGB-D image segmentation of the present invention.

[0044] Figure 2 It is a flow chart of the present invention.

[0045] Figure 3 It is a schematic diagram of the structure of the yolov3 network of the present invention.

[0046] Figure 4 It is a structural diagram of the convolutional neural network of the present invention.

[0047] Figure 5 It is a target recognition result accuracy graph of the present invention.

[0048] Figure 6It is the position estimation result error diagram of the present invention. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] It should be noted here that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions of the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0051] In addition, it should be noted that the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus.

[0052] See also Figures 1 and 2 As shown, the present invention provides a multi-target personnel recognition and positioning method based on RGB-D image segmentation, which includes two stages: offline training stage and online positioning stage.

[0053] The offline training phase includes:

[0054] Step 1: Use an RGB-D camera to collect training data, which includes RGB color images and depth images;

[0055] Step 2: Use the yolov3 network to perform multi-target personnel target detection training on the RGB color map of step 1, obtain the training results, and establish a multi-target detection model;

[0056] Step 3: Based on the training results obtained in step 2, the RGB color map is segmented, the target face data is used for classification learning, and a target recognition model is established;

[0057] Step 4: Combine the RGB color map and depth image to generate a color depth map;

[0058] Step 5: Based on the training results obtained in step 2, the color depth map obtained in step 4 is segmented and then trained offline using a convolutional neural network to obtain a position estimation model.

[0059] In step 1, when the RGB-D camera collects data, it generates an RGB color map and a depth image at the same time.

[0060] After obtaining the training data collected by the RGB-D camera, the RGB color map is fed into the YOLOv3 network for target detection training. The YOLOv3 network outputs the training results, including the target rectangle position information in the RGB color map. The target rectangle position information includes the upper left vertex position P(x1, y1) of the rectangle and the lower right vertex position Q(x2, y2) of the rectangle.

[0061] For details, see Figure 3 As shown in Figure 1, the YOLOv3 algorithm is an object recognition and positioning algorithm based on a deep neural network. It is a comprehensive improvement based on YOLOv1 and YOLOv2. The algorithm has achieved good advantages in both running time and detection accuracy. Compared with the previous two versions, the YOLOv3 algorithm has adjusted the network structure. It consists of an input layer, a basic network Darknet-53, and three network branches for multi-scale detection. The network structure is as follows: Figure 2 shown.

[0062] The Yolov3 algorithm also improves the loss function and uses the binary cross entropy loss function to represent the category loss. The Yolov3 loss function is mainly divided into three parts: target positioning offset loss L loc (l,g), target confidence loss L conf (o,c) and target classification loss L cla (O,C), where λ1, λ2, λ3 are balance coefficients:

[0063] L(O,o,C,c,l,g)=λ1L conf (o,c)+λ2L cla (O,C)+λ3L loc (l,g)

[0064] The target confidence can be understood as the probability of the target existing in the predicted target rectangle. The target confidence loss L conf (o,c) uses binary cross entropy loss (Binary Cross Entropy), where o i ∈{0,1}, indicates whether the target actually exists in the predicted target bounding box i, 0 means no target exists, and 1 means yes target exists. Indicates the Sigmoid probability of predicting whether there is a target in the target rectangle i:

[0065]

[0066]

[0067] Target category loss L cla (O, C) also uses binary cross entropy loss, where Oij ∈{0,1}, indicates whether the j-th type of target actually exists in the predicted target bounding box i, 0 means no, 1 means yes. Indicates the Sigmoid probability of the jth type of target in the network prediction target bounding box i:

[0068]

[0069]

[0070] Target localization loss L loc (l,g) uses the sum of the squares of the difference between the true deviation value and the predicted deviation value, where Indicates the predicted rectangular frame coordinate offset, Indicates the coordinate offset between the matching GTbox and the default box, (b x ,b y ,b w ,b h ) is the predicted target rectangle parameter, (c x ,c y ,c w ,c h ) is the default rectangular frame parameter, (g x ,g y ,g w ,g h ) are the parameters of the true target rectangle that matches it, and these parameters are mapped on the predicted feature map:

[0071]

[0072]

[0073]

[0074]

[0075]

[0076] Then proceed to step 3, which specifically includes:

[0077] Step 31: Based on the training results of the multi-target personnel target detection training in step 2, that is, using the obtained target rectangular frame position information, segment the RGB color map to obtain the RGB color map segmentation map of each personnel target;

[0078] Step 32: Use the Haar cascade classifier to detect the RGB color map segmentation map and extract the target face data;

[0079] Step 33: Use the Eigen Face method to perform classification learning using the target face data extracted in step 32 to establish a target recognition model.

[0080] In step 31, the RGB color map is segmented according to the target rectangular frame position information obtained by target detection, the color image within the target rectangular frame is retained, and the pixel values ​​outside the area are set to 255; then step 32 uses the Harr cascade classifier in OpenCV to detect the RGB color map segmentation map, extract and save the target face data; finally, step 33 calls the face recognition class encapsulated in the OpenCV library according to the collected face image data, and adopts the Eigen Face method to build a face-based target recognition model.

[0081] After completion, proceed to step 4. Step 4 generates a color depth map, which specifically includes:

[0082] Step 41: Use the coordinate system transformation method to align the pixels in the depth image with the RGB color map;

[0083] This step is used to adjust the pixels of the depth image and RGB color map to ensure a one-to-one correspondence. Sometimes, the intrinsic parameters and image sizes of the RGB color map and depth image are not exactly the same, resulting in some misalignment. Only after the RGB color map and depth image are registered can the pixels on the images maintain a one-to-one correspondence.

[0084] The camera model used in the present invention's test is an Intel RealSense D435, which has its own independent library pyrealsense2 in the Python environment. Calling the functions in this library can achieve pixel alignment of the RGB color map and the depth image.

[0085] Step 42: Convert the original grayscale depth image into a color depth image using a mapping method;

[0086] This step is used to render the original grayscale depth image into a color depth map. In this application, the applyColorMap() function in the OpenCV library is used to render the original grayscale depth image into a color depth map.

[0087] Step 4 of generating a color depth map also includes:

[0088] Step 43: using a median smoothing filter to remove noise from the color depth image;

[0089] This step is used to remove noise from the color depth image through a median smoothing filter. The filter used in the test of the present invention is the median filter built into the pyrealsense2 library.

[0090] Step 44: Use bilateral filtering to repair color depth image holes.

[0091] This step is used to repair holes in the color depth image to improve image quality. The hole repair technology used in the test of this invention is the bilateral filter repair function built into the pyrealsense2 library.

[0092] Then proceed to step 5, which specifically includes: obtaining the target rectangular frame position information obtained by target detection according to step 2, segmenting the preprocessed color depth image, retaining the color depth image within the target rectangular frame, and setting the depth value outside the area to 255; sending the color depth image and the corresponding position information to the convolutional neural network (CNN) training position estimation model.

[0093] See also Figure 4 As shown, convolutional neural networks (CNNs) are a widely used deep learning technology with excellent image recognition performance. Convolutional neural networks share many similarities with conventional neural networks. Both mimic the structure of human neurons and are composed of neurons with learnable weights and bias constants. Each neuron receives an input signal and, after performing calculations, outputs a score for each classification. However, the input of a convolutional neural network is generally an image. Convolutional networks perform convolution and pooling, successfully reducing the dimensionality of large images and ultimately making them trainable. Convolutional neural networks are characterized by one or more convolutional and pooling layers. Convolutional neural networks have three unique concepts: local filters, max pooling, and weight sharing. There are three main types of layers in a convolutional neural network: convolutional layers, pooling layers, and fully connected layers.

[0094] Assume that a picture X in the training set is the input of the convolutional neural network, s(i, j) is the feature matrix of the i-th layer of the convolutional neural network, and s(i, j) can be expressed as the following formula:

[0095]

[0096] Where n is the number of input matrices, or the dimension of the last dimension of the tensor. k Represents the kth input matrix. W k Represents the kth subconvolution kernel matrix of the convolution kernel. s(i,j) is the value of the corresponding position element of the output matrix corresponding to the convolution kernel W, and b represents the offset matrix of the i-th layer.

[0097] After the convolutional layer, the convolutional neural network enters the pooling layer. This layer downsamples the feature maps calculated by the convolutional layer, reducing the complexity of the network model and balancing rapidly changing feature maps. After multiple convolution and sampling operations, the fully connected layer of the convolutional neural network finally generates a one-dimensional feature vector for subsequent processing.

[0098] The two activation functions ReLU and Elu that are often used in the entire convolution pooling process are as follows:

[0099] ReLU activation function:

[0100]

[0101] Elu activation function:

[0102]

[0103] The ReLU activation function is a piecewise linear function with unilateral inhibition, which gives neurons sparse activation. The ReLU activation function has negative values, which push the output mean of the activated unit closer to 0, achieving batch normalization and reducing computational complexity. Convolutional training is prone to overfitting and produces simplistic models. Therefore, a Dropout layer should be added after the fully connected layer, with a typical value range of 0.4-0.6. The model should be trained using the backpropagation algorithm.

[0104] Take the color depth image after image segmentation as an example: the input image data is first converted to 64×64×3. After the first convolution (3×3×32), the data becomes 64×64×32. After the second convolution (3×3×32), the data becomes 32×32×32. After the first pooling, the data becomes 32×32×32. After the third convolution (3×3×64), the data becomes 8×8×64. After the second pooling, the data becomes 4×4×64. After the fourth convolution (3×3×128), the data becomes 2×2×128. After the third pooling, the data becomes 1×1×128. After flattening, it becomes 128 features. After passing through four fully connected layers, 1 value is output, that is, the target position coordinate x or y is obtained. Using two models at the same time, the target coordinates (x, y) can be obtained.

[0105] The online orientation phase includes:

[0106] Step 6: Use the RGB-D camera to collect online data, which includes RGB color images and depth images;

[0107] Step 7: Use the multi-target detection model in step 2 and the target recognition model in step 3 to complete person recognition;

[0108] Step 8: Combine the RGB color map and depth image to generate a color depth map;

[0109] Step 9: Use the position estimation model in step 5 to complete the positioning of the human target.

[0110] The process of collecting online data using an RGB-D camera in step 6 is actually the same as that in step 1.

[0111] Step 7 is as follows:

[0112] Step 71: Send the RGB color map of the online data collected by the RGB-D camera in step 6 to the yolov3 network for multi-target detection of people;

[0113] Step 72: Extracting facial data of each target based on the multi-target detection results of step 71;

[0114] Step 73: The collected facial data is fed into the target recognition model in step 3 to complete person recognition.

[0115] The process of generating the color depth map in step 8 is the same as the process of generating the color depth map in step 4, and will not be repeated here.

[0116] Step 9 specifically includes:

[0117] Step 91: Use the target detection results output by the multi-target detection model in step 7 to perform image segmentation on the color depth map in step 8;

[0118] Step 92: Input the color depth map after image segmentation in step 91 into the position estimation model in step 5 to obtain the position of the person and complete the positioning of the person target

[0119] See also Figure 5 and Figure 6 As shown in the figure, to demonstrate the performance of the person recognition model, images of a single target at different reference points were collected. Faces were detected using a cascade classifier to generate a face dataset. 900 images were collected for training and 250 images were collected for testing for each target. A total of four different targets were collected, so the training set data size was 3600 images and the test set size was 1000 images. Different sample sizes were selected for comparison. Figure 5 Comparison of the accuracy of the person recognition model under different sample sizes. Figure 5 As can be seen from the figure, the more training samples there are, the higher the accuracy of person recognition and classification. When the number of samples is 3600, the accuracy has reached 93.1%.

[0120] We validated offline training performance using 400 color and depth images collected from two single targets at 55 reference points. We divided these 400 images into 300 training images for each reference point, with 100 images used as the test set. The offline training set was sized at 2×55×300, totaling 33,000 images, while the offline test set was sized at 2×55×100, totaling 11,000 images. Figure 6 The figure compares the mean absolute error (MAE) and root mean square error (RMSE) of the position estimation model for different sample sizes. As can be seen, the larger the offline training set, the higher the model accuracy. When the training set size is 33,000, the MAE is 1.88 cm and the RMSE is 2.25 cm.

[0121] Experimental results show that the present invention can achieve high-quality target recognition and position estimation performance and has practical value.

[0122] The present invention uses the Yolov3 network to perform target detection on the RGB color map, and then segments the generated color depth map according to the target detection result. It comprehensively utilizes the information of the RGB color map and the depth image, trains the target recognition model and the position estimation model respectively, and completes the recognition and positioning of multiple targets.

[0123] At the same time, the present invention preprocesses the original depth image captured by the RGB-D camera, corresponds the depth image to each pixel of the RGB color map, renders the grayscale depth image into a color depth image, removes most of the noise in the color depth image through a spatial filter, and repairs the holes in the color depth image to improve the image quality. After regression learning, the positioning accuracy is improved.

[0124] In summary, the multi-target personnel recognition and positioning method based on RGB-D image segmentation of the present invention generates a color depth map and then performs image segmentation according to the recognition result of the yolov3 network, and inputs the segmented image into the position estimation model to complete the positioning of the personnel target, thereby solving the problems of insufficient flexibility and accuracy of the existing multi-target personnel recognition and positioning method based on RGB-D image segmentation.

[0125] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A multi-target person recognition and localization method based on RGB-D image segmentation, comprising two stages: an offline training stage and an online localization stage, characterized by: The offline training phase includes: Step 1: Use an RGB-D camera to collect training data, which includes RGB color images and depth images; Step 2: Use the yolov3 network to perform multi-target personnel target detection training on the RGB color map of step 1, obtain training results, and establish a multi-target detection model; the training results include the target rectangular box position information of the target in the RGB color map output by the yolov3 network; Step 3: Based on the training results obtained in step 2, the RGB color map is segmented, the target face data is used for classification learning, and a target recognition model is established; Step 4: Combine the RGB color map and depth image to generate a color depth map; Step 5: Based on the training results obtained in step 2, the color depth map obtained in step 4 is segmented and then trained offline using a convolutional neural network to obtain a position estimation model. The online orientation phase includes: Step 6: Use the RGB-D camera to collect online data, which includes RGB color images and depth images; Step 7: Use the multi-target detection model in step 2 and the target recognition model in step 3 to complete person recognition; Step 8: Combine the RGB color map and depth image to generate a color depth map; Step 9: Use the position estimation model in step 5 to complete the positioning of the human target; Among them, step 4 of generating a color depth map specifically includes: Step 41: Use the coordinate system transformation method to align the pixels in the depth image with the RGB color map; Step 42: Convert the original grayscale depth image into a color depth image using a mapping method; Step 5 specifically includes: according to the training results obtained in step 2, the color depth map is segmented, the depth image within the target rectangular frame is retained, and the depth value outside the area is set to 255; the segmented color depth map and the corresponding position information are sent to the convolutional neural network to train the position estimation model.

2. The multi-target personnel recognition and positioning method based on RGB-D image segmentation according to claim 1 is characterized in that: Step 3 specifically includes: Step 31: Based on the training results of the multi-target personnel target detection training in step 2, the obtained target rectangular frame position information is used to segment the RGB color map to obtain an RGB color map segmentation map of each person target; Step 32: Use the Haar cascade classifier to detect the RGB color map segmentation map and extract the target face data; Step 33: Use the Eigen Face method to perform classification learning using the target face data extracted in step 32 to establish a target recognition model.

3. The multi-target person recognition and positioning method based on RGB-D image segmentation according to claim 1 is characterized in that: Step 4 of generating a color depth map also includes: Step 43: using a median smoothing filter to remove noise from the color depth image; Step 44: Use bilateral filtering to repair color depth image holes.

4. The multi-target person recognition and positioning method based on RGB-D image segmentation according to claim 1, characterized in that: In steps 1 and 6, when the RGB-D camera collects online data, it generates an RGB color map and a depth image at the same time.

5. The multi-target personnel recognition and positioning method based on RGB-D image segmentation according to claim 1 is characterized in that: Step 7 is as follows: Step 71: Send the RGB color map collected by the RGB-D camera in step 6 to the YOLOv3 network for multi-target detection of people; Step 72: Extracting facial data of each target based on the multi-target detection results of step 71; Step 73: The collected facial data is fed into the target recognition model in step 3 to complete person recognition.

6. The multi-target person recognition and positioning method based on RGB-D image segmentation according to claim 1, characterized in that: The process of generating the color depth map in step 8 is the same as the process of generating the color depth map in step 4.

7. The multi-target personnel recognition and positioning method based on RGB-D image segmentation according to claim 4 is characterized in that: Step 9 specifically includes: Step 91: Use the target detection results output by the multi-target detection model in step 7 to perform image segmentation on the color depth map in step 8; Step 92: Input the color depth map obtained after image segmentation in step 91 into the position estimation model in step 5 to obtain the position of the person, thereby completing the positioning of the person target.

Images