Robot target following method and system, computer and storage medium

Abstract

The invention relates to the technical field of computer vision processing, and provides a robot target following method and system, a computer and a storage medium, and the robot target following method comprises the steps: obtaining a human body image, and extracting multi-scale features and low-scale features; channel-level attention is embedded into the low-scale features and the multi-scale features, and feature fusion is carried out to extract a semantic image; extracting a plurality of key joint points from the semantic image; performing attitude estimation based on the plurality of key joint points to obtain a plurality of key attitude data packets; and constructing a following path, and controlling the robot to follow by taking the following path as a standard. By adopting the method, the recognition precision, the recognition speed and the path generation efficiency of the target and the target action in the target following process of the robot can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision processing technology, and in particular to a robot target following method, system, computer and storage medium. Background Art

[0002] Object following technology for mobile robots is an important research area involving a variety of related technologies and methods. Object following refers to the ability of a mobile robot to automatically identify and track a specific target in a dynamic environment. This technology is often used in applications such as navigation, autonomous driving, and drones. The target can be a person, vehicle, animal, etc. The robot uses different sensors and algorithms to achieve target following.

[0003] Target following technology integrates a variety of sensor technologies, algorithms and control strategies, involving fields such as visual processing, motion planning, and machine learning.

[0004] However, existing target following methods are not very effective in handling continuous following in complex scenes. Targets in complex scenes are difficult to identify, the recognition of target actions lacks accuracy, and the processing speed of collected images is insufficient. When generating a following path, a large number of nodes need to be traversed, which increases the system load and processing time, and easily leads to poor or even incorrect following path planning of the robot. Summary of the invention

[0005] The embodiments of the present application provide a robot target following method, system, computer and storage medium to solve the problems of insufficient recognition accuracy of targets and target actions, insufficient processing speed and insufficient path planning efficiency in the target following process of mobile robots in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a robot target following method, comprising the following steps: Acquire a human body image, extract high-level attention features and low-level attention features from the human body image, extract multi-scale features from the high-level attention features, and extract low-scale features from the low-level attention features; Embedding the low-scale features and the multi-scale features into channel-level attention, and performing feature fusion to obtain fused features, and extracting semantic images from the fused features; Extracting a number of key joint points from the semantic image; Performing posture estimation based on the key joint points to obtain a number of key posture data packets; Based on the plurality of key posture data packets, a following path is constructed, and the robot is controlled to follow the following path as a standard.

[0007] Furthermore, the steps of extracting high-level attention features and low-level attention features from the human body image, extracting multi-scale features from the high-level attention features, and extracting low-scale features from the low-level attention features include: Building a backbone network based on MobileNetV2, and inputting the human body image into the backbone network to obtain high-level features and low-level features; Constructing an attention module based on CBAM, and inputting the high-level features and the low-level features into the attention module to obtain high-level attention features and low-level attention features; Constructing a pooling layer, a 1*1 convolution layer, and several 3*3 dilated convolution layers to form a multi-scale module, inputting the high-level attention features into the multi-scale module to obtain transition features, and performing convolution and upsampling processing on the transition features to obtain multi-scale features; The low-level attention features are convolved to obtain low-scale features.

[0008] Furthermore, the step of embedding the low-scale features and the multi-scale features into channel-level attention and fusing the features to obtain fused features includes: Connecting the low-scale features and the multi-scale features, and performing a GAP operation to obtain an average feature; Obtain a weight module, and input the average feature into the weight module to obtain a channel-level attention feature; Based on the channel-level attention features, the low-scale features and the multi-scale features, calculating the channel-focused low-scale features and the channel-focused multi-scale features; The channel-focused low-scale features and the channel-focused multi-scale features are fused into fused features.

[0009] Furthermore, the step of extracting a plurality of key joint points from the semantic image includes: Extracting a plurality of image pixel coordinates and a plurality of heat maps from the semanticized image; Based on the plurality of image pixel coordinates and the plurality of heat maps, a plurality of key joint points are obtained according to the soft-argmax integral regression method.

[0010] Furthermore, the step of performing posture estimation based on the key joint points to obtain a plurality of key posture data packets includes: Extracting a plurality of postures to be detected from a plurality of the key joint points; Measuring the similarity of the postures to be detected by the distance between the postures to be detected, so as to perform redundant detection on a plurality of the postures to be detected, and extracting a plurality of key postures from the plurality of the postures to be detected; Extracting a plurality of key posture features and key posture parameters corresponding to the plurality of key posture features from the plurality of key postures; Using a ReLU activation function to process the key posture features to obtain a plurality of feature values ​​to be packaged; Processing a number of the key posture parameters using a sigmoid activation function to obtain a number of parameter values ​​to be packaged; Vandermonde product processing is performed on the characteristic values ​​to be packaged and the parameter values ​​to be packaged corresponding to the characteristic values ​​to be packaged to obtain a plurality of key posture data packets.

[0011] Furthermore, the step of constructing a following path based on a plurality of the key posture data packets and controlling the robot to follow the following path as a standard includes: Extracting a plurality of basic nodes from a plurality of the key gesture data packets, establishing an OPENLIST list according to a heap structure, and storing the plurality of the basic nodes in the OPENLIST list; Based on the A-star algorithm and the MPC algorithm, a number of target nodes are extracted from the OPENLIST list to construct a follow path; Based on the three-degree-of-freedom vehicle dynamics model, the robot is controlled to follow the following path as a standard.

[0012] In a second aspect, an embodiment of the present application provides a robot target following system, which is applied to the robot target following method as described in the above technical solution, and the system includes: An acquisition module, used for acquiring a human body image, extracting high-level attention features and low-level attention features from the human body image, extracting multi-scale features from the high-level attention features, and extracting low-scale features from the low-level attention features; A fusion module, used for embedding the low-scale features and the multi-scale features into channel-level attention, and performing feature fusion to obtain fused features, and extracting a semantic image from the fused features; An extraction module, used for extracting a number of key joint points from the semantic image; An estimation module, used for performing posture estimation based on a plurality of the key joint points to obtain a plurality of key posture data packets; The following module is used to construct a following path based on a plurality of the key posture data packets, and control the robot to follow the following path as a standard.

[0013] In a third aspect, an embodiment of the present application provides a computer, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the robot target following method as described in the first aspect above is implemented.

[0014] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the robot target following method as described in the first aspect above.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: by adopting the MobileNetV2 to construct the backbone network, the backbone network is more lightweight than the backbone network in the traditional DeepLabV3 model for image semantic segmentation, the computational complexity of the network is reduced without sacrificing accuracy, the feature extraction efficiency of the robot target following method for the human body image is improved, and the CBAM (Convolutional Block Attention) based method is embedded. The attention module of the attention module can introduce an attention mechanism for the two features extracted from the backbone network to enhance feature representation, which is beneficial to improving the perception ability of the network and improving performance without increasing the complexity of the network. By constructing the multi-scale module, preliminary semantic segmentation can be performed, and the two attention features that have undergone preliminary semantic segmentation are embedded in the channel-level attention again to weight the features, which can further enhance the accuracy of real-time semantic segmentation, which is beneficial to quickly process images in complex scenes to efficiently and accurately identify targets and their dynamics; by combining the heat map with the integral regression method, the negative impact of the accuracy of the traditional method that only uses the heat map is avoided due to the limitation of the image resolution, the accuracy of obtaining the key joint points is improved, and the redundant postures are removed after posture estimation, thereby improving the efficiency and accuracy of posture recognition; by constructing the OPENLIST list of the heap structure, the storage structure of the basic nodes is optimized, and the speed of traversing all the basic nodes during path planning is increased, which is beneficial to faster generation of accurate following paths. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of a robot target following method in the first embodiment of the present invention; Figure 2 It is a schematic structural diagram of a robot target following system in a second embodiment of the present invention; The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0017] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0018] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0020] See also Figure 1 In a first embodiment of the present invention, a robot target following method is provided, comprising the following steps: Step S10: acquiring a human body image, extracting high-level attention features and low-level attention features from the human body image, extracting multi-scale features from the high-level attention features, and extracting low-scale features from the low-level attention features; Preferably, an improved DeepLabV3+ model is established based on the DeepLabV3 model to process the human body image, the human body image is used as an input value, and the DeepLabV3+ model is input, and the backbone network of the DeepLabV3+ model is first input.

[0021] Specifically, the step S10 includes: S110: constructing a backbone network based on MobileNetV2, and inputting the human body image into the backbone network to obtain high-level features and low-level features; MobileNetV2 is a lightweight network. Compared with the backbone network of the DeepLabV3 model, MobileNetV2 has fewer parameters and can reduce the computational complexity of the model and increase the computing speed without sacrificing the performance of the model in extracting features.

[0022] S120: constructing an attention module based on CBAM, and inputting the high-level features and the low-level features into the attention module to obtain high-level attention features and low-level attention features; CBAM includes a channel attention module CAM and a spatial attention module SAM. The features of the attention module built based on CBAM are defined as initial features. The initial features will first enter the CAM, and form two features, i.e., two outputs, through global maximum pooling and GAP operation processing. The two outputs are fed into a multilayer perceptron, and the multilayer perceptron generates an intermediate output. The intermediate output is element-summed and activated by a sigmoid function to generate a channel attention feature. The channel attention feature is multiplied by the initial feature element to obtain a weighted output. The weighted output enters the SAM and undergoes global maximum pooling and GAP operation processing to generate two matrices of the same dimension. The two matrices are connected in series according to the channel dimension, and then convolution and activation function processing are performed to obtain a spatial attention feature. The spatial attention feature is multiplied element-by-element with the weighted output to obtain a refined attention feature. Both the high-level feature and the low-level feature are processed by the above process in the attention module to obtain the high-level attention feature and the low-level attention feature. It can be understood that by introducing the attention mechanism into the two features extracted by the backbone network through the attention module, it is beneficial to obtain more refined features, to improve the perception ability of the DeepLabV3+ model, and to improve the model's precise performance in image processing without increasing the complexity of the network.

[0023] S130: constructing a pooling layer, a 1*1 convolution layer and a plurality of 3*3 dilated convolution layers to form a multi-scale module, inputting the high-level attention features into the multi-scale module to obtain transition features, and performing convolution and upsampling processing on the transition features to obtain multi-scale features; Preferably, the multi-scale module is constructed based on ASPP (Atrous Spatial Pyramid Pooling), and the multi-scale module includes a 1*1 convolution layer, a 3*3 dilated convolution layer with a dilation rate of 6, a 3*3 dilated convolution layer with a dilation rate of 12, and a 3*3 dilated convolution layer with a dilation rate of 18. In the tasks of image understanding and semantic segmentation, the model's ability to capture multi-scale information of images can be enhanced by combining dilated convolutions with different dilation rates, wherein the 1*1 convolution layer is used to reduce the number of channels, thereby saving resources, and preventing overfitting, thereby speeding up the processing speed of the multi-scale module for image features, and multiple dilated convolution layers are run in parallel, and the output features are fused by splicing to form the multi-scale features. It can be understood that the multi-scale module can efficiently capture multi-scale information of image features, which is beneficial to completing preliminary semantic segmentation to improve the overall real-time semantic segmentation performance of the DeepLabV3+ model.

[0024] S140: Perform convolution processing on the low-level attention features to obtain low-scale features.

[0025] Preferably, a 1*1 convolution is performed on the low-level attention features to obtain the low-scale features.

[0026] Step S20: embedding the low-scale features and the multi-scale features into channel-level attention, and performing feature fusion to obtain fused features, and extracting semantic images from the fused features; Preferably, an attention feature fusion module is constructed based on AFFM (Adaptive Fuzzy Feature Map) so that channel-level attention is still embedded in the fusion process of the low-scale features and the high-scale features. In image recognition, AFFM can improve recognition accuracy and adjust feature weights through adaptive fuzzy processing. It can be understood that embedding channel-level attention in the feature fusion module and obtaining the fused features with channel-level attention can avoid the loss of important information and improve the semantic segmentation performance.

[0027] Specifically, the step S20 includes: S210: Connecting the low-scale features and the multi-scale features, and performing a GAP operation to obtain an average feature; The GAP calculation uses the average value of the features input into the attention feature fusion module as the overall representation, which can replace the fully connected layer to reduce the number of parameters of the model, which not only reduces the complexity of the calculation, but also reduces the risk of overfitting.

[0028] S220: Obtain a weight module, input the average feature into the weight module to obtain a channel-level attention feature; Preferably, the weight module includes several sub-modules, each of which is composed of a 1*1 convolutional layer and a S-type function, and each of the sub-modules corresponds to a channel and is used for channel-level compression and decompression, thereby generating the channel-level attention features.

[0029] S230: Calculating a channel-focused low-scale feature and a channel-focused multi-scale feature based on the channel-level attention feature, the low-scale feature, and the multi-scale feature; The channel-level attention features are respectively added element-by-element to the low-scale features and the multi-scale features to form channel-level low-scale features with channel attention and channel-level high-scale features with channel attention, respectively.

[0030] S240: Fusing the channel-focused low-scale features and the channel-focused multi-scale features into fused features.

[0031] The channel-focused low-scale features and the channel-focused multi-scale features are spliced ​​to obtain the fused features with channel-level attention to further output the semantic image. It can be understood that the processing of the human body image in the robot target following method can achieve real-time semanticization and obtain information of complex scenes for subsequent viewpoint adjustment and posture recognition based on the extracted feature information.

[0032] Step S30: extracting a number of key joint points from the semantic image; For human body images, key point detection is usually based on regression or heat map. The regression-based method is to let the model directly regress the key point coordinates after inputting the image. For example, the image is input into a convolutional neural network and the key point coordinates are predicted by linear regression. However, the accuracy of the direct regression coordinates is not high. The heat map-based method predicts the probability of the key point at each position, generates a heat map, and obtains the key point coordinates by the maximum value method. The heat map is more in line with the structure of the neural network, and the model accuracy is relatively higher, but the computational consumption is large.

[0033] Specifically, the step S30 includes: S310: extracting a plurality of image pixel coordinates and a plurality of heat maps from the semanticized image; Preferably, the heat map is normalized to obtain the likelihood of pixels on the normalized heat map, so as to facilitate weighting in combination with the image pixel coordinates when calculating coordinates by integral regression. The size of the heat map is the proportionally reduced size of the semantic image, which is beneficial to reducing the amount of calculation.

[0034] S320: Based on the plurality of image pixel coordinates and the plurality of heat maps, a plurality of key joint points are obtained according to a soft-argmax integral regression method.

[0035] Preferably, downsampling of traditional heat maps in convolutional neural networks reduces the amount of calculation, but restoring the output coordinates to the original image size is prone to deviation. The use of soft-argmax integral regression can alleviate the problem of quantization accuracy loss and calculate the coordinates of the key points, that is, obtain the key joint points.

[0036] Step S40: performing posture estimation based on the key joint points to obtain a plurality of key posture data packets; Preferably, a top-down framework is adopted, and a target detection algorithm is used to identify human bodies in complex scenes, and a plurality of bounding boxes are generated to identify a plurality of human bodies, and then a follow target and the posture of the follow target are further identified from the plurality of bounding boxes.

[0037] Specifically, the step S40 includes: S410: extracting a plurality of postures to be detected from a plurality of the key joint points; Preferably, a PGA (Pose-Guided Attention) module is introduced to focus the extracted posture features on the human body, which is beneficial for identifying the human body in complex scenes.

[0038] S420: measuring the similarity of the postures to be detected by the distance between the postures to be detected, so as to perform redundant detection on a plurality of the postures to be detected, and extracting a plurality of key postures from the plurality of the postures to be detected; It can be understood that removing redundant postures after performing redundant detection and extracting the key postures is beneficial to improving the efficiency of posture estimation.

[0039] S430: extracting a plurality of key posture features and key posture parameters corresponding to the plurality of key posture features from the plurality of key postures; Preferably, the key posture features are obtained through feature mapping, and the key posture parameters are obtained through soft masking.

[0040] S440: Processing the plurality of key posture features using a ReLU activation function to obtain a plurality of feature values ​​to be packaged; S450: using a sigmoid activation function to process the key posture parameters to obtain a plurality of parameter values ​​to be packaged; S460: Perform Vandermonde product processing on the characteristic values ​​to be packaged and the parameter values ​​to be packaged corresponding to the characteristic values ​​to be packaged, so as to obtain a plurality of key posture data packets.

[0041] Preferably, the data packets packaged in S430 to S460 are used to be transmitted to the motion following module of the robot.

[0042] Step S50: constructing a following path based on a plurality of the key posture data packets, and controlling the robot to follow the following path as a standard.

[0043] Preferably, the path is planned based on the A-star algorithm, but in the OPENLIST structure used by the traditional A-star algorithm, the size of the node is not calculated when adding a node. If you need to obtain the node with the smallest value, you need to traverse the storage container. If the number of nodes is large, the duration of following the target is long, and the path is long, it takes a long time to traverse the nodes. It can be understood that optimizing the storage node method is beneficial to optimizing the algorithm and reducing computing time and hardware load.

[0044] Specifically, the step S50 includes: S510: extracting a plurality of basic nodes from a plurality of the key gesture data packets, establishing an OPENLIST list according to a heap structure, and storing the plurality of the basic nodes in the OPENLIST list; In the OPENLIST list of the heap structure, the root node of the heap is the minimum value in the OPENLIST list. When adding data using the OPENLIST list of the heap structure, the new node to be added is first inserted into the tail of the heap structure, and the value of the new node is the value of the inserted node. Then the value of the inserted node is compared with the value of the parent node at the insertion position. If the value of the inserted node is less than the value of the parent node, the positions of the two nodes are swapped, and the comparison process is performed in turn until the value of the inserted node is greater than the value of the parent node at the insertion position. It can be understood that by optimizing the storage method of data, the time for adding, deleting, checking and modifying is greatly reduced, and the path planning process can be accelerated.

[0045] S520: extracting a number of target nodes from the OPENLIST list based on the A-star algorithm and the MPC algorithm to construct a follow path; Preferably, the A-star algorithm is a heuristic search algorithm used in the field of path optimization. Based on the A-star algorithm, smooth transition nodes can be designed to replace adjacent intermediate nodes without obstacles, which can reduce the total number of nodes and smooth the path. The MPC algorithm constrains the feasibility of the path, and the path constructed by the A-star algorithm can be secondary planned to obtain the following path that is accurate and suitable for the robot to follow.

[0046] S530: Based on the three-degree-of-freedom vehicle dynamics model, control the robot to follow the following path as a standard.

[0047] It can be understood that the use of the three-degree-of-freedom vehicle dynamics model can increase the analysis of the vertical dynamics and lateral dynamics of the mobile robot, and prevent the following deviation caused by not considering the robot's own dynamic properties when planning the path.

[0048] See also Figure 2 The second embodiment of the present invention provides a robot target following system, which is applied to the robot target following method described in the above embodiment, and will not be repeated here. As used below, the terms "module", "unit", "subunit", etc. can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0049] The system comprises: An acquisition module 10 is used to acquire a human body image, extract high-level attention features and low-level attention features from the human body image, extract multi-scale features from the high-level attention features, and extract low-scale features from the low-level attention features; The acquisition module 10 includes: The first unit is used to build a backbone network based on MobileNetV2, and input the human body image into the backbone network to obtain high-level features and low-level features; The second unit is used to construct an attention module based on CBAM, and input the high-level features and the low-level features into the attention module to obtain high-level attention features and low-level attention features; The third unit is used to construct a pooling layer, a 1*1 convolution layer and a plurality of 3*3 dilated convolution layers to form a multi-scale module, input the high-level attention feature into the multi-scale module to obtain a transition feature, and perform convolution and up-sampling processing on the transition feature to obtain a multi-scale feature; The fourth unit is used to perform convolution processing on the low-level attention features to obtain low-scale features.

[0050] A fusion module 20, used for embedding the low-scale features and the multi-scale features into channel-level attention, and performing feature fusion to obtain fused features, and extracting a semantic image from the fused features; The fusion module 20 includes: A fifth unit is used to connect the low-scale features and the multi-scale features and obtain average features through GAP operation; A sixth unit is used for obtaining a weight module, inputting the average feature into the weight module to obtain a channel-level attention feature; A seventh unit is used to calculate a channel-attention low-scale feature and a channel-attention multi-scale feature based on the channel-level attention feature, the low-scale feature and the multi-scale feature; The eighth unit is used to fuse the channel-focused low-scale features and the channel-focused multi-scale features into a fused feature.

[0051] An extraction module 30, used to extract a number of key joint points from the semantic image; The extraction module 30 comprises: A ninth unit, used for extracting a plurality of image pixel coordinates and a plurality of heat maps from the semantic image; The tenth unit is used to obtain a plurality of key joint points based on a plurality of image pixel coordinates and a plurality of heat maps according to a soft-argmax integral regression method.

[0052] An estimation module 40, configured to perform posture estimation based on a plurality of the key joint points to obtain a plurality of key posture data packets; The estimation module 40 comprises: An eleventh unit is used to extract a plurality of postures to be detected from a plurality of the key joint points; A twelfth unit is used for measuring the similarity of the postures to be detected by the distance between the postures to be detected, so as to perform redundant detection on a plurality of the postures to be detected, and extract a plurality of key postures from the plurality of the postures to be detected; A thirteenth unit is used to extract a plurality of key posture features and key posture parameters corresponding to the plurality of key posture features from the plurality of key postures; A fourteenth unit is used to process a plurality of the key posture features by using a ReLU activation function to obtain a plurality of feature values ​​to be packaged; A fifteenth unit is used for processing a plurality of the key posture parameters by using a sigmoid activation function to obtain a plurality of parameter values ​​to be packaged; The sixteenth unit is used to perform Vandermonde product processing on the characteristic values ​​to be packaged and the parameter values ​​to be packaged corresponding to the characteristic values ​​to be packaged, so as to obtain a plurality of key posture data packets.

[0053] The following module 50 is used to construct a following path based on a number of the key posture data packets, and control the robot to follow the following path as a standard.

[0054] The following module 50 comprises: The seventeenth unit is used to extract a plurality of basic nodes from a plurality of the key gesture data packets, establish an OPENLIST list according to a heap structure, and store the plurality of the basic nodes into the OPENLIST list; The eighteenth unit is used to extract a plurality of target nodes from the OPENLIST list based on the A-star algorithm and the MPC algorithm to construct a follow path; The nineteenth unit is used to control the robot to follow the following path based on the three-degree-of-freedom vehicle dynamics model.

[0055] The present invention also provides a computer, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the robot target following method as described in the first embodiment described above is implemented.

[0056] The present invention also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the robot target following method as described in the first embodiment above is implemented.

[0057] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0058] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A robot target following method, characterized in that: The steps include: Acquire a human body image, extract high-level attention features and low-level attention features from the human body image, extract multi-scale features from the high-level attention features, and extract low-scale features from the low-level attention features; Embedding the low-scale features and the multi-scale features into channel-level attention, and performing feature fusion to obtain fused features, and extracting semantic images from the fused features; Extracting a number of key joint points from the semantic image; Performing posture estimation based on the key joint points to obtain a number of key posture data packets; Based on the plurality of key posture data packets, a following path is constructed, and the robot is controlled to follow the following path as a standard.

2. The robot target following method according to claim 1, characterized in that: The steps of extracting high-level attention features and low-level attention features from the human body image, extracting multi-scale features from the high-level attention features, and extracting low-scale features from the low-level attention features include: Building a backbone network based on MobileNetV2, and inputting the human body image into the backbone network to obtain high-level features and low-level features; Constructing an attention module based on CBAM, and inputting the high-level features and the low-level features into the attention module to obtain high-level attention features and low-level attention features; Constructing a pooling layer, a 1*1 convolution layer, and several 3*3 dilated convolution layers to form a multi-scale module, inputting the high-level attention features into the multi-scale module to obtain transition features, and performing convolution and upsampling processing on the transition features to obtain multi-scale features; The low-level attention features are convolved to obtain low-scale features.

3. The robot target following method according to claim 1, characterized in that: The step of embedding the low-scale features and the multi-scale features into channel-level attention and fusing the features to obtain fused features includes: Connecting the low-scale features and the multi-scale features, and performing a GAP operation to obtain an average feature; Obtain a weight module, and input the average feature into the weight module to obtain a channel-level attention feature; Based on the channel-level attention features, the low-scale features and the multi-scale features, calculating the channel-focused low-scale features and the channel-focused multi-scale features; The channel-focused low-scale features and the channel-focused multi-scale features are fused into fused features.

4. The robot target following method according to claim 1, characterized in that: The step of extracting a plurality of key joint points from the semantic image comprises: Extracting a plurality of image pixel coordinates and a plurality of heat maps from the semanticized image; Based on the plurality of image pixel coordinates and the plurality of heat maps, a plurality of key joint points are obtained according to the soft-argmax integral regression method.

5. The robot target following method according to claim 1, characterized in that: The step of performing posture estimation based on the key joint points to obtain the key posture data packets includes: Extracting a plurality of postures to be detected from a plurality of the key joint points; Measuring the similarity of the postures to be detected by the distance between the postures to be detected, so as to perform redundant detection on a plurality of the postures to be detected, and extracting a plurality of key postures from the plurality of the postures to be detected; Extracting a plurality of key posture features and key posture parameters corresponding to the plurality of key posture features from the plurality of key postures; Using a ReLU activation function to process the key posture features to obtain a plurality of feature values ​​to be packaged; Processing a number of the key posture parameters using a sigmoid activation function to obtain a number of parameter values ​​to be packaged; Vandermonde product processing is performed on the characteristic values ​​to be packaged and the parameter values ​​to be packaged corresponding to the characteristic values ​​to be packaged to obtain a plurality of key posture data packets.

6. The robot target following method according to claim 1, characterized in that: The step of constructing a following path based on a plurality of the key posture data packets and controlling the robot to follow the following path as a standard comprises: Extracting a plurality of basic nodes from a plurality of the key gesture data packets, establishing an OPENLIST list according to a heap structure, and storing the plurality of the basic nodes in the OPENLIST list; Based on the A-star algorithm and the MPC algorithm, a number of target nodes are extracted from the OPENLIST list to construct a follow path; Based on the three-degree-of-freedom vehicle dynamics model, the robot is controlled to follow the following path as a standard.

7. A robot target following system, characterized in that: The robot target following method according to any one of claims 1 to 6, characterized in that the system comprises: An acquisition module, used for acquiring a human body image, extracting high-level attention features and low-level attention features from the human body image, extracting multi-scale features from the high-level attention features, and extracting low-scale features from the low-level attention features; A fusion module, used for embedding the low-scale features and the multi-scale features into channel-level attention, and performing feature fusion to obtain fused features, and extracting a semantic image from the fused features; An extraction module, used for extracting a number of key joint points from the semantic image; An estimation module, used for performing posture estimation based on a plurality of the key joint points to obtain a plurality of key posture data packets; The following module is used to construct a following path based on a plurality of the key posture data packets, and control the robot to follow the following path as a standard.

8. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the robot target following method according to any one of claims 1 to 6 is implemented.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the robot target following method according to any one of claims 1 to 6 is implemented.

Images