A target recognition method, system and sweeping robot based on lightweight network

Abstract

The present invention provides a target recognition method, system and sweeping robot based on a lightweight network, the method comprising: collecting a picture data set of different objects in an actual indoor environment, calibrating the pictures, and dividing the data set into a training set, a verification set and a test set; based on the YOLOv4‑tiny network structure, establishing a lightweight network model including a feature extraction backbone network, a hybrid channel attention mechanism module and a decoupling head module; training the lightweight network model using the training set and the verification set, and saving the model weights when the loss function converges; loading the saved model weights, and using the trained lightweight network model to recognize indoor objects; and planning the motion path of the sweeping robot according to the recognition results. The present invention trains a lightweight network model with stronger generalization ability and higher recognition accuracy, improves the detection accuracy of small objects, and ensures high real-time performance while greatly improving the detection accuracy.

Description

Technical Field

[0001] The present invention relates to the field of computer target detection technology, and in particular to a target recognition method and system based on a lightweight network and a sweeping robot. Background Art

[0002] Breakthroughs in artificial intelligence technology have brought huge opportunities to the research of sweeping robots. At present, sweeping robots have been successfully applied in a variety of environments such as supermarkets and family museums. The target detection technology of sweeping robots refers to the robot's automatic identification of objects and obstacle avoidance through visual sensors without human intervention. The detection target depends on the specific environment in which the robot is located. However, while the target detection performance has been improved, the shortcomings of deep neural networks, such as large parameter scale and high computational complexity, have become increasingly obvious. In order to promote the deployment of deep learning networks on the embedded side, lightweight networks have been rapidly developed, with the aim of reducing the number of model parameters and ensuring real-time performance and detection accuracy. At present, the research on lightweight networks is mainly divided into two directions: (1) compressing the existing network model. Commonly used methods include parameter pruning, low-rank decomposition, compact filter design, and knowledge distillation. Through the above methods, the number of network parameters and the amount of computation can be reduced while maintaining the network performance. (2) adopting lightweight ideas in model design to obtain more features in a limited number of layers. Although the number of parameters and the amount of computation are greatly reduced, it is at the expense of a certain degree of detection accuracy. For target detection, multi-scale feature extraction, especially for small objects, is very important. In order to balance the accuracy and speed of target detection, it is necessary to propose a lightweight network model suitable for sweeping robots.

[0003] Taking YOLOv4-tiny (You Only Look Once, YOLO)[1][2][3] as an example, the structure and feature extraction process of the lightweight model are briefly described:

[0004] YOLOv4-tiny is a typical lightweight network model and a simplified version of YOLOv4, with its parameters reduced to one tenth of YOLOv4. Compared with the 162-layer YOLOv4 network structure, this network is reduced to 38 layers, three residual units are used in the backbone network, the activation function uses LeakyReLU, the merged effective feature layer uses the feature pyramid (FPN) structure, and the prediction output layer (YoloHead) is reduced from three to two.

[0005] YOLOv4-tiny has the characteristics of multi-task, end-to-end, attention mechanism and multi-scale. Multi-task can be understood as completing the classification and regression of the target at the same time, realizing parameter sharing and avoiding overfitting; end-to-end means that the network model automatically learns and obtains features after receiving image information, and directly gives classification and regression prediction information; the attention mechanism focuses on the features of the target area to be detected, so as to quickly obtain effective features; multi-scale usually fuses upsampled and downsampled data, so as to detect targets of multiple scales. When training the model, you can use methods such as Mosaci data enhancement, label smoothing, and learning rate cosine annealing attenuation to improve the training speed and detection accuracy of the model. The structural diagram of the YOLOv4-tiny network model is shown below. Figure 1 shown.

[0006] Here, the feature extraction process of YOLOv4-tiny is described in detail. When the input image size is 416×416×3, it becomes 208×208×32 after filtering through a convolution filter with a step size of 2 (3×3×3×32, where the first two parameters constitute 3×3, which is the kernel size of the convolution filter, the third parameter 3 is the number of channels, and corresponds to the number of channels of the input image, and the fourth parameter 32 is the number of convolution filters, which corresponds to the number of channels of the output feature). Similarly, it becomes 104×104×64 after filtering through a convolution filter with a step size of 2 (3×3×3×64). In the further processing process, the residual unit is introduced, and the output is obtained in combination with the pooling layer to be 52×52×128. After three similar residual structures are repeatedly used to extract features, the first output layer with a size of 13×13×255 is finally obtained. In order to improve the detection capability of multi-scale objects, the 13×13×512 layer is convolved and upsampled to 26×26×128, and fused with 26×26×256 to get an output of 26×26×384, which is passed through a convolution filter (3×3×384×256) to get 26×26×256, and convolved again (1×1×256×255) to get a second predicted output size of 26×26×255. After the above steps, the feature extraction process of the lightweight network is formed.

[0007] YOLOv4-tiny has a small number of parameters. Although its detection speed is very high, it sacrifices accuracy. First, the lightweight network has fewer layers and cannot extract deep semantic information, which is very important for positioning in target detection. Second, when the input size is 416×416, the prediction output layer of the network is 13×13 and 26×26, which reduces a very important prediction output layer of 52×52 compared to YOLOv4, resulting in a decrease in the detection performance of multi-scale objects, especially small objects.

[0008] References:

[0009] [1]Redmon J,Farhadi A.YOLO9000:Better,Faster,Stronger[C] / / IEEEConference on Computer Vision&Pattern Recognition.IEEE,2017:6517-6525.

[0010] [2]Redmon J,Farhadi A.YOLOv3:An Incremental Improvement[J].arXiv e-prints,2018.

[0011] [3]Bochkovskiy A,Wang CY,Liao H.YOLOv4: Optimal Speed ​​and Accuracy ofObject Detection[J].2020. Summary of the invention

[0012] To this end, the embodiments of the present invention provide a target recognition method, system and sweeping robot based on a lightweight network, which are used to solve the problem that the current lightweight network structure in the prior art greatly reduces the number of parameters and the number of network layers, but only considers the real-time detection and sacrifices the detection accuracy, especially the detection performance of small objects.

[0013] In order to solve the above problems, an embodiment of the present invention provides a target recognition method based on a lightweight network, the method comprising:

[0014] S1: Collect image datasets of different objects in real indoor environments, calibrate the images, and divide the datasets into training sets, validation sets, and test sets;

[0015] S2: Based on the YOLOv4-tiny network structure, a lightweight network model including a feature extraction backbone network, a hybrid channel attention mechanism module, and a decoupling head module is established;

[0016] S3: Train the lightweight network model using the training set and the validation set, and save the model weight when the loss function converges;

[0017] S4: Load the saved model weights and use the trained lightweight network model to identify indoor objects;

[0018] S5: planning the motion path of the sweeping robot according to the recognition result;

[0019] Among them, the feature extraction backbone network includes five downsamplings, two of which are convolutions with a step size of 2 and three of which are maximum pooling structures; the 10th, 18th, and 26th layers of the feature extraction backbone network are maximum pooling structures, and mixed channel attention mechanism modules are added after the 24th and 27th layers of the feature extraction backbone network, respectively, and the 31st and 38th layers of the feature extraction backbone network are decoupling head modules.

[0020] Preferably, the feature extraction backbone network includes five downsamplings, two of which are convolutions with a step size of 2 and three of which are maximum pooling structures; the 10th, 18th, and 26th layers of the feature extraction backbone network are maximum pooling structures, and mixed channel attention mechanism modules are added after the 24th and 27th layers of the feature extraction backbone network, respectively, and the 31st and 38th layers of the feature extraction backbone network are decoupling head modules.

[0021] Preferably, the maximum pooling structure consists of two parallel branches, the first branch is the maximum pooling combined with a convolution with a step size of 1, and the second branch is the convolution with a step size of 2 combined with a convolution with a step size of 1.

[0022] Preferably, the hybrid channel attention mechanism module includes a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism performs maximum pooling and average pooling on the input feature map from a spatial perspective, and sends it to a shared network. Finally, the output is multiplied by the input feature map to obtain the output of the channel attention mechanism; the input of the spatial attention mechanism is the output of the channel attention mechanism, which performs maximum pooling and average pooling on the new input feature map from a channel perspective, and concatenates the two results. After the convolution operation, it is multiplied again with the new input feature map to obtain the final output.

[0023] Preferably, the decoupling head module is used to reduce the amount of calculation. The decoupling head module includes first using a 1×1 convolution to compress the number of channels of the input feature map, and then adding two parallel branches. The first branch uses two 3×3 convolutions to process the classification task, and the second branch also uses two 3×3 convolutions to process the regression task, and the branch for detecting the object confidence is added to the branch of the regression task.

[0024] Preferably, the loss function adopts the Focal Loss function, and its formula is:

[0025] FL(p t )=-α t (1-p t ) γ log(p t )

[0026] Among them, α is the weight for adjusting positive and negative samples, γ is the weight for adjusting difficult and easy samples, and pt Indicates confidence.

[0027] Preferably, before training the improved lightweight network model, a data augmentation operation is performed on the training set.

[0028] Preferably, the step S4 specifically includes:

[0029] In the testing phase, the trained network model weights are loaded, and the test set is input into the trained lightweight network model. By judging the confidence threshold, the bounding box coordinate position, confidence and category probability of all prediction boxes that may contain obstacle avoidance objects are obtained. Then, redundant detection boxes are removed through non-maximum suppression to obtain the final detection result.

[0030] Preferably, the training set, validation set and test set account for 80%, 10% and 10% respectively.

[0031] The embodiment of the present invention further provides a target recognition system based on a lightweight network, the system comprising:

[0032] The data collection and processing module is used to collect image data sets of different objects in the actual indoor environment, calibrate the images, and divide the data sets into training sets, verification sets, and test sets;

[0033] A lightweight network model building module, which is used to build a lightweight network model based on the YOLOv4-tiny network structure, including a feature extraction backbone network, a hybrid channel attention mechanism module, and a decoupling head module;

[0034] A network model training module, used to train the lightweight network model using a training set and a validation set, and save the model weights when the loss function converges;

[0035] The test module is used to load the saved model weights and use the trained lightweight network model to identify indoor objects;

[0036] A path planning module is used to plan the motion path of the sweeping robot according to the recognition results;

[0037] Among them, the feature extraction backbone network includes five downsamplings, two of which are convolutions with a step size of 2 and three of which are maximum pooling structures; the 10th, 18th, and 26th layers of the feature extraction backbone network are maximum pooling structures, and mixed channel attention mechanism modules are added after the 24th and 27th layers of the feature extraction backbone network, respectively, and the 31st and 38th layers of the feature extraction backbone network are decoupling head modules.

[0038] The present invention also provides a sweeping robot, comprising the above-mentioned lightweight network-based target recognition system.

[0039] It can be seen from the above technical solutions that the present invention has the following advantages:

[0040] The embodiment of the present invention provides a target recognition method, system and sweeping robot based on a lightweight network. The present invention is based on the YOLOv4-tiny network structure, and establishes a lightweight network model including a feature extraction backbone network, a hybrid channel attention mechanism module and a decoupling head module. The maximum pooling structure is introduced in the feature extraction backbone network, and the suboptimal features are retained while downsampling, which effectively improves the feature extraction ability of the backbone network; the hybrid channel attention mechanism module is introduced in the neck of the feature extraction backbone network, and the deep semantic information is refined from the two perspectives of channel and space, which effectively improves the weight of important features and pays attention to key information; the decoupling head module is introduced to process the classification and regression tasks separately; and the weighted Focal Loss loss function is introduced to solve the problems of imbalance between positive and negative samples and imbalance between difficult and easy samples. The lightweight network model proposed by the present invention has stronger generalization ability and higher recognition accuracy, and is suitable for target detection of sweeping robot, which greatly improves the detection accuracy while ensuring high real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the implementation cases of the present invention or the technical solutions in the prior art, the following is a brief description of the drawings required for use in the embodiments. By referring to the drawings, the features and advantages of the present invention will be more clearly understood. The drawings are schematic and should not be understood as limiting the present invention in any way. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0042] Figure 1 This is a schematic diagram of the structure of the YOLOv4-tiny network model;

[0043] Figure 2 is a flowchart of a target recognition method based on a lightweight network provided in an embodiment;

[0044] Figure 3 A schematic diagram of the overall architecture of a lightweight network model (YOLOSR-tiny) provided in an embodiment of the present invention;

[0045] Figure 4 Graph showing the experimental results of the difficult samples in the embodiment, where Figure 4 (a) is the detection accuracy diagram of difficult samples under different weights. Figure 4 (b) is a schematic diagram showing that when α=0.5, γ=0.5, the accuracy of the hard sample is the highest, which is 70.45%;

[0046] Figure 5This is a recognition effect diagram of the sweeping robot in the embodiment of the target detection in the indoor home environment;

[0047] Figure 6 A block diagram of a lightweight network-based target recognition system provided in an embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] like Figure 2 As shown, an embodiment of the present invention proposes a target recognition method based on a lightweight network, the method comprising:

[0050] S1: Collect image datasets of different objects in real indoor environments, calibrate the images, and divide the datasets into training sets, validation sets, and test sets;

[0051] S2: Based on the YOLOv4-tiny network structure, a lightweight network model including a feature extraction backbone network, a hybrid channel attention mechanism module, and a decoupling head module is established;

[0052] S3: Train the lightweight network model using the training set and the validation set, and save the model weight when the loss function converges;

[0053] S4: Load the saved model weights and use the trained lightweight network model to identify indoor objects;

[0054] S5: planning the motion path of the sweeping robot according to the recognition result;

[0055] Among them, the feature extraction backbone network includes five downsamplings, two of which are convolutions with a step size of 2 and three of which are maximum pooling structures; the 10th, 18th, and 26th layers of the feature extraction backbone network are maximum pooling structures, and mixed channel attention mechanism modules are added after the 24th and 27th layers of the feature extraction backbone network, respectively, and the 31st and 38th layers of the feature extraction backbone network are decoupling head modules.

[0056] The present invention provides a target recognition method based on a lightweight network. The present invention is based on the YOLOv4-tiny network structure, and establishes a lightweight network model including a feature extraction backbone network, a hybrid channel attention mechanism module, and a decoupling head module. The maximum pooling structure is introduced in the feature extraction backbone network, and the suboptimal features are retained while downsampling, which effectively improves the feature extraction ability of the backbone network; the hybrid channel attention mechanism module is introduced in the neck of the feature extraction backbone network, and the deep semantic information is refined from the two perspectives of channel and space, which effectively improves the weight of important features and focuses on key information; the decoupling head module is introduced to process the classification and regression tasks separately; and the weighted Focal Loss loss function is introduced to solve the problems of imbalance between positive and negative samples and imbalance between difficult and easy samples. The lightweight network model proposed by the present invention has stronger generalization ability and higher recognition accuracy, and is suitable for target detection of sweeping robot, which greatly improves the detection accuracy while ensuring high real-time performance.

[0057] like Figure 3 As shown in the figure, a new lightweight network model YOLOSR-tiny is proposed based on the YOLOv4-tiny network structure, which specifically includes a feature extraction backbone network, a hybrid channel attention mechanism module (Convolutional BlockAttention Module, CBAM) and a decoupled head module (Decoupled Head, DH).

[0058] The backbone network of the YOLOv4-tiny network structure contains five downsamplings, two of which are convolutions with a step size of 2 and three maximum poolings. Directly using maximum pooling only retains the optimal features, but in a lightweight network, suboptimal features are equally important. Therefore, the lightweight network model YOLOSR-tiny proposed in this invention introduces a maximum pooling structure (Improvedmax-pooling, IM), such as Figure 3 As shown in ①. The structure consists of two parallel branches. The first branch is a maximum pooling combined with a convolution with a step size of 1, and the second branch is a convolution with a step size of 2 combined with a convolution with a step size of 1. In order to ensure that the number of input channels and output channels is equal, the number of channels is halved in the two branches through a convolution with a step size of 1, and then the channels are downsampled separately and fused again to restore the original number of channels.

[0059] The CBAM module is added to the neck of the lightweight network model YOLOSR-tiny, such as Figure 3As shown in ②, the CBAM module includes a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism performs maximum pooling and average pooling on the input feature map from a spatial perspective, and sends it to a shared network. Finally, the output is multiplied by the input feature map to obtain the output of the channel attention mechanism; the input of the spatial attention mechanism is the output of the channel attention mechanism, which performs maximum pooling and average pooling on the new input feature map from a channel perspective, and concatenates the two results. After the convolution operation, it is multiplied again with the new input feature map to obtain the final output.

[0060] In the lightweight network model YOLOSR-tiny, the coupling head (CoupleHead, CH) module in the YOLOv4-tiny network structure is replaced by the DH module, such as Figure 3 As shown in ③, the DH module includes first using a 1×1 convolution to compress the number of channels of the input feature map, and then adding two parallel branches. The first branch uses two 3×3 convolutions to process the classification task, and the second branch also uses two 3×3 convolutions to process the regression task, and the branch for detecting the object confidence is added to the branch of the regression task. Reduce the amount of calculation.

[0061] In the lightweight network model YOLOSR-tiny, the classification loss uses the Focal Loss function. The specific formula is:

[0062] FL(p t )=-α t (1-p t ) γ log(p t )

[0063] Among them, α is the weight for adjusting positive and negative samples, γ is the weight for adjusting difficult and easy samples, and p t Represents confidence. When the positive and negative samples are unbalanced, the distribution of the loss function will be tilted. Adding the weight factor α can increase the weight of the minority class in the loss function and balance the distribution of the loss function. Similarly, the weight factor γ can increase the weight of the difficult sample in the loss function, making the loss function tend to the difficult sample, which helps to improve the accuracy of the difficult sample.

[0064] Specifically, as shown in Table 1, the 10th, 18th, and 26th layers of the feature extraction backbone network are maximum pooling structures (IM), and hybrid channel attention mechanism modules (CBAM) are added after the 24th and 27th layers of the feature extraction backbone network, respectively, and the 31st and 38th layers of the feature extraction backbone network are decoupling head modules (DH).

[0065] Table 1

[0066]

[0067]

[0068] In order to achieve the purpose of the present invention, as shown in Table 2, in one embodiment of the present invention, specific experimental results of the improved strategy are provided. First, the effectiveness of each part of the proposed method is verified by ablation experiments. The main parameters in the experiment are set as follows: the pre-trained weight is set to empty (the new network architecture cannot load the pre-trained weight), and trained for 400 epochs, the block size is set to 16, the input image size is 512×512×3, the momentum is set to 0.9, the weight decay regularization coefficient is set to 0.0005, the initial learning rate α0 is set to 0.001, and during the training process, according to the formula α=0.95 epoch _num ·α0 gradually reduces the learning rate. The computer configuration is as follows: the processor is Intel Core i9-9900K CPU3.6GHz, the graphics card is GeForce RTX 2080Ti, the video memory capacity is 11GB, the type is GDDR 6, the memory is 32G, the system is Ubuntu18.04, and the model training takes about 14 hours to complete.

[0069] The dataset used in the experiment is a benchmark dataset for sweeping robots, totaling 11,104 images, of which 80% are training sets, totaling 8,883 images, 10% are validation sets, totaling 1,110 images, and 10% are test sets, totaling 1,110 images. The specific implementation is as follows:

[0070] 1. By adding the maximum pooling (IM) structure alone, the accuracy increased from 77.65% to 79.14%, the number of parameters increased by 1.12M, and the speed reached 368FPS.

[0071] 2. By adding the decoupling head (DH) module alone, the accuracy increased from 77.65% to 83.16%, the number of parameters increased by 15.92M, and the speed reached 222FPS.

[0072] 3. By adding the hybrid channel attention mechanism (CBAM) module alone, the accuracy increased from 77.65% to 80.99%, the number of parameters increased by 0.09M, and the speed reached 335FPS.

[0073] 4. By adding the IM structure and DH module at the same time, the accuracy can be improved to 84.80%, the parameter volume is 22.95M, and the speed reaches 198FPS.

[0074] 5. By adding the IM structure, DH module and CBAM module at the same time, the accuracy can be improved to 86.50%, the parameter volume is 23.0M, and the speed reaches 180FPS.

[0075] 6. After adding all modules (structures), the accuracy can reach 86.12% and the speed is 180FPS. The addition of the FL module is mainly to improve the accuracy of difficult examples, so the slight decrease in mAP has almost no effect on the recognition of the sweeping robot.

[0076] Table 2

[0077]

[0078]

[0079] Experimental results show that the size of the YOLOSR-tiny model increases very little, but its accuracy is greatly improved, which fully demonstrates the high efficiency of the model.

[0080] In order to further verify the effectiveness of the present invention, the present invention optimizes the two weights in the loss function. When α=0.5,γ=0.5, the result is optimal. The experimental results are shown in Table 3. When α=0.2,γ=0.75, the overall average accuracy is only 58%, and the detection accuracy of small objects is the highest, with a value of 16.5%; when α=0.5,γ=0.75, the overall average accuracy is only 57.8%; when α=0.5,γ=0.5, the overall average accuracy is the highest, reaching 59.5%. When the intersection-over-union ratio is changed to 0.5 and 0.75 respectively, the accuracy remains the highest, which is 86.1% and 67.0% respectively. In addition, for the detection accuracy of large, medium and small scale objects, the accuracy of medium scale and large scale is also the highest, which is 37.1% and 64.3% respectively. When the weight α is further increased and β is reduced, the accuracy is no longer improved.

[0081] Table 3

[0082]

[0083] In the present invention, the wire object to be identified becomes a difficult sample due to its irregular shape. The detection accuracy of the difficult sample can be effectively improved by adjusting the above weights. Figure 4 As shown in (a), the horizontal axis 1 represents α = 0.75, γ = 0.2, 2 represents α = 0.75, γ = 0.5, and so on. When α = 0.5, γ = 0.5, the accuracy of the hard example is the highest, which is 70.45%. Figure 4 (b) as shown.

[0084] In one embodiment, the present invention is mainly used for obstacle avoidance of a sweeping robot in an indoor home environment, and the process is mainly divided into four parts: First, a data set is collected in an actual indoor environment, totaling 11,104 pictures and 28,381 instances, including 18 types of objects, namely beds, sofas, cabinets, stools, tables, toilets, trash cans, slippers, wires, socks, carpets, books, feces, fully open doors, half-open doors, closed doors, stairs and curtains. Then, the pictures are manually calibrated, and the data set is divided into a training set, a validation set and a test set, with a ratio of 80%, 10% and 10% respectively, and data enhancement is performed before the model is loaded with the training set. Secondly, the enhanced data set is loaded, the anchor frame of the self-made data set is calculated using a clustering algorithm, and the hyperparameters of the training are set accordingly. In order to speed up the convergence of the network, the pre-trained weights are loaded in the initialization stage for fine-tuning, and the model training ends when the loss function of the network converges. Again, the constructed model is trained using the test set, and verified by the validation set, and the model weights when the loss function converges are saved. In the test phase, the trained network model weights are loaded, and the test set is input into the trained lightweight network model. Through the judgment of the confidence threshold, the bounding box coordinate position, confidence and category probability of all prediction boxes that may contain obstacle avoidance objects are obtained. Then, redundant detection boxes are removed through non-maximum suppression to obtain the final detection result. Finally, the motion path of the sweeping robot is planned according to the detection results to achieve autonomous obstacle avoidance.

[0085] like Figure 5 As shown in the figure, the recognition effect diagram of the target detection of the sweeping robot in the actual scene, the different detection frames in the picture represent the detection of different object categories, for example, the detection frame in ①) recognizes the stairs, the detection frame in ② recognizes the books, and the detection frame in ⑤ recognizes the trash can and the toilet. It can be seen from the figure that the target detection model proposed in the present invention can correctly recognize multi-category and multi-scale objects in complex scenes, so as to guide the sweeping robot to effectively avoid obstacles.

[0086] like Figure 6 As shown, an embodiment of the present invention further provides a target recognition system based on a lightweight network, the system comprising:

[0087] The data collection and processing module 100 is used to collect image data sets of different objects in the actual indoor environment, calibrate the images, and divide the data sets into a training set, a validation set, and a test set;

[0088] A lightweight network model building module 200 is used to build a lightweight network model including a feature extraction backbone network, a hybrid channel attention mechanism module and a decoupling head module based on a YOLOv4-tiny network structure;

[0089] A network model training module 300 is used to train the lightweight network model using a training set and a validation set, and save the model weights when the loss function converges;

[0090] The testing module 400 is used to load the saved model weights and use the trained lightweight network model to identify indoor objects;

[0091] A path planning module 500 is used to plan the motion path of the sweeping robot according to the recognition result;

[0092] Among them, the feature extraction backbone network includes five downsamplings, two of which are convolutions with a step size of 2 and three of which are maximum pooling structures; the 10th, 18th, and 26th layers of the feature extraction backbone network are maximum pooling structures, and mixed channel attention mechanism modules are added after the 24th and 27th layers of the feature extraction backbone network, respectively, and the 31st and 38th layers of the feature extraction backbone network are decoupling head modules.

[0093] The system is used to implement the above-mentioned target recognition method based on a lightweight network. In order to avoid redundancy, it will not be described here.

[0094] The present invention also provides a sweeping robot, which includes the above-mentioned lightweight network-based target recognition system.

[0095] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

[0096] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0097] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0098] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0099] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A target recognition method based on a lightweight network, characterized in that: include: S1: Collect image datasets of different objects in real indoor environments, calibrate the images, and divide the datasets into training sets, validation sets, and test sets; S2: Based on the YOLOv4-tiny network structure, a lightweight network model including a feature extraction backbone network, a hybrid channel attention mechanism module, and a decoupling head module is established; S3: Train the lightweight network model using the training set and the validation set, and save the model weights when the loss function converges; The loss function adopts the Focal Loss function, and its formula is: in, is to adjust the weights of positive and negative samples, is to adjust the weight of difficult and easy samples, Indicates confidence, when , When , the network performance of the lightweight network model is optimal; S4: Load the saved model weights and use the trained lightweight network model to identify indoor objects; S5: planning the motion path of the sweeping robot according to the recognition result; Among them, the feature extraction backbone network contains five downsamplings, two of which are convolutions with a step size of 2 and three of which are maximum pooling structures; the maximum pooling structure consists of two parallel branches, the first branch is maximum pooling combined with convolutions with a step size of 1, and the second branch is convolutions with a step size of 2 combined with convolutions with a step size of 1; the 10th, 18th, and 26th layers of the feature extraction backbone network are maximum pooling structures, and mixed channel attention mechanism modules are added after the 24th and 27th layers of the feature extraction backbone network, respectively, and the 31st and 38th layers of the feature extraction backbone network are decoupling head modules.

2. The target recognition method based on lightweight network according to claim 1 is characterized in that: The hybrid channel attention mechanism module includes a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism performs maximum pooling and average pooling on the input feature map from a spatial perspective, and sends it to a shared network. Finally, the output is multiplied by the input feature map to obtain the output of the channel attention mechanism. The input of the spatial attention mechanism is the output of the channel attention mechanism. It performs maximum pooling and average pooling on the new input feature map from a channel perspective, and concatenates the two results. After the convolution operation, it is multiplied again with the new input feature map to obtain the final output.

3. The target recognition method based on lightweight network according to claim 1 is characterized in that: The decoupling head module is used to reduce the amount of calculation. The decoupling head module includes first using a 1×1 convolution to compress the number of channels of the input feature map, and then adding two parallel branches. The first branch uses two 3×3 convolutions to process the classification task, and the second branch also uses two 3×3 convolutions to process the regression task, and the branch for detecting the object confidence is added to the branch of the regression task.

4. The target recognition method based on lightweight network according to claim 1 is characterized in that: Before training the improved lightweight network model, data augmentation operations are performed on the training set.

5. The target recognition method based on lightweight network according to claim 1 is characterized in that: The step S4 specifically includes: In the testing phase, the trained network model weights are loaded, and the test set is input into the trained lightweight network model. By judging the confidence threshold, the bounding box coordinate position, confidence and category probability of all prediction boxes that may contain obstacle avoidance objects are obtained. Then, redundant detection boxes are removed through non-maximum suppression to obtain the final detection result.

6. The target recognition method based on lightweight network according to claim 1 is characterized in that: The training set, validation set, and test set account for 80%, 10%, and 10%, respectively.

7. A target recognition system based on a lightweight network, characterized in that: The system is used to implement the lightweight network-based target recognition method according to any one of claims 1 to 6, comprising: The data collection and processing module is used to collect image data sets of different objects in the actual indoor environment, calibrate the images, and divide the data sets into training sets, verification sets, and test sets; A lightweight network model building module, which is used to build a lightweight network model based on the YOLOv4-tiny network structure, including a feature extraction backbone network, a hybrid channel attention mechanism module, and a decoupling head module; A network model training module, used to train the lightweight network model using a training set and a validation set, and save the model weights when the loss function converges; The test module is used to load the saved model weights and use the trained lightweight network model to identify indoor objects; A path planning module is used to plan the motion path of the sweeping robot according to the recognition results; Among them, the feature extraction backbone network includes five downsamplings, two of which are convolutions with a step size of 2 and three of which are maximum pooling structures; the 10th, 18th, and 26th layers of the feature extraction backbone network are maximum pooling structures, and mixed channel attention mechanism modules are added after the 24th and 27th layers of the feature extraction backbone network, respectively, and the 31st and 38th layers of the feature extraction backbone network are decoupling head modules.

8. A sweeping robot, characterized in that: Including the lightweight network-based target recognition system described in claim 7.

Images