Service-oriented robot target identification method based on visual analysis

Abstract

The invention relates to the technical field of target recognition, in particular to a visual analysis-based service robot target recognition method, which comprises the following steps of: acquiring a training sample of a target object, and dividing a real frame, a body region and a background region of the training sample of the target object from the training sample of the target object; obtaining the diversity of the target object according to the contours of the body regions of all the training samples of the target object; and the optimal target value of each normal sample prediction frame of the target object is obtained by combining gray distribution in background areas of all training samples of the target object, and then a confidence loss function of the target object in a YOLO algorithm training stage is constructed, so that a target object recognition model is trained, and the service robot accurately recognizes the target object. According to the invention, a training stage in a traditional YOLO algorithm is optimized by analyzing forms and background features of various target objects, so that a model capable of accurately identifying various target objects in a complex and changeable environment is obtained through training.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target recognition, and particularly relates to a service robot target recognition method based on visual analysis. BACKGROUND

[0002] The service robot refers to the robot that directly provides beneficial services for human beings in a non-manufacturing environment. Compared with the industrial robot, the service robot emphasizes more on the interaction with human beings and autonomous operation in a complex and changeable environment. In order to enable the service robot to better autonomously operate in the complex and changeable environment, the service robot needs to have the ability to recognize various target objects in the complex and changeable environment.

[0003] The traditional YOLO algorithm-based recognition of various target objects is mainly based on that the IoU value (intersection over union) between the prediction box and the real box is used to distinguish the positive and negative samples in the training stage, and the IoU value is used as the main basis of the confidence loss function. The morphological differences of the target objects are not considered, such as the changes in size, contour and folding state of the same type of object, and the influence of the environment background of the target object is also ignored. Therefore, the traditional YOLO algorithm cannot accurately recognize various target objects in a complex and changeable environment. SUMMARY

[0004] The present application provides a service robot target recognition method based on visual analysis to solve the existing problem that the traditional YOLO algorithm cannot accurately recognize various target objects in a complex and changeable environment.

[0005] The service robot target recognition method based on visual analysis of the present application adopts the following technical scheme: The method comprises the following steps: Collecting training samples of target objects, and dividing real boxes, body regions and background regions of the training samples of the target objects from the training samples of the target objects; According to the contour of the body region of all the training samples of the target objects, the diversity of the target objects is obtained, and then all the negative sample prediction boxes, positive sample prediction boxes of the target objects and the IoU value between the real box of the target objects and each positive sample prediction box thereof are obtained; According to the gray scale distribution in the background region of all the training samples of the target objects, the texture vector of the background region of all the training samples of the target objects is constructed. According to the texture vector of the background region of different training samples of the target objects, the background weight of each training sample of the target objects is obtained, the best target value of each positive sample prediction box of the target objects is obtained by combining the IoU value between the real box of the target objects and each positive sample prediction box thereof and the diversity of the target objects, and then the confidence loss function of the target objects in the training stage of the YOLO algorithm is constructed; The confidence loss function of the target object in the training phase of the YOLO algorithm is used to train the target object recognition model, so that the service robot can accurately recognize the target object.

[0006] Preferably, the training sample of the target object is collected, and the real frame, the body region and the background region of the training sample of the target object are divided from the training sample of the target object, including the specific method: All the items to be identified are recorded as target objects, and a preset number of learning samples is set The training sample threshold is set For the first target object, collect images as learning samples, so that at least images containing the first target object are included in all learning samples as the training sample of the first target object. For the first target object, the real frame and the body region of the first training sample of the first target object are marked by artificial marking, and the region of the first training sample of the first target object except the body region is recorded as the background region of the first training sample of the first target object.

[0007] Preferably, the diversity of the target object is obtained according to the contour of the body region of all training samples of the target object, including the specific method: For the first target object, the convex hull detection result of the body region of each training sample of the first target object is obtained by using the convex hull detection algorithm. For the first training sample of the first target object, the ratio of the number of pixel points in the body region of the first training sample of the first target object to the number of pixel points in the convex hull detection result of the body region of the first training sample of the first target object is taken as the regularity of the body region of the first training sample of the first target object. The regularity of the body region of all training samples of the first target object is obtained; and the regularity of the body region of the first ​The information entropy of the regularity of the ontology region of all training samples of the target object is mapped to the interval from -1 to 1, and the mapping result of the information entropy is used as the first... The diversity of target objects.

[0008] Preferably, the specific method for obtaining all negative sample prediction boxes, positive sample prediction boxes, and the actual bounding box of the target object and the IoU value between each of its positive sample prediction boxes is as follows: Preset an initial IoU threshold; for the th The target object, the first The product of the diversity of each target species and 1 multiplied by the initial IoU threshold is used as the product of the diversity of each target species and 1. IoU threshold for a target object; According to the The IoU threshold of the target object is obtained by the YOLO algorithm. Several positive and several negative prediction boxes for a target object are generated, and the first prediction box is obtained. The IoU value between the ground truth bounding box of a target object and each of its positive prediction bounding boxes.

[0009] Preferably, the specific method for constructing the texture vector of the background region of all training samples of the target object based on the grayscale distribution of the background region of all training samples of the target object is as follows: For the The first type of target object The training sample is obtained. The first type of target object The gray-level co-occurrence matrix of the background region of each training sample; according to the... The first type of target object The energy and entropy of the gray-level co-occurrence matrix of the background region of the training samples are used to construct the first... The first type of target object Texture vectors of the background region of each training sample ,in Indicates the first The first type of target object Texture vectors of the background region of each training sample Indicates the first The first type of target object The energy of the gray-level co-occurrence matrix of the background region of a training sample Indicates the first The first type of target object The entropy of the gray-level co-occurrence matrix of the background region of a training sample.

[0010] Preferably, the specific method for obtaining the background weight of each training sample of the target object based on the texture vector of the background region of different training samples of the target object is as follows: In the formula, Indicates the first The first type of target object Background weights for each training sample; Indicates the first Consistency of the target object's environmental background; Indicates the first The number of training samples for each target object; Indicates the first The first type of target object Texture vectors of the background region of each training sample; Indicates the first The first type of target object Texture vectors of the background region of each training sample; This represents the cosine function.

[0011] Preferably, the specific method for obtaining the optimal target value for each positive prediction box of the target object includes: For the The first type of target object The first positive prediction box will be the first... The true bounding box of the target object and its first The IoU value between the first positive prediction boxes, multiplied by the first... The true bounding box of the target object and its first The product of the background weights of the training samples corresponding to the predicted bounding boxes is used as the product of the predicted bounding boxes and the background weights of the training samples. The first type of target object The target value for environmental correction of the first positive prediction frame; subtract the first from 1. The difference in diversity of target species multiplied by the first The first type of target object The product of the environmental correction target values ​​of the first positive prediction box is used as the product of the first positive prediction box. The first type of target object The optimal target value for a positive prediction box.

[0012] Preferably, the specific method for constructing the confidence loss function of the target object in the YOLO algorithm training phase is as follows: For the For each target object, a target value of a pre-defined negative sample prediction box is used. ;Will As the first The target value of each negative sample prediction box for each type of target object; the YOLO algorithm is used to obtain the target value of the first type of target object. The confidence prediction values ​​for each positive and negative prediction box of the target object; based on the first... The confidence prediction values ​​of each positive and negative prediction box for each target object, and the first... The target value of each negative sample prediction box for each target object and the first target object The first type of target object The optimal target value of each positive prediction bounding box is used to construct the YOLO algorithm training phase. The confidence loss function for a target object.

[0013] Preferably, the YOLO algorithm training phase 1 The specific method for constructing the confidence loss function for a target object is as follows: In the formula, This indicates the training phase of the YOLO algorithm. Confidence loss function for a target object; Indicates the first The number of positive prediction frames for a target object; Indicates the first The number of negative sample prediction frames for a target object; Indicates the first The first type of target object The optimal target value for a positive prediction box; Indicates the first The first type of target object The target value of each negative sample prediction box; Indicates the first The first type of target object The confidence prediction value of a positive prediction box; Indicates the first The first type of target object The confidence prediction value of each negative sample prediction box; This represents the binary cross-entropy loss function.

[0014] Preferably, the specific method for training the target object recognition model based on the confidence loss function of the target object during the YOLO algorithm training phase includes: For the For each target object, the YOLO network model parameters are iteratively optimized multiple times using the stochastic gradient descent algorithm to minimize the overall loss until the loss function converges, at which point the training process stops, yielding the final result. A target object recognition model.

[0015] The beneficial effects of the technical solutions of the present application are as follows: the present application divides the real box, the body region and the background region of the training sample of the target object in the training sample of the target object; the traditional training process of the recognition model for recognizing various target objects in the training stage of the YOLO algorithm does not consider the morphological differences of different target objects, wherein the more diverse the morphology of the target object in the training sample of the target object is, the less the morphology of the target object in the training sample of the target object can cover all morphologies of the target object under normal circumstances, and thus the more likely it is to appear a state not covered by the training sample when actually recognizing the target object with diverse morphology, and thus in order to train the recognition model capable of accurately recognizing various target objects, it is necessary to analyze and train in combination with the diversity of the morphology of various target objects, and the more diverse the morphology of the target object is, the smaller the target value of the positive sample prediction box of the target object should be, so as to obtain a model capable of accurately recognizing the target object in a complex and changeable environment.

[0016] It needs to be further explained that generally the positions of various target objects have certain rules, that is, the background environment of the target object also contains important information for training the target object recognition model, and in order to train a model capable of accurately recognizing various target objects in a complex and changeable environment, when the background environment of the target object is more unified, the background region of the target object can be used as a basis for recognizing the target object, at this time, the more consistent the background environment of a certain training sample of the target object with other training samples is, the more information capable of recognizing the target object is contained in the background region of the training sample of the target object, and thus the background weight of the training sample of the target object is obtained as a basis to construct the confidence loss function of the target object in the training stage of the YOLO algorithm; the recognition model of the target is trained to obtain a model capable of accurately recognizing various target objects in a complex and changeable environment. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without any creative effort.

[0018] Figure 1 The step flow chart of the target recognition method of the service robot based on visual analysis of the present application. DETAILED DESCRIPTION

[0019] In order to further clarify the technical means and effects taken by the present application to achieve the predetermined inventive purpose, the specific implementation, structure, features and effects of the target recognition method of the service robot based on visual analysis according to the present application are described in detail below in combination with the drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0021] The specific scheme of the target recognition method of the service robot based on visual analysis provided by the present application is described below in combination with the drawings.

[0022] Please refer to Figure 1 , which shows the step flowchart of the target recognition method of the service robot based on visual analysis provided by one embodiment of the present application, which includes the following steps: Step S001: Collect training samples of target objects, and divide the real frame, the body region and the background region of the training samples of the target objects from the training samples of the target objects.

[0023] It should be noted that the service robot refers to a robot that directly provides beneficial services to humans in a non-manufacturing environment. The service robot emphasizes more on interaction with humans and autonomous operation in a complex and variable environment compared to the industrial robot. In order to enable the service robot to better autonomously operate in a complex and variable environment, the service robot needs to have the ability to recognize various target objects in a complex and variable environment. Therefore, it is necessary to collect a number of images in the working environment of the service robot as learning samples for training various target object recognition models.

[0024] Specifically, all the objects that need to be recognized are recorded as target objects, and a learning sample quantity is preset , and the training sample threshold is set . The specific values of the above parameters can be set by oneself according to the actual situation, and the present embodiment does not make hard requirements. In the present embodiment, the values of , are taken as examples; for the first target object, a number of images are collected as learning samples, and at least learning samples containing the first target object are included in all learning samples as the first target object Training samples for a target object (the shape of the target object in the training samples is random. For example, if the target object is a towel, the training samples for towels include both images of towels laid flat and images of towels folded). Furthermore, regarding the first The first type of target object The training samples are labeled manually. The first type of target object The ground truth bounding boxes and ontology regions of the training samples, and the first training sample's... The first type of target object The region outside the body region within the ground truth bounding box of the training sample is denoted as the i-th training sample. The first type of target object The background region of each training sample.

[0025] It should be noted that the ground truth bounding box of the training sample contains the complete object body and the surrounding background environment, while the object body region of the training sample only contains the complete object body. The ground truth bounding box of the training sample is a feature in the training sample during the YOLO algorithm training phase.

[0026] Step S002: Based on the contours of the ontology regions of all training samples of the target object, obtain the diversity of the target object, and then obtain all negative sample prediction boxes, positive sample prediction boxes, and the IoU value between the target object's ground truth box and each of its positive sample prediction boxes.

[0027] It should be noted that traditional YOLO algorithm training models for recognizing various objects did not consider the morphological differences of different objects. The more diverse the shapes of the objects in the training samples, the less likely the training samples will cover all possible shapes of the objects under normal circumstances. Therefore, when actually recognizing objects with diverse shapes, it is more likely that the training samples will not cover all possible shapes. For example, towels have many shapes and high diversity, so the training samples for towels are less likely to cover all shapes of towels. Mobile phones have stable shapes and low diversity, so the training set for mobile phones can cover all shapes of mobile phones. Therefore, to train a recognition model that can accurately identify various objects, it is necessary to analyze and train the model in conjunction with the diversity of the shapes of various objects to obtain a recognition model that can accurately identify various objects.

[0028] It needs to be further explained that the training stage of the YOLO algorithm contains several iteration processes, and the embodiment is a target recognition method of a service robot based on visual analysis, specifically, the training process of the YOLO algorithm is optimized to obtain an identification model for accurately identifying various target objects. The analysis logic and actual operation steps of the training stage of the YOLO algorithm in each iteration process are the same, so in this embodiment, any iteration process in the training stage of the YOLO algorithm is taken as an example for description, and subsequent analysis and operation of the iteration process in the training stage of the YOLO algorithm are performed, so the specific iteration order will not be described again.

[0029] Preferably, in a specific embodiment of the present application, for the first target object, the convex hull detection result of the body region of each training sample of the first target object is obtained by using the convex hull detection algorithm. Since the convex hull detection algorithm is a known prior art, it will not be described again in this embodiment. For the first training sample of the first target object, the ratio of the number of pixel points in the body region of the first training sample of the first target object to the number of pixel points in the convex hull detection result of the body region of the first training sample of the first target object is taken as the regularity of the body region of the first training sample of the first target object. Further, the regularity of the body region of all training samples of the first target object is obtained. The information entropy of the regularity of the body region of all training samples of the first target object is mapped to the interval of -1 to 1, and the mapping result of the information entropy is taken as the diversity of the first target object. (In this embodiment, the premnmx function is used for mapping, and the specific mapping range is the information entropy of the regularity of the body region of all training samples of the first

[0030] It needs to be explained that the greater the ratio of the number of pixel points in the body region of the training sample of the target object to the number of pixel points in the convex hull detection result, the more full the body region of the training sample of the target object, and the more regular its shape. The greater the entropy of the regularity of the body region of all training samples of the target object, the more inconsistent the shape of each training sample of the target object, that is, the more diverse the shape of the target object.

[0031] It should be further explained that the greater the diversity of the target objects, the greater the differences in their appearance or boundary contours. In this case, if a low IoU threshold is used during the YOLO training phase, it is easy to misclassify predicted boxes that only roughly overlap with the ground truth bounding boxes as positive predicted boxes (positive predicted boxes are those with a high degree of overlap with the target object during the YOLO algorithm training phase, while negative predicted boxes are those with a low degree of overlap with the target object during the YOLO algorithm training phase). This reduces the ability of the final target object recognition model to distinguish the ground truth boundaries. In this case, it is necessary to increase the target object IoU threshold to ensure that predicted boxes that are closer to the ground truth bounding boxes are used as positive samples. This effectively improves the rigor and reliability of positive sample selection in highly diverse target categories, ensuring that the model can learn more accurate boundary features and enhance its robustness and generalization ability in complex environments.

[0032] Preferably, in a specific embodiment of the present invention, an initial IoU threshold is preset (the IoU threshold is a parameter in the YOLO algorithm training phase). The specific value of the initial IoU threshold can be set according to the actual situation. This embodiment does not make a hard requirement. In this embodiment, the initial IoU threshold is equal to 0.5 as an example for description. For the first... The target object, the first The product of the diversity of each target species and 1 multiplied by the initial IoU threshold is used as the product of the diversity of each target species and 1. IoU threshold for a target object; Furthermore, according to the first The IoU threshold of the target object is obtained by the YOLO algorithm. Several positive and several negative prediction boxes for a target object are generated, and the first prediction box is obtained. The IoU value between the ground truth bounding box of the target object and each of its positive and negative prediction bounding boxes (when obtaining the positive and negative prediction bounding boxes of the target object, the YOLO algorithm first calculates the IoU value between the ground truth bounding box of the target object and each prediction box, and then filters out the positive and negative prediction bounding boxes of the target object from the prediction boxes based on the calculated IoU value between the ground truth bounding box of the target object and each prediction box and the IoU threshold of the target object). Since the YOLO algorithm is a well-known prior art, it will not be described in detail in this embodiment.

[0033] At this point, we have obtained all the negative and positive prediction bounding boxes for each target object, as well as the IoU values ​​between the target object's true bounding box and each of its positive prediction bounding boxes.

[0034] Step S003: constructing a texture vector of the background region of each training sample of the target object according to the gray scale distribution in the background region of all training samples of the target object; obtaining a background weight of each training sample of the target object according to the texture vectors of the background regions of different training samples of the target object, combining the IoU value between the real box of the target object and each positive sample prediction box thereof, and the diversity of the target object, obtaining an optimal target value of each positive sample prediction box of the target object, and then constructing a confidence loss function of the target object in the training stage of the YOLO algorithm.

[0035] It should be noted that the embodiment is a service robot target recognition method based on visual analysis, and a model capable of accurately recognizing various target objects in a complex and variable environment needs to be trained. Generally, the positions of various target objects have certain rules, that is, the background environment of the target object also contains important information for training the target object recognition model. For example, a pillow is more credible on a bed than on a desk. In order to train a model capable of accurately recognizing various target objects in a complex and variable environment, the embodiment further analyzes the background environment of the target object in the training sample. When the background environment of the target object is more uniform, the background region of the target object can be used as a basis for recognizing the target object. At this time, the background environment of a certain training sample of the target object is more consistent with other training samples. The background region of the training sample of the target object contains more information capable of recognizing the target object. Therefore, the background weight of the training sample of the target object is obtained based on this, and the confidence loss function of the target object in the training stage of the YOLO algorithm is further constructed by combining the IoU value between the real box of the target object and each positive sample prediction box thereof and the diversity of the target object, which is used for training the target recognition model to obtain a model capable of accurately recognizing various target objects in a complex and variable environment.

[0036] Preferably, in a specific embodiment of the present application, for the first training sample of the first target object, the gray level co-occurrence matrix of the background region of the first training sample of the first target object is obtained; the texture vector of the background region of the first training sample of the first target object is constructed according to the energy and entropy of the gray level co-occurrence matrix of the background region of the first training sample of the first target object , wherein ​​​​​​​​​​​​​the energy of the gray level co-occurrence matrix of the background region of the i-th training sample of the j-th target object, the entropy of the gray level co-occurrence matrix of the background region of the i-th training sample of the j-th target object, the entropy of the gray level co-occurrence matrix of the background region of the i-th training sample of the j-th target object, the entropy of the gray level co-occurrence matrix of the background region of the i-th training sample of the j-th target object,

[0037] Further, for the i-th training sample of the j-th target object, the background weight of the i-th training sample of the j-th target object is obtained according to the texture vector of the background region of the i-th training sample of the j-th target object and other training samples, and the specific calculation formula is as follows: In the formula, Wij represents the background weight of the i-th training sample of the j-th target object;

[0038] It should be noted that the texture vector of the background region of the training sample of the target object represents the texture of the background region of the training sample of the target object, and the more similar the texture vectors of the background regions of different training samples of the target object are, the more similar the textures of the background regions of the training samples of the target object are; therefore the greater the value of Wij is, the more consistent the textures of the background regions of the training samples of the target object are; and when the textures of the background regions of the training samples of the target object are more consistent, and the background environment of a certain training sample of the target object is more consistent with other training samples, the background region of the training sample of the target object contains more information capable of identifying the target object, and therefore the greater the value of Wij is, the higher the background weight of the training sample of the target object is.

[0039] ​​​​​​​​​​​​​​​​​​​​It should be further explained that when the background weight of the training samples of the target object is higher, it means that the model is less affected by interference in recognizing the target object, the recognition task is simpler, and the recognition result is more reliable. Therefore, when training the model to recognize the target object, a larger target value can be set for the target object to guide the model to make more confident predictions, thereby enabling the model to output higher confidence and consolidate its correct judgment. Furthermore, by combining the IoU value between the ground truth bounding box of the target object and each of its normal prediction bounding boxes, as well as the diversity of the target object, the optimal target value of the normal prediction bounding box of the target object can be obtained.

[0040] Preferably, in a specific embodiment of the present invention, for the first The first type of target object The first positive prediction box will be the first... The true bounding box of the target object and its first The IoU value between the first positive prediction boxes, multiplied by the first... The true bounding box of the target object and its first The product of the background weights of the training samples corresponding to the predicted bounding boxes is used as the product of the predicted bounding boxes and the background weights of the training samples. The first type of target object The target value for environmental correction of the first positive prediction frame; subtract the first from 1. The difference in diversity of target species multiplied by the first The first type of target object The product of the environmental correction target values ​​of the first positive prediction box is used as the product of the first positive prediction box. The first type of target object The optimal target value for a positive prediction box.

[0041] As an example, obtaining the first The first type of target object The specific formula for calculating the optimal target value of a positive prediction box is as follows: In the formula, Indicates the first The first type of target object The optimal target value for a positive prediction box; Indicates the first The true bounding box of the target object and its first Each positive prediction bounding box corresponds to the background weight of the training sample; Indicates the first The true bounding box of the target object and its first The IoU value between each positive prediction bounding box; Indicates the first The diversity of target objects.

[0042] It should be noted that the greater the background weight of the training samples corresponding to the ground truth bounding box and the normal predicted bounding box of the target object, the larger the target value should be set for the optimal target value of the normal predicted bounding box of the target object, so as to guide the model to make more confident predictions. When the IoU value between the ground truth bounding box and the normal predicted bounding box of the target object is larger, it means that the normal predicted bounding box is more able to identify the location of the target object, and it needs to be stimulated more. However, the higher the diversity of the target objects during training, the more diverse the shapes of the target objects, and the less the training set can cover all the shapes of the target objects. This reflects the uncertainty caused by insufficient coverage of training samples, making the model more conservative in the learning process. Therefore, when the background weight of the training samples corresponding to the ground truth bounding box and the normal predicted bounding box of the target object is larger, the IoU value between the ground truth bounding box and the normal predicted bounding box of the target object is larger, and the diversity of the target objects is lower, the normal predicted bounding box of the target object should be given a larger target value, so as to obtain a model that can accurately identify the target objects in complex and changing environments.

[0043] It should be further explained that after obtaining the excitation value of each normal prediction box of the target object, the confidence loss function for training the recognition model of each target object can be constructed based on the excitation value of each normal prediction box of all target objects. Therefore, the recognition model of each target object is obtained through training, thereby enabling the service robot to accurately identify target objects in complex and ever-changing environments.

[0044] Preferably, in a specific embodiment of the present invention, for the first For each target object, a target value of a pre-defined negative sample prediction box is used. The The specific value can be set according to the actual situation. This embodiment does not make a hard requirement. In this embodiment, it is used as... Let's take an example to illustrate; As the first The target value of each negative sample prediction box for each type of target object; the YOLO algorithm is used to obtain the target value of the first type of target object. The confidence prediction values ​​for each positive and negative prediction box of the target object; based on the first... The confidence prediction values ​​of each positive and negative prediction box for each target object, and the first... The target value of each negative sample prediction box for each target object and the first target object The first type of target object The optimal target value of each positive prediction bounding box is used to construct the YOLO algorithm training phase. The confidence loss function for the target object, the first stage of the YOLO algorithm training phase The confidence loss function for the target object is: In the formula, represents the confidence loss function of the YOLO algorithm training stage target object; represents the number of positive sample prediction boxes of the target object; represents the number of negative sample prediction boxes of the target object; represents the number of positive sample prediction boxes of the target object; represents the number of negative sample prediction boxes of the target object; represents the optimal target value of the i-th positive sample prediction box of the target object; represents the target value of the i-th negative sample prediction box of the target object; represents the target value of the i-th negative sample prediction box of the target object; represents the confidence prediction value of the i-th positive sample prediction box of the target object; represents the confidence prediction value of the i-th negative sample prediction box of the target object; represents the confidence prediction value of the i-th negative sample prediction box of the target object; represents the confidence prediction value of the i-th negative sample prediction box of the target object; represents the confidence prediction value of the i-th negative sample prediction box of the target object; represents the confidence prediction value of the i-th negative sample prediction box of the target object; represents the confidence prediction value of the i-th negative sample prediction box of the target object; represents the confidence prediction value of the i-th negative sample prediction box of the target object; represents the confidence prediction value of the i-th negative sample prediction box of the target object; represents the confidence prediction value of the i-th negative sample prediction box of the target object; represents the confidence prediction value of the i-th negative sample prediction box of the target object.

[0045] It should be noted that for the positive sample prediction box, the target value is determined by the IoU of the real box and the prediction box, the environment weight and the target object diversity, to ensure that the model gives higher confidence learning to the effective prediction box in complex environment; for the negative sample prediction box, the target value is usually set to a fixed low value, to suppress the confidence of irrelevant or false prediction box. Then, the deviation between the predicted confidence of the positive and negative sample prediction boxes and the target value is calculated by the binary cross entropy loss function, and is weighted and accumulated in the overall loss, so as to promote the model to gradually improve the confidence prediction accuracy of the real target object and reduce the false detection rate in the training process.

[0046] Thus, the confidence loss function of the YOLO algorithm training stage target object is obtained.

[0047] Step S004: training the target object recognition model according to the confidence loss function of the YOLO algorithm training stage target object, so that the service robot can accurately recognize the target object.

[0048] It should be noted that after obtaining the confidence loss function of the YOLO algorithm training stage target object by step S003, the target object recognition model can be trained according to the confidence loss function of the YOLO algorithm training stage target object, so that the service robot can accurately recognize various target objects in complex and variable environment.

[0049] Specifically, for the i-th target object, The parameters of the YOLO network model are iteratively optimized for multiple rounds by using a random gradient descent algorithm to minimize the overall loss (including a confidence loss, a positioning loss and a classification loss), until the loss function converges, the training process stops, and the final first The target object recognition model, since the random gradient descent algorithm is a known prior art, will not be described in detail in this embodiment; Further, the first The target object recognition model is integrated into a vision processing unit of a service robot. In actual recognition, the robot captures real-time environment images through a camera; then, the images are input into the deployed recognition model to perform forward inference, and the model outputs a series of prediction boxes, each of which contains class information, a confidence score and boundary box coordinates.

[0050] Thus, the embodiment is completed.

[0051] The above merely provides the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A target recognition method for service robots based on visual analysis, characterized in that, The method includes the following steps: Collect training samples of the target object, and divide the training samples of the target object into the ground truth bounding box, the ontology region and the background region; Based on the contours of the ontology regions of all training samples of the target object, the diversity of the target object is obtained, and then all negative sample prediction boxes, positive sample prediction boxes, and the IoU values ​​between the target object's ground truth box and each of its positive sample prediction boxes are obtained. Based on the grayscale distribution of the background regions of all training samples of the target object, the texture vectors of the background regions of all training samples of the target object are constructed. Based on the texture vectors of the background regions of different training samples of the target object, the background weight of each training sample of the target object is obtained. Combining the IoU value between the ground box of the target object and each of its normal prediction boxes and the diversity of the target object, the optimal target value of each normal prediction box of the target object is obtained, and then the confidence loss function of the target object in the training stage of the YOLO algorithm is constructed. The target object recognition model is trained based on the confidence loss function of the target object during the training phase of the YOLO algorithm, so that the service robot can accurately identify the target object.

2. The service robot target recognition method based on visual analysis according to claim 1, characterized in that, The specific methods for collecting training samples of the target object and dividing the training samples of the target object into the ground truth bounding box, the body region, and the background region are as follows: All items that need to be identified are designated as target objects, and a preset number of learning samples is set. With training sample threshold For the first Target objects, collection Zhang images are used as learning samples, ensuring that all learning samples contain at least one image. Zhang Hanbao The training samples of the target object are used as the first Training samples of the target object; For the The first type of target object The training samples are labeled manually. The first type of target object The ground truth bounding boxes and ontology regions of the training samples, and the first training sample's... The first type of target object The region outside the body region within the ground truth bounding box of the training sample is denoted as the i-th training sample. The first type of target object The background region of each training sample.

3. The service robot target recognition method based on visual analysis according to claim 1, characterized in that, The method for obtaining the diversity of the target object based on the contours of the ontology regions of all training samples of the target object includes the following specific methods: For the For a target object, the convex hull detection algorithm is used to obtain the first target object. The convex hull detection results of the ontology region for each training sample of the target object; For the The first type of target object The training sample will be the first one. The first type of target object The number of pixels in the body region of the training sample, compared with the number of pixels in the training sample. The first type of target object The ratio of the number of pixels in the convex hull detection result of the body region of the training sample is used as the ratio of the number of pixels in the training sample's convex hull detection result to the number of pixels in the training sample's convex hull detection result. The first type of target object The regularity of the ontology region of each training sample; Get the The regularity of the ontology regions of all training samples of the target object; the degree of regularity of the ontology regions of the target object; The information entropy of the regularity of the ontology region of all training samples of the target object is mapped to the interval from -1 to 1, and the mapping result of the information entropy is used as the first... The diversity of target objects.

4. The service robot target recognition method based on visual analysis according to claim 1, characterized in that, The specific method for obtaining all negative sample prediction boxes, positive sample prediction boxes, and the IoU value between the target object and each of its actual boxes is as follows: Preset an initial IoU threshold; for the th The target object, the first The product of the diversity of each target species and 1 multiplied by the initial IoU threshold is used as the product of the diversity of each target species and 1. IoU threshold for a target object; According to the The IoU threshold of the target object is obtained by the YOLO algorithm. Several positive and several negative prediction boxes for a target object are generated, and the first prediction box is obtained. The IoU value between the ground truth bounding box of a target object and each of its positive prediction bounding boxes.

5. The service robot target recognition method based on visual analysis according to claim 1, characterized in that, The specific method for constructing the texture vector of the background region of all training samples of the target object based on the gray-level distribution within the background region of all training samples of the target object is as follows: For the The first type of target object The training sample is obtained. The first type of target object The gray-level co-occurrence matrix of the background region of each training sample; according to the... The first type of target object The energy and entropy of the gray-level co-occurrence matrix of the background region of the training samples are used to construct the first... The first type of target object Texture vectors of the background region of each training sample ,in Indicates the first The first type of target object Texture vectors of the background region of each training sample Indicates the first The first type of target object The energy of the gray-level co-occurrence matrix of the background region of a training sample Indicates the first The first type of target object The entropy of the gray-level co-occurrence matrix of the background region of a training sample.

6. The service robot target recognition method based on visual analysis according to claim 1, characterized in that, The specific method for obtaining the background weight of each training sample of the target object based on the texture vector of the background region of different training samples of the target object is as follows: In the formula, Indicates the first The first type of target object Background weights for each training sample; Indicates the first Consistency of the target object's environmental background; Indicates the first The number of training samples for each target object; Indicates the first The first type of target object Texture vectors of the background region of each training sample; Indicates the first The first type of target object Texture vectors of the background region of each training sample; This represents the cosine function.

7. The service robot target recognition method based on visual analysis according to claim 1, characterized in that, The specific method for obtaining the optimal target value for each positive prediction box of the target object is as follows: For the The first type of target object The first positive prediction box will be the first... The true bounding box of the target object and its first The IoU value between the first positive prediction boxes, multiplied by the first... The true bounding box of the target object and its first The product of the background weights of the training samples corresponding to the predicted bounding boxes is used as the product of the predicted bounding boxes and the background weights of the training samples. The first type of target object The target value for environmental correction of the first positive prediction frame; subtract the first from 1. The difference in diversity of target species multiplied by the first The first type of target object The product of the environmental correction target values ​​of the first positive prediction box is used as the product of the first positive prediction box. The first type of target object The optimal target value for a positive prediction box.

8. The service robot target recognition method based on visual analysis according to claim 1, characterized in that, The specific method for constructing the confidence loss function of the target object in the YOLO algorithm training phase is as follows: For the For each target object, a target value of a pre-defined negative sample prediction box is used. ;Will As the first The target value of each negative sample prediction box for each type of target object; the YOLO algorithm is used to obtain the target value of the first type of target object. The confidence prediction values ​​for each positive and negative prediction box of the target object; based on the first... The confidence prediction values ​​of each positive and negative prediction box for each target object, and the first... The target value of each negative sample prediction box for each target object and the first target object The first type of target object The optimal target value of each positive prediction bounding box is used to construct the YOLO algorithm training phase. The confidence loss function for a target object.

9. The service robot target recognition method based on visual analysis according to claim 8, characterized in that, The YOLO algorithm training phase The specific method for constructing the confidence loss function for a target object is as follows: In the formula, This indicates the training phase of the YOLO algorithm. Confidence loss function for a target object; Indicates the first The number of positive prediction frames for a target object; Indicates the first The number of negative sample prediction frames for a target object; Indicates the first The first type of target object The optimal target value for a positive prediction box; Indicates the first The first type of target object The target value of each negative sample prediction box; Indicates the first The first type of target object The confidence prediction value of a positive prediction box; Indicates the first The first type of target object The confidence prediction value of each negative sample prediction box; This represents the binary cross-entropy loss function.

10. The service robot target recognition method based on visual analysis according to claim 1, characterized in that, The specific method for training the target object recognition model based on the confidence loss function of the target object during the YOLO algorithm training phase includes: For the For each target object, the YOLO network model parameters are iteratively optimized multiple times using the stochastic gradient descent algorithm to minimize the overall loss until the loss function converges, at which point the training process stops, yielding the final result. A target object recognition model.

Images