Multi-task learning method for intelligent inspection robot in data center

Abstract

The present invention provides a multi-task learning method for a data center intelligent inspection robot, which uses a unified deep neural network model architecture to detect QR codes and barcodes in images, segment servers, and identify server models based on images of server front panels. It includes the following steps: 1) construct a large number of data center server images from a component training set, manually annotate the images, and the images and annotation information form a training set; 2) use a deep learning programming framework to build a deep neural network model and set a loss function; 3) train the model; 4) deploy the trained model on the inspection robot, which greatly reduces the computing and storage resources required to complete the task, improves the robot's inspection efficiency, and also reduces the complexity of model deployment, facilitating engineering applications. Based on this model, the data center inspection robot can carry out high-level inspection tasks.

Description

Technical Field

[0001] The present invention relates to a multi-task learning method, in particular to a multi-task learning method for an intelligent inspection robot in a data center, and belongs to the field of information technology. Background Art

[0002] With the expansion of data center scale, the increase in manual inspection costs, and the improvement of robot intelligence, inspection robots have appeared in data center computer rooms to assist with manual inspections. However, the current inspection robot has a low intelligence level or a complex solution, which is not conducive to widespread promotion and has low economic benefits. Advanced inspection tasks generally include asset inventory, checking whether equipment assets such as servers are safe, whether the server is working properly, or what kind of abnormalities exist. These high-level tasks rely on visual information. Generally, robot inspections first use high-definition cameras to capture the server working screen, and then use various models or algorithms to perform independent analysis, and finally summarize the analysis results to form an inspection report. There are often similar architectures between multiple models, and the tasks are interrelated, with serious redundant calculations, resulting in cumbersome model deployment process, high model inference delay, serious memory usage, and long model training cycle. Summary of the invention

[0003] The purpose of the present invention is to provide a multi-task learning method for an intelligent inspection robot in a data center, which uses a unified deep neural network model architecture to detect QR codes and barcodes in images, segment servers, and identify server models based on images of server front panels.

[0004] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0005] A multi-task learning method for a data center intelligent inspection robot comprises the following steps:

[0006] 1) Component training set

[0007] Construct a large number of data center server images, manually annotate the images, and use the images and annotation information to form a training set;

[0008] 2) Use the deep learning programming framework to build a deep neural network model and set the loss function;

[0009] 3) Train the model;

[0010] 4) The trained model is deployed on the inspection robot

[0011] (4.1) After the robot takes an image, it is input into the trained model;

[0012] (4.2) The model outputs the location of the object in the image, segments the server front panel area, and calculates the feature vector of each server front panel;

[0013] (4.4) The upper layer service decodes the code according to the type and position of the code output by the code detection module and reads the information in the code;

[0014] (4.5) Further extract the server status information contained in the panel based on the area of ​​the server front panel, and retrieve the server with high similarity in the database based on the feature vector of the server front panel to identify the model of the server in order to perform asset inventory or knowledge graph retrieval.

[0015] The preferred solution of the multi-task learning method of the data center intelligent inspection robot is that the architecture of the deep neural network model consists of a unified encoder as the backbone network for common feature extraction modules, and three independent decoders are designed according to the characteristics of the tasks they are responsible for. The specific contents are as follows:

[0016] The feature extraction module is composed of a multi-layer convolutional neural network. The module is specifically divided into two sub-modules. The first sub-module progressively reduces the dimension of the input image layer by layer to extract abstract features. The data width and height decrease, and the number of feature channels increases. The second sub-module uses the abstract features output by the first sub-module for upsampling and performs feature fusion with the output of the three intermediate layers of the first sub-module to obtain three fusion layers.

[0017] Three independent decoders are responsible for detection tasks, semantic segmentation tasks, and recognition tasks respectively; for the code detection task, the outputs of the three feature fusion layers are used as the three inputs of the code detection module, and the three inputs are processed by a full convolutional neural network to form three branches. Each branch predicts the position and category of the QR code or barcode of the corresponding size; for the instance segmentation task, the feature map finally output by the second submodule of the feature extraction module is further upsampled to obtain a feature map of the same size as the original input image, and the probability of each pixel on the feature map belonging to the server or the background is predicted respectively; for the model recognition module, the areas in the feature map output by the segmentation module whose probability exceeds the threshold are segmented, that is, these areas are cut out as server features, and a multi-layer convolutional neural network is established for fine-grained feature encoding. The final feature map of the module is pooled channel by channel, that is, the maximum value of each channel is output to form the feature vector of the server.

[0018] The preferred solution of the multi-task learning method of the data center intelligent inspection robot and the training method of the deep neural network model include the following steps:

[0019] 1) Construct a loss function for the output of the code detection module and the annotation information of the sample. The loss function consists of two parts: one is the position regression error loss function, and the other is the category error loss function. The weighted sum of the two constitutes the loss function of the code detection module;

[0020] 2) Construct a semantic segmentation loss function based on the output of the segmentation module and the sample annotation information. The loss function calculates the category error of each pixel;

[0021] 3) The loss function of the recognition module is a similarity measurement function. When the feature map area input to the module is the same model of server, the similarity of the output feature vector should be at a higher level, otherwise it should be at a lower level. The loss function value that meets this fact is smaller than the loss function value that does not meet this fact.

[0022] 4) Perform a weighted sum of the loss functions obtained in the above steps as the global loss function.

[0023] The advantages of the present invention are:

[0024] A unified deep neural network model architecture is used to detect QR codes and barcodes in images, segment servers, and identify server models based on images of the server front panels, so that high-level inspection tasks can be carried out. The three tasks of detection, semantic segmentation, and model recognition are unified in one model. Compared with non-unified conventional models, this model greatly reduces the computing and storage resources required to complete the task, improves the robot's inspection efficiency, and also reduces the complexity of model deployment, facilitating engineering applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0026] Figure 1 A schematic diagram of a server cabinet according to an embodiment of the present invention.

[0027] Figure 2 Schematic diagram of a multi-task model architecture according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] A multi-task learning method for a data center intelligent inspection robot comprises the following steps:

[0030] 1. Component training set

[0031] like Figure 1 ,Build a large number of data center server images. The content in the images includes complete servers, QR codes, barcodes and other objects to be detected, segmented or recognized. ,Manually annotate the images, use rectangular boxes to represent the positions of QR codes or barcodes, use polygonal areas to represent the server front panel areas, and record the categories of QR codes and barcodes, as well as the models corresponding to each server front panel. ,The images and annotation information constitute the training set;

[0032] 2. Use deep learning programming framework to build deep neural network model, such as Figure 2 , and set the loss function. The loss function is not described in detail here. It only needs to meet the characteristics of the invention. The model is trained according to the method of the invention combined with the data set in step 1;

[0033] 3. Train the model;

[0034] 4. The trained model is deployed on the inspection robot

[0035] (4.1) After the robot takes an image, it is input into the trained model;

[0036] (4.2) The model outputs the location of the object in the image, segments the server front panel area, and calculates the feature vector of each server front panel;

[0037] (4.4) The upper layer service decodes the code according to the type and position of the code output by the code detection module and reads the information in the code;

[0038] (4.5) Further extract the server status information contained in the panel based on the area of ​​the server front panel, and retrieve the server with high similarity in the database based on the feature vector of the server front panel to identify the model of the server in order to perform asset inventory or knowledge graph retrieval.

[0039] In this embodiment, the architecture of the deep neural network model consists of a unified encoder as the backbone network for common feature extraction modules, and three independent decoders are designed according to the characteristics of their respective tasks. The specific contents are as follows:

[0040] The feature extraction module is composed of a multi-layer convolutional neural network. The module is specifically divided into two sub-modules. The first sub-module progressively reduces the dimension of the input image layer by layer to extract abstract features. The data width and height decrease, and the number of feature channels increases. The second sub-module uses the abstract features output by the first sub-module for upsampling and performs feature fusion with the output of the three intermediate layers of the first sub-module to obtain three fusion layers.

[0041] Three independent decoders are responsible for detection tasks, semantic segmentation tasks, and recognition tasks respectively; for the code detection task, the outputs of the three feature fusion layers are used as the three inputs of the code detection module, and the three inputs are processed by a full convolutional neural network to form three branches. Each branch predicts the position and category of the QR code or barcode of the corresponding size; for the instance segmentation task, the feature map finally output by the second submodule of the feature extraction module is further upsampled to obtain a feature map of the same size as the original input image, and the probability of each pixel on the feature map belonging to the server or the background is predicted respectively; for the model recognition module, the areas in the feature map output by the segmentation module whose probability exceeds the threshold are segmented, that is, these areas are cut out as server features, and a multi-layer convolutional neural network is established for fine-grained feature encoding. The final feature map of the module is pooled channel by channel, that is, the maximum value of each channel is output to form the feature vector of the server.

[0042] In this embodiment, the training method of the deep neural network model includes the following steps:

[0043] 1) Construct a loss function for the output of the code detection module and the annotation information of the sample. The loss function consists of two parts: one is the position regression error loss function, and the other is the category error loss function. The weighted sum of the two constitutes the loss function of the code detection module;

[0044] 2) Construct a semantic segmentation loss function based on the output of the segmentation module and the sample annotation information. The loss function calculates the category error of each pixel;

[0045] 3) The loss function of the recognition module is a similarity measurement function. When the feature map area input to the module is the same model of server, the similarity of the output feature vector should be at a higher level, otherwise it should be at a lower level. The loss function value that meets this fact is smaller than the loss function value that does not meet this fact.

[0046] 4) Perform a weighted sum of the loss functions obtained in the above steps as the global loss function.

[0047] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-task learning method for a data center intelligent inspection robot, characterized in that: The steps include: Construct a training set; Construct a large number of data center server images, manually annotate the images, and use the images and annotation information to form a training set; Use the deep learning programming framework to build a deep neural network model and set the loss function; Train the model; The trained model is deployed on the inspection robot. The steps are as follows: When the robot takes an image, it is input into the trained model; The model outputs the location of the object in the image, segments the server front panel area, and calculates the feature vector of each server front panel; The upper layer service decodes the code according to the type and position of the code output by the code detection module and reads the information in the code; Further extract the server status information contained in the panel according to the area of ​​the server front panel, and search the database for servers with high similarity according to the feature vector of the server front panel to identify the model of the server, so as to perform asset inventory or knowledge graph retrieval; The training method of the deep neural network model is as follows: 1) Construct a loss function based on the output of the code detection module and the annotation information of the sample. The loss function consists of two parts: one is the position regression error loss function, and the other is the category error loss function. The weighted sum of the two parts constitutes the loss function of the code detection module; 2) Construct a semantic segmentation loss function based on the output of the server segmentation module and the sample annotation information. The loss function calculates the category error of each pixel; 3) The loss function of the model recognition module is a similarity measurement function. When the feature map area input to the module is a server of the same model, the similarity of the output feature vector should be at a higher level, otherwise it should be at a lower level. The loss function value when this fact is met is smaller than the loss function value when this fact is not met. 4) The weighted sum of the loss functions obtained in the above steps is used as the global loss function; The architecture of the deep neural network model consists of a unified encoder and three independent decoders. The unified encoder serves as the backbone network for common feature extraction, and the three independent decoders are designed according to the characteristics of their respective tasks. The specific contents are as follows: The common feature extraction module is composed of a multi-layer convolutional neural network. The module is specifically divided into two sub-modules. The first sub-module progressively reduces the dimension of the input image layer by layer to extract abstract features. The data width and height decrease, and the number of feature channels increases. The second submodule uses the abstract features output by the first submodule for upsampling and fuses the features with the outputs of the three intermediate layers of the first submodule to obtain three fusion layers. Three independent decoders are responsible for code detection tasks, server segmentation tasks, and model recognition tasks respectively; For the code detection task, the outputs of the three feature fusion layers are used as the three inputs of the code detection module. The three inputs are processed by the full convolutional neural network to form three branches. Each branch predicts the position and category of the QR code or barcode of the corresponding size. For the server segmentation task, the feature map finally output by the second submodule of the feature extraction module is further upsampled to obtain a feature map of the same size as the original input image, and the probability of each pixel on the feature map belonging to the server or the background is predicted respectively. For the model recognition task, the areas with probability exceeding the threshold in the feature map output by the server segmentation module are segmented, that is, these areas are cut out and used as server features. A multi-layer convolutional neural network is established for fine-grained feature encoding, and the final feature map of the module is pooled channel by channel, that is, the maximum value of each channel is output to form the feature vector of the server.

Images