A method, device and medium for monitoring usage of a data center network device

Abstract

The embodiment of the specification discloses a kind of data center network equipment's use monitoring method, equipment and medium, involve artificial intelligence technical field, method includes: obtaining the interface use image that inspection robot is collected at specified inspection task point, and obtain the specified template image corresponding to preset specified inspection task point, specified template image is the image under normal use condition;Interface use image and the specified template image are input into the interface use detection model pre-trained, and the detection annotation file of the same size as interface use image and the specified template image is output, and detection annotation file includes the class identifier of pixel point;The class identifier of each pixel point in detection annotation file is obtained, and the interface use state of network equipment at specified inspection task point is determined according to the class identifier of each pixel point, and the use of data center network equipment is monitored based on interface use state, interface use state includes abnormal use state and normal use state.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of artificial intelligence, and in particular, to a data center network equipment usage monitoring method, device and medium. BACKGROUND

[0002] With the development of Internet technology, data centers are becoming larger and larger. Large data centers have a large amount of network data transmission, and a large number of network lines are laid, which increases the difficulty of network communication equipment inspection. If there is abnormal use of network equipment, it will not be found in time, which will cause unnecessary loss and serious consequences, such as illegal private connection of network lines, theft of data center network resources, malicious removal of network lines to physically disconnect servers, and even theft of communication data. Under normal circumstances, full-time personnel are arranged for manual inspection, which cannot guarantee uninterrupted inspection on the one hand, and on the other hand, for the large number of network lines in the data center or the dense network lines on a certain switch or router, even if a private optical fiber appears, the human eye is easy to miss, causing missed inspection. For similar tasks, the use of inspection robots has obvious advantages.

[0003] Robot inspection mainly shoots equipment images through high-definition cameras to monitor the equipment. Due to abnormal use of equipment, especially interface abnormal use, the visual changes are small and there are no clear trace features, and the current detection accuracy and fullness rate are not high, which cannot timely and accurately monitor the interface abnormal use problem. SUMMARY

[0004] One or more embodiments of the present specification provide a data center network equipment usage monitoring method, device and medium, which are used to solve the following technical problems: due to abnormal use of equipment, especially interface abnormal use, the visual changes are small and there are no clear trace features, and the current detection accuracy and fullness rate are not high, which cannot timely and accurately monitor the interface abnormal use problem.

[0005] One or more embodiments of the present specification adopt the following technical solutions:

[0006] The one or more embodiments of the specification provide a method for monitoring the use of a data center network device, applied to an inspection robot, the method comprising: acquiring an interface use image collected by the inspection robot at a specified inspection task point, and acquiring a specified template image corresponding to the specified inspection task point set in advance, wherein the image size of the interface use image is the same as the image size of the specified template image, and the specified template image is an image under normal use of the network device at the specified inspection task point; inputting the interface use image and the specified template image into a pre-trained interface use detection model, and outputting a detection annotation file with the same size as the interface use image and the specified template image, wherein the detection annotation file includes the class identification of each pixel point; acquiring the class identification of each pixel point in the detection annotation file, determining the interface use state of the network device at the specified inspection task point according to the class identification of each pixel point, and monitoring the use of the data center network device based on the interface use state, wherein the interface use state includes an abnormal use state and a normal use state.

[0007] Further, before inputting the interface use image and the specified template image into the pre-trained interface use detection model, the method further comprises: using a neural network structure that fuses multi-scale features to construct an initial model, wherein the initial model includes a feature extraction network using convolution and pooling operations and an up-sampling network using up-sampling and feature map fusion.

[0008] Further, before inputting the interface use image and the specified template image into the pre-trained interface use detection model, the method further comprises: generating a simulation data set, specifically comprising: acquiring an interface element image corresponding to the interface, wherein the interface element image includes a plurality of specified interface images, and each specified interface image includes an interface idle state image or an interface occupied state image; acquiring a plurality of template images, wherein the template images are images under normal use of a plurality of network devices; covering the interface element image in each template image according to a random covering rule to generate an initial interface abnormal use image corresponding to each template image; generating a specified annotation file with the same size as the initial interface abnormal use image according to the initial interface abnormal use image, wherein the specified annotation file includes a rule-violating interface identifier and a normal interface identifier; performing color space transformation and geometric space transformation on the abnormal use image corresponding to each template image to generate an interface abnormal use image corresponding to each template image; grouping the interface abnormal use image corresponding to each template image and the template image corresponding to the interface abnormal use image into an image pair; and generating a simulation data set according to the image pair and the specified annotation file.

[0009] Further, after the simulation data set is generated, the method further comprises: training the initial model using data in the simulation data set to obtain a required interface use detection model, specifically comprising: inputting the image pairs in the simulation data set into the initial model to output a presumed annotation file; and adjusting parameters of the initial model according to the presumed annotation file and the specified annotation file in the simulation data set to obtain a required interface use detection model.

[0010] Further, according to the category identifier of each pixel point, the interface use state of the network device at the specified inspection task point is determined, specifically comprising: judging whether the category identifier of each pixel point in the detection annotation file exists the first category identifier, if the first category identifier exists, it is determined that the interface use state at the specified inspection task point is abnormal use; wherein the category identifier includes a first category identifier and a second category identifier, the first category identifier is used to indicate that the interface state corresponding to the pixel point is an abnormal use state, and the second category identifier is used to indicate that the interface state corresponding to the pixel point is a normal use state; and according to the category identifier, the pixel points in the detection annotation file are divided into first category pixel points and second category pixel points, wherein the category identifier of each pixel point in the first category pixel points is the first category identifier, and the category identifier of each pixel point in the second category pixel points is the second category identifier; the region composed of the first category pixel points is regarded as an interface abnormal use region, and the region position coordinate data of the interface abnormal use region is determined; and according to the region position coordinate data, the interface abnormal use region is visually displayed in the interface use image and the specified template image.

[0011] Further, before the specified template image corresponding to the specified inspection task point is obtained, the method further comprises: setting a plurality of inspection task points in advance; collecting a template image at the inspection task point by an inspection robot, and recording a robot pose state of the inspection robot; establishing a corresponding relationship between the template image and the robot pose state as a first corresponding relationship; setting a place identifier for each inspection task point, and establishing a corresponding relationship between the inspection task point and the template image as a second corresponding relationship according to the place identifier of each inspection task point and the template image collected at each inspection task point.

[0012] Further, the interface use image collected by the inspection robot at the designated inspection task point is acquired, specifically including: determining a designated location identifier corresponding to the designated inspection task point; determining a designated template image corresponding to the designated inspection task point in the second correspondence relationship according to the designated location identifier; determining a designated robot pose state corresponding to the designated template image in the first correspondence relationship based on the designated template image corresponding to the designated inspection task point; and collecting the interface use image at the designated inspection task point using the designated robot pose state.

[0013] Further, before the interface use image and the designated template image are input into the pre-trained interface use detection model, the method further includes: aligning and transforming the interface use image and the designated template image according to a preset image alignment manner to obtain an aligned and transformed interface use image, so as to input the aligned and transformed interface use image and the designated template image into the pre-trained interface use detection model.

[0014] One or more embodiments of the present specification provide a usage monitoring device of a data center network device, including:

[0015] at least one processor; and

[0016] a memory in communication connection with the at least one processor; wherein

[0017] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:

[0018] acquire an interface use image collected by an inspection robot at a designated inspection task point, and acquire a designated template image corresponding to the designated inspection task point set in advance, wherein the image size of the interface use image is the same as the image size of the designated template image, and the designated template image is an image under normal use condition of the network device at the designated inspection task point; input the interface use image and the designated template image into a pre-trained interface use detection model, and output a detection annotation file with the same size as the interface use image and the designated template image, wherein the detection annotation file includes a class identifier of each pixel point; acquire the class identifier of each pixel point in the detection annotation file, and determine an interface use state of the network device at the designated inspection task point according to the class identifier of each pixel point, to monitor the usage of the data center network device based on the interface use state, wherein the interface use state includes an abnormal use state and a normal use state.

[0019] The one or more embodiments of the specification provide a non-volatile computer storage medium storing computer executable instructions configured to:

[0020] An interface use image collected by the inspection robot at a specified inspection task point is acquired, and a specified template image corresponding to the specified inspection task point is acquired, wherein the image size of the interface use image is the same as the image size of the specified template image, and the specified template image is an image under normal use of network equipment at the specified inspection task point; the interface use image and the specified template image are input into a pre-trained interface use detection model, and a detection annotation file with the same size as the interface use image and the specified template image is output, wherein the detection annotation file includes a class identifier of each pixel point; the class identifier of each pixel point in the detection annotation file is acquired, and the interface use state of the network equipment at the specified inspection task point is determined according to the class identifier of each pixel point, so as to monitor the use of the data center network equipment based on the interface use state, wherein the interface use state includes an abnormal use state and a normal use state.

[0021] The above at least one technical solution adopted by the embodiments of the specification can achieve the following beneficial effects: through the above technical solution, the detection annotation file of the class identifier of each pixel point is generated according to the actually collected interface use image and the template image at the position, compared with only using the actual image, the result obtained is more accurate, and is more in line with the characteristics of the equipment at the current inspection task point. According to the detection annotation file, the interface use state at the task point is determined, the interface use difference is presented in the form of a class identifier, the small visual changes and unclear trace features caused by the illegal use are quantitatively displayed, the detection accuracy of the interface illegal use is improved, and the interface illegal use problem of the data center network equipment can be found in time. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the specification or the prior art, the drawings needed to be used in the embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the specification, and those skilled in the art can also obtain other drawings according to these drawings without creative labor. In the drawings:

[0023] Figure 1 A flowchart of a data center network equipment use monitoring method provided by the embodiments of the specification is shown in the figure;

[0024] Figure 2A structural diagram of a data center network equipment use monitoring device provided by an embodiment of the present specification. DETAILED DESCRIPTION

[0025] In order for those skilled in the art to better understand the technical solutions in the present specification, the technical solutions in the present specification will be described clearly and completely below in combination with the drawings in the embodiments of the present specification. Obviously, the described embodiments are only a part of the embodiments of the present specification, not all the embodiments. Based on the embodiments of the present specification, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present specification.

[0026] With the development of Internet technology, the scale of data centers is getting larger and larger. The network data transmission volume of large data centers is large, and the network lines laid are numerous, which increases the difficulty of network communication equipment inspection. If there is abnormal use behavior of network equipment, it will cause unnecessary loss and serious consequences if not discovered in time, such as illegal private connection of network lines, unauthorized use of data center network resources, malicious removal of network lines to physically disconnect servers, and even theft of communication data. Under normal circumstances, full-time personnel are arranged for manual inspection, which on the one hand cannot guarantee uninterrupted inspection, and on the other hand, for the numerous network lines of the data center or the dense network lines on a certain switch or router, even if a private optical fiber appears, the human eye is easy to miss, causing missed inspection. For similar tasks, the advantages of inspection robots are obvious.

[0027] Robot inspection mainly shoots equipment images through high-definition cameras to monitor the equipment. Due to abnormal use of equipment, especially visual changes caused by abnormal use of interfaces, the current detection accuracy and fullness rate are not high, and the problem of abnormal use of interfaces cannot be monitored in time and accurately.

[0028] The present specification provides a data center network equipment use monitoring method, which is applied to an inspection robot. It should be noted that the execution subject in the present specification can be a server or any device with data processing capability. Figure 1 A flowchart of a data center network equipment use monitoring method provided by an embodiment of the present specification is shown in FIG. 1, which mainly includes the following steps: Figure 1

[0029] Step S101, acquiring an interface use image collected by the inspection robot at a designated inspection task point, and acquiring a designated template image corresponding to the designated inspection task point pre-set.

[0030] Among them, the image size of the interface use image is the same as the image size of the designated template image. ​

[0031] Before acquiring the specified template image corresponding to the specified inspection task point, the method further comprises: setting a plurality of inspection task points in advance; collecting a template image at the inspection task point by the inspection robot, and recording a robot pose state of the inspection robot; establishing a corresponding relationship between the template image and the robot pose state as a first corresponding relationship; setting a location identifier for each inspection task point, and establishing a corresponding relationship between the inspection task point and the template image according to the location identifier of each inspection task point and the template image collected at each inspection task point as a second corresponding relationship.

[0032] In an embodiment of the present specification, the inspection robot is initialized and deployed, a plurality of inspection task points are set in the data center in advance, the inspection task points are recorded in the inspection system, the inspection robot is supported by the inspection system to perform the inspection task, and the inspection robot is controlled to arrive at the inspection task point to perform the interface use monitoring task. In addition, since the number of devices and the composition of device lines of the network equipment in each inspection task point are different, in order to ensure the integrity of the collected interface use image, the robot pose state of the inspection robot, that is, the shooting angle of the inspection robot, needs to be set at each inspection task point. At each inspection task point, a template image corresponding to the inspection task point is collected by the inspection robot, and the template image refers to an image under normal use of the network equipment. The robot pose state for collecting the template image is recorded, and a corresponding relationship between the template image and the robot pose state is established. In addition, in order to facilitate the control of the inspection robot, a location identifier also needs to be set for each inspection task point, and a corresponding relationship between the location identifier and the template image corresponding to each inspection task point is established.

[0033] In an embodiment of the present specification, since the number of devices and the composition of device lines of the network equipment in each inspection task point are different, that is, the inspection complexity of each inspection task point is different, the number of template images required by the inspection task point is determined according to the inspection complexity of each inspection task point. For example, the number of template images required by each inspection task point can be determined according to the number of network equipment at each inspection task point, wherein the number of template images includes one or more; when the number of template images required by the inspection task point is one, one template image is collected at the inspection task point by the inspection robot based on a specified robot pose state; when the number of template images required by the inspection task point is multiple, multiple template images are collected at the inspection task point by the inspection robot based on the specified robot pose state.

[0034] The interface use image collected by the inspection robot at the specified inspection task point includes: determining a specified location identifier corresponding to the specified inspection task point; determining a specified template image corresponding to the specified inspection task point in the second correspondence relationship according to the specified location identifier; determining a specified robot pose state corresponding to the specified template image in the first correspondence relationship based on the specified template image corresponding to the specified inspection task point; and collecting the interface use image at the specified inspection task point using the specified robot pose state.

[0035] In an embodiment of the present specification, when it is necessary to monitor the device usage of the network device of the data center, the inspection robot is controlled to reach a specified inspection task point, and a specified location identifier corresponding to the specified inspection task point is determined. A specified template image corresponding to the specified inspection task point is determined in the correspondence relationship between the location identifier and the template image corresponding to each inspection task point according to the specified location identifier. Then, a specified robot pose state corresponding to the specified template image is determined in the correspondence relationship between the template image and the robot pose state according to the specified template image corresponding to the specified inspection task point. The inspection robot is controlled to collect an interface use image at the specified inspection task point using the specified robot pose state.

[0036] In step S102, the interface use image and the specified template image are input into the pre-trained interface use detection model, and a detection annotation file with the same size as the interface use image and the specified template image is output.

[0037] The detection annotation file can be a mask image, and the mask image includes a class identifier of each pixel point.

[0038] Before the interface use image and the specified template image are input into the pre-trained interface use detection model, the method further includes: aligning and transforming the interface use image and the specified template image according to a preset image alignment manner to obtain an aligned and transformed interface use image, so as to input the aligned and transformed interface use image and the specified template image into the pre-trained interface use detection model.

[0039] In an embodiment of the present specification, an alignment tool or an alignment method for image alignment of the auxiliary image is pre-set, and the alignment tool or the alignment method is deployed in an initialization stage of the inspection robot. After the inspection robot collects the interface use image, the image alignment tool or the image alignment method is used to perform image alignment on the interface use image and the template image corresponding to the specified inspection task point. The aligned and transformed interface use image and the specified template image are input into the pre-trained interface use detection model, and a detection annotation file with the same size as the interface use image and the specified template image is output.

[0040] Before inputting the interface use image and the specified template image into the pre-trained interface use detection model, the method further comprises: using a neural network structure that fuses multi-scale features to construct an initial model, wherein the initial model comprises a feature extraction network that uses convolution and pooling operations and an up-sampling network that uses up-sampling and feature map fusion.

[0041] In one embodiment of the present specification, because the pixel occupied by the illegal interface part is very small, only one percent or even less, relative to the whole image, in this embodiment, a neural network structure is adopted when constructing the initial model, and the neural network structure fuses multi-scale features. The first half is a feature extraction network mainly composed of convolution and pooling operations, and the second half is an up-sampling network composed of up-sampling and feature map fusion. In the up-sampling network part, each time the up-sampling is performed, the same scale fusion is performed with the corresponding channel number of the feature extraction network part, and then the up-sampling is continued until the feature map scale is consistent with the template size. The output of the last layer of the model is called the illegal interface probability heat map. The illegal interface probability heat map is composed of two channels. The value of each pixel point in each channel is set to be between 0 and 1. Channel 1 represents the probability of no illegal interface, and channel 2 represents the probability of illegal interface. The values of the same pixel point in the two channels of the illegal interface probability heat map are compared, and the larger value is taken as the class label of the pixel point. In this way, the class label of each pixel point is obtained, and a final output label file is generated according to the class label of each pixel point.

[0042] Before inputting the interface use image and the specified template image into the pre-trained interface use detection model, the method further comprises: generating a simulation data set, specifically comprising: obtaining an interface element image corresponding to the interface, wherein the interface element image comprises a plurality of specified interface images, and each specified interface image comprises an interface idle state image or an interface occupied state image; obtaining a plurality of template images, wherein the template images are images under normal use of a plurality of network devices; covering the interface element image in each template image according to a random covering rule to generate an initial interface abnormal use image corresponding to each template image; generating a specified label file with the same size as the initial interface abnormal use image according to the initial interface abnormal use image, wherein the specified label file comprises illegal interface labels and normal interface labels; performing color space transformation and geometric space transformation on the abnormal use image corresponding to each template image to generate an interface abnormal use image corresponding to each template image; grouping the interface abnormal use image corresponding to each template image and the template image corresponding to the interface abnormal use image into an image pair; and generating a simulation data set according to the image pair and the specified label file.

[0043] In an embodiment of the present specification, after the model is constructed, the model needs to be trained, and a training data set is needed here. Due to the rarity of network device misuse, the sample is scarce, so an efficient method for generating a simulated data set is proposed to simulate the images collected in the real inspection scenario and automatically generate a simulated training data set. The method for generating a simulated data set includes generating an initial interface abnormal use image and generating visual changes of the image.

[0044] The generation of the initial interface misuse image needs an interface element image and a corresponding template image at the inspection task point. The interface element image includes multiple interface images, which include different types of interfaces and different states of each interface. The types of interfaces include network interfaces, fiber interfaces, etc. The states of each interface include interface occupation states and interface idle states. The template image is an image of the normal use of the network device collected by the control inspection robot. The interface element image is overlaid on each template image according to the random overlay rule to generate an initial interface abnormal use image corresponding to each template image. The random overlay principle here refers to random interface types, random interface positions, and random interface quantities overlaid in the template image.

[0045] In order to enhance the adaptability of the model in the actual task, the generated interface misuse image also needs to be subjected to image visual transformation. The image visual transformation here refers to the transformation of the image in the color space and the geometric space. The color space mainly transforms brightness, contrast, and hue to simulate the changes of light and camera shooting parameters in the real task. The geometric space mainly performs light affine transformation and perspective transformation to simulate the slight changes of the camera shooting angle caused by the robot posture control error in the real situation.

[0046] In addition, while generating the initial interface abnormal use image, a specified annotation file with the same size as the initial interface abnormal use image is generated, which is the expected output of the model. The specified annotation file includes misuse interface identification and normal interface identification. The misuse interface identification can be set to 1, and the normal interface identification can be set to 0. The normal interface identification can also be called non-misuse interface identification. The misuse interface identification can also be called abnormal interface identification. Each template image corresponding to the interface abnormal use image and the interface abnormal use image corresponding to the template image form an image pair, which is called an image pair. According to the image pair and the specified annotation file, a simulated data set is generated.

[0047] That is, one template image corresponds to one or more generated interface violation usage images. The training data input into the model in this embodiment is composed or generated by a template image and its corresponding generated interface violation usage image, that is, one training data refers to a pair of images and a label file, one of the pair of images is a template image, and the other is an interface violation usage image generated from the template.

[0048] After generating the simulation data set, the method further includes training the initial model using the data in the simulation data set to obtain a required interface usage detection model, specifically including: inputting the image pair in the simulation data set into the initial model to output a predicted label file; and adjusting parameters of the initial model according to the predicted label file and the specified label file in the simulation data set to obtain a required interface usage detection model.

[0049] In an embodiment of the present specification, the initial model is trained using the data in the simulation data set to obtain a required interface usage detection model. The image pair in the simulation data set is input into the initial model to output a predicted label file. The initial model is adjusted according to the predicted label file and the specified label file in the simulation data set to obtain a required interface usage detection model. In addition, the learning goal of the model is defined as the difference between the predicted label file and the specified label file, such as the sum of squares of differences, cross-entropy, etc., which can be used for the learning goal.

[0050] In step S103, the class identifier of each pixel point in the detection label file is obtained, and the interface usage state of the network device at the specified inspection task point is determined according to the class identifier of each pixel point, so as to monitor the usage of the data center network device based on the interface usage state.

[0051] The interface usage state includes an abnormal usage state and a normal usage state.

[0052] According to the category identifier of each pixel point, the interface use state of the network device at the specified inspection task point is determined, specifically including: judging whether the category identifier of each pixel point in the detection annotation file exists a first category identifier, if the first category identifier exists, it is determined that the interface use state at the specified inspection task point is abnormal use; wherein the category identifier includes a first category identifier and a second category identifier, the first category identifier is used to indicate that the interface state corresponding to the pixel point is an abnormal use state, and the second category identifier is used to indicate that the interface state corresponding to the pixel point is a normal use state; and according to the category identifier, the pixel points in the detection annotation file are divided into first type pixel points and second type pixel points, wherein the category identifier of each pixel point in the first type pixel points is the first category identifier, and the category identifier of each pixel point in the second type pixel points is the first category identifier; the region composed of the first type pixel points is regarded as an interface abnormal use region, and the region position coordinate data of the interface abnormal use region is determined; according to the region position coordinate data, the interface abnormal use region is visually displayed on the interface use image and the specified template image.

[0053] In an embodiment of the present specification, according to the output detection annotation file, the class label corresponding to each pixel point in the detection annotation file is obtained. It should be noted that the detection annotation file can be a mask image, and the size of the mask image is the same as the image size of the input interface usage image and the specified template image. It is determined whether the class label of each pixel point in the detection annotation file exists the first class label, if the first class label exists, it is determined that the interface usage state at the specified inspection task point is abnormal usage, it should be noted that the class label includes the first class label and the second class label, the first class label is used to indicate that the interface state corresponding to the pixel point is an abnormal usage state, that is, the first class label can be called a violation interface label, and the violation interface label is set to 1 in the constructed simulation data set, so the first class label here is 1. The second class label is used to indicate that the interface state corresponding to the pixel point is a normal usage state, similarly, the normal interface label is set to 0 in the constructed simulation data set, so the second class label here is 0. If the class label of the pixel point in the detection annotation file is 1, it is determined that the interface usage state of the network device at the inspection task point exists an abnormal usage state. The first class pixel point in the detection annotation file, that is, the large connected region with the class label of 1, is screened, the region composed of the first class pixel point is regarded as an interface abnormal usage region, and the region position coordinate data of the interface abnormal usage region is determined. Since the size of the detection annotation file is the same as the size of the interface usage image and the specified template image, according to the region position coordinate data of the interface abnormal usage region in the detection annotation file, the interface abnormal usage region can be corresponded to the interface usage image and the specified template image for visual display, and the use of the data center network device can be further audited, confirmed and monitored according to the display content.

[0054] According to the actual collected interface usage image and the template image at the position, the detection annotation file of the class label of each pixel point is generated, compared with only using the actual image, the result obtained is more accurate, and is more in line with the device characteristics of the current inspection task point. According to the detection annotation file, the interface usage state at the task point is determined, the interface usage difference is presented in the form of a class label, the small visual changes and ambiguous trace features caused by violation usage are quantitatively displayed, the detection accuracy of interface violation usage is improved, and the interface violation usage problem of the data center network device can be found in time.

[0055] The embodiment of the present specification also provides a data center network device usage monitoring device, such as Figure 2As shown, the device comprises: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:

[0056] An interface use image collected by the inspection robot at a designated inspection task point is acquired, and a designated template image corresponding to the designated inspection task point is acquired, wherein the image size of the interface use image is the same as the image size of the designated template image, and the designated template image is an image under normal use of network equipment at the designated inspection task point; the interface use image and the designated template image are input into a pre-trained interface use detection model, and a detection annotation file with the same size as the interface use image and the designated template image is output, wherein the detection annotation file includes a class identifier of each pixel point; the class identifier of each pixel point in the detection annotation file is acquired, the interface use state of the network equipment at the designated inspection task point is determined according to the class identifier of each pixel point, and the use of the network equipment of the data center is monitored based on the interface use state, wherein the interface use state includes an abnormal use state and a normal use state.

[0057] The embodiments of the present specification also provide a non-volatile computer storage medium, which stores computer executable instructions, and the computer executable instructions are configured to:

[0058] An interface use image collected by the inspection robot at a designated inspection task point is acquired, and a designated template image corresponding to the designated inspection task point is acquired, wherein the image size of the interface use image is the same as the image size of the designated template image, and the designated template image is an image under normal use of network equipment at the designated inspection task point; the interface use image and the designated template image are input into a pre-trained interface use detection model, and a detection annotation file with the same size as the interface use image and the designated template image is output, wherein the detection annotation file includes a class identifier of each pixel point; the class identifier of each pixel point in the detection annotation file is acquired, the interface use state of the network equipment at the designated inspection task point is determined according to the class identifier of each pixel point, and the use of the network equipment of the data center is monitored based on the interface use state, wherein the interface use state includes an abnormal use state and a normal use state.

[0059] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0060] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0061] The devices, media, and methods provided in the embodiments of this specification are one-to-one correspondences. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0062] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0063] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0064] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0065] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0066] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0067] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer readable media.

[0068] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0069] It is also to be noted that the terms "comprising", "including", and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0070] The foregoing merely illustrates one or more embodiments of the present specification and is not intended to limit the scope of the specification. One or more embodiments of the present specification can be modified and varied and one or more embodiments of the present specification can be made without departing from the spirit and principles of the present specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of one or more embodiments of the present specification should be included in the scope of the claims of the present specification.

Claims

1. A method for monitoring the use of data center network equipment, characterized in that, The method, applied to inspection robots, includes: The system acquires interface usage images collected by the inspection robot at a designated inspection task point, and acquires a pre-set designated template image corresponding to the designated inspection task point. The image size of the interface usage image is the same as the image size of the designated template image, and the designated template image is an image of the network device at the designated inspection task point under normal use. The interface usage image and the specified template image are input into a pre-trained interface usage detection model, and a detection annotation file of the same size as the interface usage image and the specified template image is output. The detection annotation file includes the category identifier of each pixel. Obtain the category identifier of each pixel in the detection annotation file, and determine the interface usage status of the network device at the specified inspection task point based on the category identifier of each pixel, so as to monitor the use of the data center network device based on the interface usage status, wherein the interface usage status includes abnormal usage status and normal usage status. Before inputting the interface usage image and the specified template image into the pre-trained interface usage detection model, the method further includes: Generate a simulated dataset, specifically including: Obtain the interface element image corresponding to the interface, wherein the interface element image includes multiple specified interface images, and each specified interface image includes an interface idle state image or an interface occupied state image; Obtain multiple template images, wherein the template images are images of multiple network devices under normal use; The interface element image is overlaid on each template image according to a random overlay rule to generate an initial interface exception usage image corresponding to each template image; Based on the initial abnormal interface usage image, a specified annotation file with the same size as the initial abnormal interface usage image is generated, wherein the specified annotation file includes a violation interface identifier and a normal interface identifier; Perform color space transformation and geometric space transformation on the abnormal usage image corresponding to each template image to generate the interface abnormal usage image corresponding to each template image; Each template image is paired with the interface error usage image corresponding to the interface error usage image and the template image corresponding to the interface error usage image. Generate a simulated dataset based on the image pairs and the specified annotation file; The initial model is trained using data from the simulated dataset to obtain a detection model that meets the requirements for interface use.

2. The method for monitoring the use of data center network equipment according to claim 1, characterized in that, Before inputting the interface usage image and the specified template image into the pre-trained interface usage detection model, the method further includes: An initial model is constructed using a neural network structure that integrates multi-scale features, wherein the initial model includes a feature extraction network using convolution and pooling operations and an upsampling network using upsampling and feature map fusion.

3. The method for monitoring the use of data center network equipment according to claim 1, characterized in that, After generating the simulated dataset, the method further includes: The image pairs in the simulated dataset are input into the initial model, and the inferred annotation file is output. Based on the inferred annotation file and the specified annotation file in the simulated dataset, the parameters of the initial model are adjusted to obtain an interface detection model that meets the requirements.

4. The method for monitoring the use of data center network equipment according to claim 1, characterized in that, Based on the category identifier of each pixel, the interface usage status of the network device at the designated inspection task point is determined, specifically including: Determine whether the category identifier of each pixel in the detection annotation file exists as a first category identifier. If the first category identifier exists, determine that the interface usage status at the specified inspection task point is abnormal. The category identifier includes a first category identifier and a second category identifier. The first category identifier is used to indicate that the interface corresponding to the pixel is in an abnormal usage state, and the second category identifier is used to indicate that the interface corresponding to the pixel is in a normal usage state. According to the category identifier, the pixels in the detection annotation file are divided into a first category of pixels and a second category of pixels, wherein the category identifier of each pixel in the first category is the first category identifier, and the category identifier of each pixel in the second category is the second category identifier. The region composed of the first type of pixels is taken as the abnormal use area of ​​the interface, and the regional location coordinate data of the abnormal use area of ​​the interface is determined. Based on the location coordinate data of the region, the abnormal usage area of ​​the interface is visualized in the interface usage image and the specified template image.

5. The method for monitoring the use of data center network equipment according to claim 1, characterized in that, Before obtaining the specified template image corresponding to the pre-set specified inspection task point, the method further includes: Multiple inspection task points are pre-set; At the inspection task point, the inspection robot collects template images and records the robot's pose state. Establish a correspondence between the template image and the robot's pose state, which serves as the first correspondence; A location identifier is set for each inspection task point. Based on the location identifier of each inspection task point and the template image collected at each inspection task point, a correspondence between the inspection task point and the template image is established as a second correspondence.

6. The method for monitoring the use of data center network equipment according to claim 5, characterized in that, Acquire interface usage images collected by the inspection robot at designated inspection task points, specifically including: Determine the designated location identifier corresponding to the designated inspection task point; Based on the designated location identifier, the designated template image corresponding to the designated inspection task point is determined in the second correspondence; Based on the specified template image corresponding to the specified inspection task point, the specified robot pose state corresponding to the specified template image is determined in the first correspondence. Using the specified robot pose state, images are acquired at the specified inspection task point using the interface.

7. The method for monitoring the use of data center network equipment according to claim 1, characterized in that, Before inputting the interface usage image and the specified template image into the pre-trained interface usage detection model, the method further includes: According to a preset image alignment method, the interface usage image and the specified template image are aligned and transformed to obtain an aligned and transformed interface usage image, so that the aligned and transformed interface usage image and the specified template image can be input into a pre-trained interface usage detection model.

8. A data center network equipment usage monitoring device, characterized in that, The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-7.

9. A non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as described in any one of claims 1-7.

Images