Machine room monitoring method and system of multi-sensor fusion intelligent inspection robot

Abstract

The invention relates to the technical field of power machine room monitoring, and discloses a machine room monitoring method and system for a multi-sensor fusion intelligent inspection robot, and the system comprises a machine room data collection module, a knowledge graph construction module, an inspection route planning module, a data recognition module, a quadratic planning module, a risk recognition module, an early warning module and an intelligent control module. Object identification can be performed on the obstacle, matching with the constructed machine room knowledge graph is facilitated, if matching succeeds, the obstacle is an original object in the machine room, the risk possibility to the machine room is zero, only the original placing position is replaced, and if matching fails, the obstacle is not the original object in the machine room, and the machine room is dangerous. Therefore, the risk needs to be further analyzed; according to the method, obstacle risk categories are obtained, article risk early warning notices of different levels are sent, if the obstacles have risks, workers can be notified in time to check and remove the obstacles, and it is guaranteed that articles in the machine room meet the safety requirement.

Description

Technical Field

[0001] The present invention relates to the technical field of power room monitoring, and more specifically to a method and system for monitoring a room using a multi-sensor fusion intelligent inspection robot. Background Art

[0002] With the development of information technology, data center computer rooms have become the core place for the operation of key businesses in the power industry. The normal operation of equipment in the computer rooms of the power industry is crucial to ensuring the normal operation of electricity. Traditional computer room inspections mainly rely on manual work, and staff are required to regularly conduct physical inspections of various equipment in the power room to ensure the normal operation of the equipment in the room and promptly discover and handle possible abnormal situations. However, traditional manual inspection methods are time-consuming and labor-intensive. In order to improve efficiency, some power rooms use intelligent inspection robots for inspections, which improves inspection efficiency, monitors the computer room environment and equipment status in real time, improves the accuracy of computer room monitoring, and reduces operation and maintenance costs.

[0003] In order to ensure the normal operation of the equipment in the power room, the stacking of debris is usually not allowed in the power room, and the placement of items is prohibited from being changed at will. Therefore, when the environment in the power room changes, such as the placement of items in the room changes, it will affect the inspection route of the inspection robot. At this time, the inspection route of the inspection robot needs to be reset, but the existing inspection robots cannot automatically change to the correct inspection route according to the environment of items in the room, and the existing inspection robots cannot identify temporary obstacles and cannot determine the risk of obstacles. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, the present invention provides a machine room monitoring method and system of a multi-sensor fusion intelligent inspection robot to solve the problems existing in the above-mentioned background technology.

[0005] The present invention provides the following technical solution: a machine room monitoring method of a multi-sensor fusion intelligent inspection robot, comprising the following steps:

[0006] Step S01: Collect data: collect environmental data and object data in the computer room;

[0007] Step S02: constructing a knowledge graph: constructing a knowledge graph of computer room objects based on the object data in the computer room collected in step S01;

[0008] Step S03: Collecting the three-dimensional point cloud data of the machine room, the inspection robot automatically plans the inspection route based on the three-dimensional point cloud data of the machine room and marks it as a long-term inspection route;

[0009] Step S04: perform inspection according to the inspection route automatically planned in step S03, collect images on the inspection route in real time, and identify obstacles. If obstacles are identified, execute step S05. If no obstacles are identified, perform inspection work normally according to the inspection route and execute step S09 at the same time.

[0010] Step S05: Obtain the image and location information of the obstacle, plan a new inspection route, and execute step S06;

[0011] Step S06: perform object recognition on the obstacle based on the constructed neural network model, and match it with the computer room object knowledge graph constructed in step S02. If the match is successful, execute step S07; if the match fails, execute step S08;

[0012] Step S07: Record the location information of the successfully matched computer room object, update the point cloud data of the computer room object, and mark the new inspection route in step S05 as a long-term inspection route for storage;

[0013] Step S08: Obtain the name of the obstacle, establish a risk category knowledge graph, obtain the obstacle risk category and send different levels of object risk warning notifications, and at the same time mark the new inspection route in step S05 as a temporary inspection route, and use this temporary inspection route as the current inspection route to continue the inspection work. When the next inspection work starts, the inspection work will still be performed according to the latest stored long-term inspection route;

[0014] Step S09: Perform risk identification on the collected environmental data. If the identification result shows that there is a risk, an environmental warning notification is sent, otherwise no warning notification is sent;

[0015] Step S10: Perform inspection work according to the latest stored inspection route.

[0016] Preferably, the environmental data includes the temperature, humidity and dust particles in the computer room, and the object data includes images and positions of objects placed in the computer room.

[0017] Preferably, the specific method of constructing the computer room object knowledge graph in step S02 is:

[0018] A computer room object knowledge graph is constructed based on the object images and corresponding object names in the collected object data. The computer room object knowledge graph includes entities, entity attribute information, relationships between entities, and entity types; entity attribute information is a label that describes entity attributes, and relationships between entities represent the semantic connection between different entities, describing various associations and interactions between entities in the knowledge graph. Entity type defines the category to which the entity belongs, and defines common attributes and relationships between entities.

[0019] Preferably, the specific method of automatically planning the inspection route is:

[0020] Step 1: Based on the 3D point cloud data of the computer room, a 3D model of the computer room is constructed, and after inputting into the inspection robot, multiple initial inspection routes are generated, and then encoded, chromosomes are obtained, and an initial population is constructed;

[0021] Step 2: Determine the fitness function;

[0022] Step 3: natural selection of chromosomes in the population;

[0023] Step 4: Perform crossover recombination on chromosomes in the population;

[0024] Step 5: mutate the chromosomes in the population;

[0025] Step 6: Get a new population, preset the population generation to be L, the fitness threshold to be Q, L is an integer greater than 0, and Q is a real number greater than 0; loop steps 3 to 5 until the generation corresponding to the new population is L or the fitness corresponding to the chromosome in the new population is greater than or equal to the fitness threshold Q, the loop ends, and the initial inspection route corresponding to the chromosome with the largest fitness in the new population is taken as the optimal inspection route and marked as a long-term inspection route.

[0026] Preferably, the neural network model in step S06 is a convolutional neural network model, including an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer;

[0027] The input layer is used to input obstacle images, and the images are input into the convolution layer after preprocessing and feature extraction;

[0028] The convolution layer uses a convolution operation to perform sliding calculations on the input data through a convolution kernel, and uses the superposition of multiple convolution kernels to learn abstract features at different levels;

[0029] The pooling layer downsamples the feature map of the convolutional layer to reduce the size and complexity of the feature map while retaining the main features;

[0030] The fully connected layer receives the feature map output by the pooling layer, and integrates the feature information for classification and prediction through a fully connected operation;

[0031] The output layer is used to output the recognition result of the obstacle image, that is, the object name of the obstacle image.

[0032] Preferably, the specific method of matching the computer room object knowledge graph constructed in step S02 in step S06 is:

[0033] Match the object name of the obstacle image output by the output layer in step S06 with the entity in the computer room object knowledge graph. The matching methods include string matching and semantic matching. String matching includes exact matching and approximate matching. Exact matching is to compare the string corresponding to the object name of the obstacle image with the string corresponding to the entity in the computer room object knowledge graph. If they are exactly the same, then match them. Approximate matching is to use a string similarity algorithm to calculate the similarity between the string corresponding to the object name of the obstacle image and the string corresponding to the entity in the computer room object knowledge graph, and match the entity in the computer room object knowledge graph with the highest similarity with the object name of the obstacle image. Semantic matching is to use word embedding. The word embedding method converts the object name of the obstacle image and the entity in the computer room object knowledge graph into vector form through a word embedding model, and matches them by calculating the similarity between the vectors.

[0034] After the match is successful, the successfully matched entity in the computer room object knowledge graph is marked as a matching entity; either of the two methods can be selected to obtain the matching entity, and the entity attribute information corresponding to the matching entity is obtained from the computer room object knowledge graph. The entity attribute information can be the location information corresponding to the matching entity.

[0035] Preferably, the specific method of obtaining the obstacle risk category is:

[0036] Step A1: Divide the risk categories into three categories, namely low risk, medium risk and high risk, summarize the objects that pose a threat to the computer room, and classify them into the three risk categories respectively, and use web crawler technology to collect object names, images and corresponding risk category information;

[0037] Step A2: Construct a risk category knowledge graph based on the collected information. The risk category knowledge graph includes entities, entity attribute information, relationships between entities, and entity types. Entity attribute information is a label that describes entity attributes. The relationship between entities represents the semantic connection between different entities and describes various associations and interactions between entities in the knowledge graph. The entity type defines the category to which the entity belongs and defines the common attributes and relationships between entities.

[0038] Step A3: Match the object name of the obstacle image output by the output layer in step S06 with the entity in the risk category knowledge graph. The matching methods include string matching and semantic matching. String matching includes exact matching and approximate matching. Exact matching is to compare the string corresponding to the object name of the obstacle image with the string corresponding to the entity in the risk category knowledge graph. If they are exactly the same, then matching is performed. Approximate matching is to use a string similarity algorithm to calculate the similarity between the string corresponding to the object name of the obstacle image and the string corresponding to the entity in the risk category knowledge graph, and match the entity in the risk category knowledge graph with the highest similarity with the object name of the obstacle image. Semantic matching is to use a word embedding method. The word embedding method converts the object name of the obstacle image and the entity in the risk category knowledge graph into a vector form through a word embedding model, and matches by calculating the similarity between the vectors.

[0039] Step A4: After the match is successful, the successfully matched entity in the risk category knowledge graph is marked as a matching entity; and the entity attribute information corresponding to the matching entity is obtained from the risk category knowledge graph, and the entity attribute information is the risk category corresponding to the matching entity.

[0040] A computer room monitoring system of a multi-sensor fusion intelligent inspection robot, comprising a computer room data acquisition module, a knowledge graph construction module, an inspection route planning module, a data recognition module, a secondary planning module, a risk recognition module, an early warning module and an intelligent control module;

[0041] The computer room data acquisition module is used to collect computer room environment data, the computer room environment data includes environment data and object data, and the computer room data acquisition module includes an environment data acquisition unit and an object data acquisition unit; the environment data acquisition unit is used to collect environment data in the computer room and transmit it to the risk identification module; the object data acquisition unit is used to collect object data in the computer room and transmit it to the data identification module;

[0042] The knowledge graph construction module constructs a computer room object knowledge graph based on the object data in the computer room;

[0043] The inspection route planning module is used to collect the three-dimensional point cloud data of the computer room, automatically plan the inspection route, and transmit it to the intelligent control module, and mark the inspection route as a long-term inspection route, that is, the computer room inspection work is carried out according to this inspection route every time the inspection is carried out;

[0044] The data recognition module is used to collect images in the inspection route in real time and perform obstacle recognition. If an obstacle is recognized, the obstacle is recognized and matched with the computer room object knowledge graph constructed in the knowledge graph construction module. After the match is successful, an update route instruction is sent to the secondary planning module, and the location information of the successfully matched object is stored at the same time; if the match fails, a temporary route planning instruction is sent to the secondary planning module, and the object recognition result is transmitted to the risk identification module at the same time; if the obstacle is not recognized, no data information is transmitted, and the inspection robot continues to perform inspection work according to the route of the inspection route planning module;

[0045] The secondary planning module is used to perform secondary planning on the inspection route. The secondary planning module includes a temporary route planning unit and an update route planning unit. The temporary route planning unit is used to receive a temporary route planning instruction, plan a new inspection route and mark it as a temporary inspection route, and then transmit it to the intelligent control module; the update route planning unit is used to receive an update route instruction, plan a new inspection route and mark it as a long-term inspection route, and then transmit it to the intelligent control module;

[0046] The risk identification module includes an environmental risk identification unit and an object risk identification unit; the environmental risk identification unit is used to receive data from the environmental data collection unit, perform risk identification on the collected environmental data, and if the identification result is that there is a risk, send an environmental warning notification to the warning module; the object risk identification unit is used to receive the object identification result of the data identification module, establish a risk category knowledge graph, obtain the obstacle risk category and send different levels of object risk warning notifications to the warning module;

[0047] The early warning module is used to receive the early warning notification from the risk identification module and issue an early warning;

[0048] The intelligent control module is used to receive an inspection route and control the inspection robot to perform inspections along the inspection route. If the received route information is a long-term inspection route, each inspection will be performed along the long-term inspection route. If the received route information is a temporary inspection route, the temporary inspection route will replace the long-term inspection route and will only be used as the inspection route for the current inspection. When the next inspection begins, the inspection will continue along the long-term inspection route. The long-term inspection route is the most recently marked long-term inspection route.

[0049] Technical effects and advantages of the present invention:

[0050] The present invention is provided with step S06 and step S08, which is conducive to object identification of obstacles and matching with the constructed computer room knowledge graph. If the match is successful, it means that the obstacle is an original item in the computer room, and the risk possibility to the computer room is zero, and only the original placement position has been changed. If the match fails, it means that the obstacle is not an original item in the computer room and may cause danger to the computer room, so it is necessary to further analyze its risk; step S08 establishes a risk category knowledge graph, obtains the obstacle risk category and sends object risk warning notifications of different levels. If there is a risk in the obstacle, the staff can be notified in time to check and remove the obstacle to ensure that the items in the computer room meet the safety requirements. Not only can the inspection route be reset to meet the new environmental requirements, but the obstacle risk can also be determined to fully ensure the environmental safety in the power room. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of the computer room monitoring method of the multi-sensor fusion intelligent inspection robot of the present invention.

[0052] Figure 2 This is a structural diagram of the computer room monitoring system of the multi-sensor fusion intelligent inspection robot of the present invention. DETAILED DESCRIPTION

[0053] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. In addition, the forms of the various structures recorded in the following embodiments are merely illustrative. The machine room monitoring method and system of a multi-sensor fusion intelligent inspection robot involved in the present invention are not limited to the various structures recorded in the following embodiments. All other implementations obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0054] like Figure 1 As shown, the present invention provides a machine room monitoring method of a multi-sensor fusion intelligent inspection robot, comprising the following steps:

[0055] Step S01: Collecting data: Collecting environmental data and object data in the computer room, wherein the environmental data includes but is not limited to the temperature, humidity and dust particles in the computer room, and the object data is the images and positions of objects placed in the computer room; the environmental data is collected by devices such as temperature sensors, humidity sensors, moisture sensors and smoke sensors; the object data is collected by image acquisition devices such as images and point cloud data of objects placed in the computer room, and the image acquisition device can be one or more devices such as 3D cameras and video cameras;

[0056] Step S02: constructing a knowledge graph: constructing a knowledge graph of computer room objects based on the object data in the computer room collected in step S01;

[0057] The computer room object knowledge graph is constructed based on the object images and corresponding object names in the collected object data. The computer room object knowledge graph includes entities, entity attribute information, relationships between entities, and entity types. Entity attribute information is a label describing entity attributes. The relationship between entities represents the semantic connection between different entities and describes various associations and interactions between entities in the knowledge graph. The entity type defines the category to which the entity belongs and defines the common attributes and relationships between entities.

[0058] Step S03: Collecting the three-dimensional point cloud data of the machine room, the inspection robot automatically plans the inspection route based on the three-dimensional point cloud data of the machine room and marks it as a long-term inspection route;

[0059] Step S04: perform inspection according to the inspection route automatically planned in step S03, collect images on the inspection route in real time, and identify obstacles. If obstacles are identified, execute step S05. If no obstacles are identified, perform inspection work normally according to the inspection route and execute step S09 at the same time.

[0060] Step S05: Obtain the image and location information of the obstacle, plan a new inspection route, and execute step S06;

[0061] Step S06: perform object recognition on the obstacle based on the constructed neural network model, and match it with the computer room object knowledge graph constructed in step S02. If the match is successful, execute step S07; if the match fails, execute step S08; if the match is successful, it means that the obstacle is an original item in the computer room, and the risk possibility to the computer room is zero, and only the original placement position has been changed; if the match fails, it means that the obstacle is not an original item in the computer room, and may cause danger to the computer room, so its risk needs to be further analyzed;

[0062] Step S07: Record the location information of the successfully matched computer room objects, update the point cloud data of the computer room objects, and mark the new inspection route in step S05 as a long-term inspection route for storage. Each subsequent inspection work is carried out according to the latest stored long-term inspection route.

[0063] Step S08: Obtain the name of the obstacle, establish a risk category knowledge graph, obtain the obstacle risk category and send different levels of object risk warning notifications, and at the same time mark the new inspection route in step S05 as a temporary inspection route, and use this temporary inspection route as the current inspection route to continue the inspection work. When the next inspection work starts, the inspection work will still be performed according to the latest stored long-term inspection route;

[0064] Step S09: Perform risk identification on the collected environmental data. If the identification result shows that there is a risk, an environmental warning notification is sent, otherwise no warning notification is sent;

[0065] Step S10: Perform inspection work according to the latest stored inspection route.

[0066] In this embodiment, it should be specifically explained that the specific method of automatically planning the inspection route is:

[0067] Step 1: Based on the 3D point cloud data of the computer room, a 3D model of the computer room is constructed, and after inputting into the inspection robot, multiple initial inspection routes are generated, and then encoded, chromosomes are obtained, and an initial population is constructed;

[0068] Step 2: Determine the fitness function;

[0069] Step 3: natural selection of chromosomes in the population;

[0070] Step 4: Crossover recombination of chromosomes in the population;

[0071] Step 5: mutate the chromosomes in the population;

[0072] Step 6: Get a new population, preset the population generation number to be L, the fitness threshold to be Q, L is an integer greater than 0, and Q is a real number greater than 0; loop steps 3 to 5 until the generation number corresponding to the new population is L or the fitness corresponding to the chromosome in the new population is greater than or equal to the fitness threshold Q, the loop ends, and the initial inspection route corresponding to the chromosome with the maximum fitness in the new population is taken as the optimal inspection route, and marked as a long-term inspection route;

[0073] In step 1, the initial inspection route is encoded as U, where U is a chromosome, and N chromosomes are randomly generated to construct an initial population: C = {U1, U2, U3, ..., U N};

[0074] In step 2, the fitness function is expressed as: R =-T R , where F R is the fitness corresponding to the Rth chromosome, T R is the inspection time corresponding to the inspection route of the Rth chromosome, R = 1, 2, 3, ..., N;

[0075] In the step 3, natural selection is performed by combining the elite method and the rotation method, wherein the elite method produces K1 offspring chromosomes, and for a population with a capacity of N, the fitness corresponding to the N chromosomes is arranged from large to small, and the K1 chromosomes at the front each produce a offspring chromosome, and the rotation method produces K2 offspring chromosomes, that is, N chromosomes produce K2 offspring chromosomes according to the corresponding rotation probability, wherein K1+K2=N;

[0076] The formula for the rotation probability is expressed as: Among them, P R is the rotation probability corresponding to the Rth chromosome;

[0077] In the step 4, G chromosomes are randomly selected from the population for crossover recombination to obtain G new chromosomes. The crossover recombination adopts the PMX method. After the chromosome crossover recombination, the fitness of the G new chromosomes is calculated, and the fitness of the G new chromosomes and the fitness of the G chromosomes are sorted from large to small, and a sorting table is generated. The G chromosomes in the sorting table are replaced with the G chromosomes in the population for crossover recombination in positive order;

[0078] In the step 5, the mutation probability is preset to T, and the N chromosomes in the population are mutated according to the mutation probability. The mutation method is to randomly select the positions of two genes in the chromosome and exchange the values ​​of the two genes.

[0079] In this embodiment, it should be specifically explained that the obstacle recognition is performed by preprocessing the image and extracting edge information in the image based on an edge detection algorithm, thereby identifying the outline of the obstacle;

[0080] The preprocessing operation is used to improve the accuracy and efficiency of obstacle detection, including but not limited to denoising and enhancing the image; removing noise in the image, such as random points, lines and other interfering information; the enhancement process can enhance the image's contrast and brightness and other features, making the obstacle more prominent;

[0081] The edge detection algorithm can be any one of a first-order derivative edge detection algorithm and a second-order derivative edge detection algorithm; the first-order derivative edge detection algorithm identifies the area with the most dramatic brightness change in the image based on the gradient, mainly performs edge recognition by calculating the first-order derivative of the image gradient, uses a convolution operation to convolve the gradient template with the image, and sets a suitable threshold to extract the image edge. The first-order derivative edge detection algorithm can adopt any one of the Robert algorithm, the Prewitt algorithm and the Soble algorithm, which has high computational efficiency and is simple and effective; the second-order derivative edge detection algorithm performs edge detection by calculating the zero-crossing point of the second-order derivative of the gradient. The second-order derivative edge detection algorithm can adopt any one of the Laplacian algorithm and the Canny algorithm, etc., which is more accurate in edge positioning.

[0082] In this embodiment, it should be specifically explained that the specific method for the inspection robot to plan a new inspection route after identifying the existence of an obstacle is: remove the long-term inspection route obtained in step six from the initial inspection route, execute steps one to six until a new optimal inspection route appears, and use it as the new inspection route.

[0083] In this embodiment, it should be specifically explained that the neural network model in step S06 is a convolutional neural network model, including an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer;

[0084] The input layer is used to input obstacle images, and the images are input into the convolution layer after preprocessing and feature extraction;

[0085] The convolution layer uses a convolution operation to perform sliding calculation on the input data through a convolution kernel, and uses the superposition of multiple convolution kernels to learn abstract features at different levels; an activation function can be added after the convolution operation, and the activation function can be either Sigmoid or Tanh;

[0086] The pooling layer downsamples the feature map of the convolution layer to reduce the size and complexity of the feature map while retaining the main features. One or more of the maximum pooling operation or the average pooling operation can be used to effectively reduce the computational burden and overfitting risk of the model. The size and stride of the pooling window can be selected according to the size of the feature map and the requirements of the model, and a regularization method is used to reduce overfitting.

[0087] The fully connected layer receives the feature map output by the pooling layer, and integrates the feature information for classification and prediction through a fully connected operation;

[0088] The output layer is used to output the recognition result of the obstacle image, that is, the object name of the obstacle image.

[0089] In this embodiment, it should be specifically explained that the specific method of matching the computer room object knowledge graph constructed in step S02 in step S06 is:

[0090] Match the object name of the obstacle image output by the output layer in step S06 with the entity in the computer room object knowledge graph. The matching methods include string matching and semantic matching. String matching includes exact matching and approximate matching. Exact matching is to compare the string corresponding to the object name of the obstacle image with the string corresponding to the entity in the computer room object knowledge graph. If they are exactly the same, then match them. Approximate matching is to use a string similarity algorithm to calculate the similarity between the string corresponding to the object name of the obstacle image and the string corresponding to the entity in the computer room object knowledge graph, and match the entity in the computer room object knowledge graph with the highest similarity with the object name of the obstacle image. Semantic matching is to use word embedding. The word embedding method converts the object name of the obstacle image and the entity in the computer room object knowledge graph into vector form through a word embedding model, and matches them by calculating the similarity between the vectors.

[0091] Step A4: After the match is successful, the successfully matched entity in the computer room object knowledge graph is marked as a matching entity; either of the two methods can be selected to obtain the matching entity, and the entity attribute information corresponding to the matching entity is obtained from the computer room object knowledge graph. The entity attribute information may be the location information corresponding to the matching entity.

[0092] In this embodiment, it should be specifically explained that the specific method of obtaining the obstacle risk category is:

[0093] Step A1: Divide the risk categories into three categories, namely low risk, medium risk and high risk, summarize the objects that pose a threat to the computer room, and classify them into three risk categories respectively, and use web crawler technology to collect information such as object names, images and corresponding risk categories;

[0094] Step A2: Construct a risk category knowledge graph based on the collected information. The risk category knowledge graph includes entities, entity attribute information, relationships between entities, and entity types. Entity attribute information is a label that describes entity attributes. The relationship between entities represents the semantic connection between different entities and describes various associations and interactions between entities in the knowledge graph. The entity type defines the category to which the entity belongs and defines the common attributes and relationships between entities.

[0095] Step A3: Match the object name of the obstacle image output by the output layer in step S06 with the entity in the risk category knowledge graph. The matching methods include string matching and semantic matching. String matching includes exact matching and approximate matching. Exact matching is to compare the string corresponding to the object name of the obstacle image with the string corresponding to the entity in the risk category knowledge graph. If they are exactly the same, then matching is performed. Approximate matching is to use a string similarity algorithm to calculate the similarity between the string corresponding to the object name of the obstacle image and the string corresponding to the entity in the risk category knowledge graph, and match the entity in the risk category knowledge graph with the highest similarity with the object name of the obstacle image. Semantic matching is to use a word embedding method. The word embedding method converts the object name of the obstacle image and the entity in the risk category knowledge graph into a vector form through a word embedding model, and matches by calculating the similarity between the vectors.

[0096] After the match is successful, the successfully matched entity in the risk category knowledge graph is marked as a matching entity; either of the two methods can be selected to obtain the matching entity, and the entity attribute information corresponding to the matching entity is obtained from the risk category knowledge graph. The entity attribute information is the risk category corresponding to the matching entity.

[0097] In this embodiment, it should be specifically explained that the specific method of risk identification of the collected environmental data is:

[0098] A risk threshold is set for each environmental data, or if the collected environmental data exceeds the risk threshold, the identification result is that there is a risk. At this time, an environmental warning notification is sent, and the staff is required to promptly adjust and survey the computer room environment to ensure the stability of the computer room environment and the normal operation of the computer room; illustratively, if the risk threshold corresponding to the temperature in the environmental data is B1, and the temperature in the collected environmental data is B2, when B2>B1, the identification result is that there is a risk. At this time, an environmental warning notification is sent, and the notification content displays the corresponding temperature warning.

[0099] In this embodiment, it should be specifically explained that the specific method of sending different levels of object risk warning notifications is:

[0100] Based on the obtained obstacle risk category, an early warning notification of the risk level corresponding to the risk category is sent. The low risk category corresponds to a low risk level, the medium risk category corresponds to a medium risk level, and the high risk category corresponds to a high risk level. If it is a low risk level, the staff can arrange to remove the obstacle in a timely manner. If it is a medium risk level or a high network level, the staff must immediately survey the computer room and remove the obstacles while ensuring safety to prevent objects from affecting the safety in the computer room, so as to ensure the normal operation of the equipment in the power computer room.

[0101] like Figure 2As shown, the present invention provides a computer room monitoring system of a multi-sensor fusion intelligent inspection robot, including a computer room data acquisition module, a knowledge graph construction module, an inspection route planning module, a data identification module, a secondary planning module, a risk identification module, an early warning module and an intelligent control module;

[0102] The computer room data acquisition module is used to collect computer room environment data, the computer room environment data includes environment data and object data, and the computer room data acquisition module includes an environment data acquisition unit and an object data acquisition unit; the environment data acquisition unit is used to collect environment data in the computer room and transmit it to the risk identification module; the object data acquisition unit is used to collect object data in the computer room and transmit it to the data identification module;

[0103] The knowledge graph construction module constructs a computer room object knowledge graph based on the object data in the computer room;

[0104] The inspection route planning module is used to collect the three-dimensional point cloud data of the computer room, automatically plan the inspection route, and transmit it to the intelligent control module, and mark the inspection route as a long-term inspection route, that is, the computer room inspection work is carried out according to this inspection route every time the inspection is carried out;

[0105] The data recognition module is used to collect images in the inspection route in real time and perform obstacle recognition. If an obstacle is recognized, the obstacle is recognized and matched with the computer room object knowledge graph constructed in the knowledge graph construction module. After the match is successful, an update route instruction is sent to the secondary planning module, and the location information of the successfully matched object is stored at the same time; if the match fails, a temporary route planning instruction is sent to the secondary planning module, and the object recognition result is transmitted to the risk identification module at the same time; if the obstacle is not recognized, no data information is transmitted, and the inspection robot continues to perform inspection work according to the route of the inspection route planning module;

[0106] The secondary planning module is used to perform secondary planning on the inspection route. The secondary planning module includes a temporary route planning unit and an update route planning unit. The temporary route planning unit is used to receive a temporary route planning instruction, plan a new inspection route and mark it as a temporary inspection route, and then transmit it to the intelligent control module; the update route planning unit is used to receive an update route instruction, plan a new inspection route and mark it as a long-term inspection route, and then transmit it to the intelligent control module;

[0107] The risk identification module includes an environmental risk identification unit and an object risk identification unit; the environmental risk identification unit is used to receive data from the environmental data collection unit, perform risk identification on the collected environmental data, and if the identification result is that there is a risk, send an environmental warning notification to the warning module; the object risk identification unit is used to receive the object identification result of the data identification module, establish a risk category knowledge graph, obtain the obstacle risk category and send different levels of object risk warning notifications to the warning module;

[0108] The early warning module is used to receive the early warning notification from the risk identification module and issue an early warning;

[0109] The intelligent control module is used to receive an inspection route and control the inspection robot to perform inspections along the inspection route. If the received route information is a long-term inspection route, each inspection will be performed along the long-term inspection route. If the received route information is a temporary inspection route, the temporary inspection route will replace the long-term inspection route and will only be used as the inspection route for the current inspection. When the next inspection begins, the inspection will continue along the long-term inspection route. The long-term inspection route is the most recently marked long-term inspection route.

[0110] In this embodiment, it should be specifically explained that the difference between this embodiment and the prior art mainly lies in that this embodiment has step S06 and step S08, which is conducive to object identification of obstacles and matching them with the constructed computer room knowledge graph. If the match is successful, it means that the obstacle is an original item in the computer room, and the risk possibility to the computer room is zero, and only the original placement position has been changed. If the match fails, it means that the obstacle is not an original item in the computer room and may cause danger to the computer room, so its risk needs to be further analyzed; step S08 establishes a risk category knowledge graph, obtains the obstacle risk category and sends object risk warning notifications of different levels. If there is a risk in the obstacle, the staff can be notified in time to check and remove the obstacle to ensure that the items in the computer room meet the safety requirements. Not only can the inspection route be reset to meet the new environmental requirements, but the obstacle risk can also be determined to fully ensure the environmental safety in the power room.

[0111] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0112] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A computer room monitoring method of a multi-sensor fusion intelligent inspection robot, characterized in that: The following steps are involved: Step S01: Collect data: collect environmental data and object data in the computer room; Step S02: constructing a knowledge graph: constructing a knowledge graph of computer room objects based on the object data in the computer room collected in step S01; Step S03: Collecting the three-dimensional point cloud data of the machine room, the inspection robot automatically plans the inspection route based on the three-dimensional point cloud data of the machine room and marks it as a long-term inspection route; Step S04: perform inspection according to the inspection route automatically planned in step S03, collect images on the inspection route in real time, and identify obstacles. If obstacles are identified, execute step S05. If no obstacles are identified, perform inspection work normally according to the inspection route and execute step S09 at the same time. Step S05: Obtain the image and location information of the obstacle, plan a new inspection route, and execute step S06; Step S06: perform object recognition on the obstacle based on the constructed neural network model, and match it with the computer room object knowledge graph constructed in step S02. If the match is successful, execute step S07; if the match fails, execute step S08; Step S07: Record the location information of the successfully matched computer room object, update the point cloud data of the computer room object, and mark the new inspection route in step S05 as a long-term inspection route for storage; Step S08: Obtain the name of the obstacle, establish a risk category knowledge graph, obtain the obstacle risk category and send different levels of object risk warning notifications, and at the same time mark the new inspection route in step S05 as a temporary inspection route, and use this temporary inspection route as the current inspection route to continue the inspection work. When the next inspection work starts, the inspection work will still be performed according to the latest stored long-term inspection route; Step S09: Perform risk identification on the collected environmental data. If the identification result shows that there is a risk, an environmental warning notification is sent, otherwise no warning notification is sent; Step S10: Perform inspection work according to the latest stored inspection route.

2. The method for monitoring a machine room of a multi-sensor fusion intelligent inspection robot according to claim 1, characterized in that: The environmental data includes the temperature, humidity and dust particles in the computer room, and the object data includes the images and positions of the objects placed in the computer room.

3. The method for monitoring a machine room of a multi-sensor fusion intelligent inspection robot according to claim 1, characterized in that: The specific method of constructing the computer room object knowledge graph in step S02 is: A computer room object knowledge graph is constructed based on the object images and corresponding object names in the collected object data. The computer room object knowledge graph includes entities, entity attribute information, relationships between entities, and entity types; entity attribute information is a label that describes entity attributes, and relationships between entities represent the semantic connection between different entities, describing various associations and interactions between entities in the knowledge graph. Entity type defines the category to which the entity belongs, and defines common attributes and relationships between entities.

4. The method for monitoring a machine room of a multi-sensor fusion intelligent inspection robot according to claim 1, characterized in that: The specific method of automatically planning the inspection route is as follows: Step 1: Based on the 3D point cloud data of the computer room, a 3D model of the computer room is constructed, and after inputting into the inspection robot, multiple initial inspection routes are generated, and then encoded, chromosomes are obtained, and an initial population is constructed; Step 2: Determine the fitness function; Step 3: natural selection of chromosomes in the population; Step 4: Crossover recombination of chromosomes in the population; Step 5: mutate the chromosomes in the population; Step 6: Get a new population, preset the population generation to be L, the fitness threshold to be Q, L is an integer greater than 0, and Q is a real number greater than 0; loop steps 3 to 5 until the generation corresponding to the new population is L or the fitness corresponding to the chromosome in the new population is greater than or equal to the fitness threshold Q, the loop ends, and the initial inspection route corresponding to the chromosome with the largest fitness in the new population is taken as the optimal inspection route and marked as a long-term inspection route.

5. The method for monitoring a machine room of a multi-sensor fusion intelligent inspection robot according to claim 1, characterized in that: The neural network model in step S06 is a convolutional neural network model, including an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer; The input layer is used to input obstacle images, and the images are input into the convolution layer after preprocessing and feature extraction; The convolution layer uses a convolution operation to perform sliding calculations on the input data through a convolution kernel, and uses the superposition of multiple convolution kernels to learn abstract features at different levels; The pooling layer downsamples the feature map of the convolutional layer to reduce the size and complexity of the feature map while retaining the main features; The fully connected layer receives the feature map output by the pooling layer, and integrates the feature information for classification and prediction through a fully connected operation; The output layer is used to output the recognition result of the obstacle image, that is, the object name of the obstacle image.

6. The method for monitoring a machine room of a multi-sensor fusion intelligent inspection robot according to claim 1, characterized in that: The specific method of matching the computer room object knowledge graph constructed in step S02 in step S06 is: Match the object name of the obstacle image output by the output layer in step S06 with the entity in the computer room object knowledge graph. The matching methods include string matching and semantic matching. String matching includes exact matching and approximate matching. Exact matching is to compare the string corresponding to the object name of the obstacle image with the string corresponding to the entity in the computer room object knowledge graph. If they are exactly the same, then match them. Approximate matching is to use a string similarity algorithm to calculate the similarity between the string corresponding to the object name of the obstacle image and the string corresponding to the entity in the computer room object knowledge graph, and match the object name of the obstacle image with the entity in the computer room object knowledge graph with the highest similarity; Semantic matching uses word embedding. The word embedding method converts the object name of the obstacle image and the entity in the knowledge graph of the computer room object into a vector form through the word embedding model, and matches them by calculating the similarity between the vectors. After the match is successful, the successfully matched entity in the computer room object knowledge graph is marked as a matching entity; either of the two methods can be selected to obtain the matching entity, and the entity attribute information corresponding to the matching entity is obtained from the computer room object knowledge graph. The entity attribute information can be the location information corresponding to the matching entity.

7. The method for monitoring a machine room of a multi-sensor fusion intelligent inspection robot according to claim 1, characterized in that: The specific method of obtaining the obstacle risk category is: Step A1: Divide the risk categories into three categories, namely low risk, medium risk and high risk, summarize the objects that pose a threat to the computer room, and classify them into the three risk categories respectively, and use web crawler technology to collect object names, images and corresponding risk category information; Step A2: Construct a risk category knowledge graph based on the collected information. The risk category knowledge graph includes entities, entity attribute information, relationships between entities, and entity types. Entity attribute information is a label that describes entity attributes. The relationship between entities represents the semantic connection between different entities and describes various associations and interactions between entities in the knowledge graph. The entity type defines the category to which the entity belongs and defines the common attributes and relationships between entities. Step A3: Match the object name of the obstacle image output by the output layer in step S06 with the entity in the risk category knowledge graph. The matching methods include string matching and semantic matching. String matching includes exact matching and approximate matching. Exact matching is to compare the string corresponding to the object name of the obstacle image with the string corresponding to the entity in the risk category knowledge graph. If they are exactly the same, then a match is performed. Approximate matching is to use a string similarity algorithm to calculate the similarity between the string corresponding to the object name of the obstacle image and the string corresponding to the entity in the risk category knowledge graph, and match the entity in the risk category knowledge graph with the highest similarity with the object name of the obstacle image. Semantic matching uses word embedding. The word embedding method converts the object name of the obstacle image and the entities in the risk category knowledge graph into vector form through the word embedding model, and matches them by calculating the similarity between the vectors. Step A4: After the match is successful, the successfully matched entity in the risk category knowledge graph is marked as a matching entity; And obtain the entity attribute information corresponding to the matching entity from the risk category knowledge graph, and the entity attribute information is the risk category corresponding to the matching entity.

8. A computer room monitoring system of a multi-sensor fusion intelligent inspection robot, used for using a computer room monitoring method of a multi-sensor fusion intelligent inspection robot according to any one of claims 1 to 7, characterized in that: It includes computer room data collection module, knowledge graph construction module, inspection route planning module, data identification module, secondary planning module, risk identification module, early warning module and intelligent control module; The computer room data acquisition module is used to collect computer room environment data, the computer room environment data includes environment data and object data, and the computer room data acquisition module includes an environment data acquisition unit and an object data acquisition unit; the environment data acquisition unit is used to collect environment data in the computer room and transmit it to the risk identification module; the object data acquisition unit is used to collect object data in the computer room and transmit it to the data identification module; The knowledge graph construction module constructs a computer room object knowledge graph based on the object data in the computer room; The inspection route planning module is used to collect the three-dimensional point cloud data of the computer room, automatically plan the inspection route, and transmit it to the intelligent control module, and mark the inspection route as a long-term inspection route, that is, the computer room inspection work is carried out according to this inspection route every time the inspection is carried out; The data recognition module is used to collect images in the inspection route in real time and identify obstacles. If an obstacle is identified, the obstacle is identified and matched with the computer room object knowledge graph constructed in the knowledge graph construction module. After the match is successful, the update route instruction is sent to the secondary planning module, and the location information of the successfully matched object is stored at the same time; If the match fails, a temporary route planning instruction is sent to the secondary planning module, and the object recognition result is transmitted to the risk identification module; If no obstacle is identified, no data information will be transmitted, and the inspection robot will continue to inspect along the route of the inspection route planning module; The secondary planning module is used to perform secondary planning on the inspection route. The secondary planning module includes a temporary route planning unit and an update route planning unit. The temporary route planning unit is used to receive a temporary route planning instruction, plan a new inspection route and mark it as a temporary inspection route, and then transmit it to the intelligent control module; The update route planning unit is used to receive the update route instruction, plan a new inspection route and mark it as a long-term inspection route, and then transmit it to the intelligent control module; The risk identification module includes an environmental risk identification unit and an object risk identification unit; The environmental risk identification unit is used to receive data from the environmental data collection unit, identify risks in the collected environmental data, and send an environmental warning notification to the warning module if the identification result shows that there is a risk; The object risk identification unit is used to receive the object identification result of the data identification module, establish a risk category knowledge graph, obtain the obstacle risk category and send different levels of object risk warning notifications to the warning module; The early warning module is used to receive the early warning notification from the risk identification module and issue an early warning; The intelligent control module is used to receive the inspection route and control the inspection robot to perform inspection work according to the inspection route. If the received route information is a long-term inspection route, each inspection work is carried out according to the long-term inspection route. If the received route information is a temporary inspection route, the temporary inspection route replaces the long-term inspection route and is only used as the inspection route for the current inspection. When the next inspection work starts, the inspection work continues to be carried out according to the long-term inspection route. The long-term inspection route is the most recently marked long-term inspection route.

Images