Machine room inspection method, device, equipment and medium

Abstract

The invention provides a machine room inspection method, device and equipment and a medium. The method comprises the following steps: acquiring environment information in a machine room through a binocular camera to obtain a first disparity map; performing optimization processing on the first disparity map to obtain a second disparity map; performing distance measurement on an object in the machine room based on the second disparity map to obtain distance information of the object in the machine room; according to the distance information of the object in the machine room, simultaneously starting path search from the position of a visual robot and a target position, and generating a first obstacle avoidance path; and when the visual robot reaches the target position along the first obstacle avoidance path, checking an object at the target position according to a tour inspection instruction. According to the method, on the basis of the distance information of the object in the machine room, the first obstacle avoidance path is generated by simultaneously starting path searching from the position of the visual robot and the target position as starting points, and the path planning efficiency is high and the real-time performance is high.

Description

Technical Field

[0001] The present invention relates to the field of intelligent operation and maintenance technology, and in particular to a computer room inspection method, device, equipment and medium. Background Art

[0002] Although robot technology is developing rapidly, it is rarely used in computer room inspection. At present, the operation and maintenance of mobile cloud computer room servers still cannot do without the support of on-site personnel. For example, on-site acceptance of servers, computer room inspection and troubleshooting all require operation and maintenance personnel to be on-site. Although there are intelligent inspection solutions in the existing technology, these solutions have the problem of low efficiency and poor real-time performance of the path planning corresponding to on-site inspection. Summary of the invention

[0003] The purpose of the present invention is to provide a computer room inspection method, device, equipment and medium to solve the problem of low efficiency and poor real-time performance of path planning corresponding to the existing computer room inspection.

[0004] In order to achieve the above object, in a first aspect, the present invention provides a machine room inspection method, which is applied to a visual robot, comprising:

[0005] Collecting the environment information in the computer room through a binocular camera to obtain a first disparity map;

[0006] performing optimization processing on the first disparity map to obtain a second disparity map;

[0007] Measuring the distance of the object in the machine room based on the second disparity map to obtain the distance information of the object in the machine room;

[0008] According to the distance information of the object in the machine room, path search is started simultaneously from the position of the visual robot and the target position to generate a first obstacle avoidance path;

[0009] When the visual robot reaches the target position along the first obstacle avoidance path, the object at the target position is inspected according to the inspection instruction.

[0010] In some embodiments, the collecting of the environment information in the machine room by using a binocular camera to obtain a first disparity map includes:

[0011] Acquire a first environment image and a second environment image in the computer room captured by the binocular camera;

[0012] Correcting the first environment image and the second environment image respectively according to the internal and external parameter information of the binocular camera to obtain a corrected first environment image and a corrected second environment image;

[0013] The first disparity map is obtained by using a local binary pattern algorithm according to the grayscale image corresponding to the corrected first environment image and the grayscale image corresponding to the corrected second environment image.

[0014] In some embodiments, the optimizing the first disparity map to obtain the second disparity map includes:

[0015] Performing anti-noise processing on the first disparity map to obtain a third disparity map;

[0016] Identifying occlusion points in the third disparity map, and performing filling processing on the occlusion points to obtain a fourth disparity map;

[0017] The fourth disparity map is filtered to obtain the second disparity map.

[0018] In some embodiments, the filling process is performed on the occluded points to obtain a fourth disparity map, including:

[0019] Based on the position of the occluding point, searching for the first non-occluding point on both sides of the occluding point simultaneously;

[0020] The non-occluded point with the smaller disparity value among the two non-occluded points is filled into the position of the occluded point to obtain the fourth disparity map.

[0021] In some embodiments, the method of simultaneously starting a path search from the position of the visual robot and the target position according to the distance information of the object in the machine room to generate a first obstacle avoidance path includes:

[0022] The position of the visual robot is taken as the first root node, and the target position is taken as the second root node, and the following steps are respectively performed until the latest node of the first path tree is the same as the latest node of the second path tree or the distance is less than a preset threshold, and the first obstacle avoidance path is obtained based on the first path tree and the second path tree, and the first path tree initially includes only the first root node, and the second path tree initially includes only the second root node:

[0023] In a computer room environment of a path to be searched, a first sampling point is selected, a first target node closest to the first sampling point in a first path tree is determined, the first target node is extended by a first preset step length along a line connecting the first target node and the first sampling point, the other end of the extension is used as a first new node, and according to distance information of objects in the computer room, whether a line connecting the first target node and the first new node encounters an obstacle is determined, if no obstacle is encountered, the first new node is added to the first path tree, and the next sampling point is selected as the first sampling point; if an obstacle is encountered, the first sampling point and the first new node are discarded, and re-sampling is performed;

[0024] In a computer room environment of a path to be searched, a second sampling point is selected, a second target node closest to the second sampling point in a second path tree is determined, the second target node is extended by a second preset step length along a line connecting the second target node and the second sampling point, the other end of the extension is used as a second new node, and according to distance information of objects in the computer room, whether a line connecting the second target node and the second new node encounters an obstacle is determined; if no obstacle is encountered, the second new node is added to the second path tree, and the next sampling point is selected as the second sampling point; if an obstacle is encountered, the second sampling point and the second new node are discarded, and re-sampling is performed; wherein the selection of the sampling point is determined by environmental factors of the computer room environment.

[0025] In some embodiments, the inspecting the object at the target location includes:

[0026] Acquire a frame image of the object through the binocular camera;

[0027] Based on the feature points of the frame image, obtaining point cloud coordinates corresponding to the frame image;

[0028] Generate a dense point cloud image and a sparse point cloud image related to the object according to the point cloud coordinates corresponding to the frame image;

[0029] An object at the target location is inspected based on the dense point cloud image and the sparse point cloud image.

[0030] In some embodiments, obtaining point cloud coordinates corresponding to the frame image based on the feature points of the frame image includes:

[0031] Performing optical flow tracking on the feature points in the frame image to obtain a first feature point;

[0032] After filtering the first feature points using a preset approximate clustering algorithm, point cloud coordinates corresponding to the frame image are obtained.

[0033] In some embodiments, generating a dense point cloud image related to the object according to the point cloud coordinates corresponding to the frame image includes:

[0034] Perform point cloud stitching according to the point cloud coordinates corresponding to the frame image to obtain a point cloud image corresponding to the frame image;

[0035] Statistical filtering is performed on the point cloud image corresponding to the frame image to obtain a dense point cloud image related to the object.

[0036] In a second aspect, the present invention further provides a machine room inspection device, which is applied to a visual robot, comprising:

[0037] A first acquisition unit is used to collect environmental information in the computer room through a binocular camera to obtain a first disparity map;

[0038] A first processing unit, configured to optimize the first disparity map to obtain a second disparity map;

[0039] a second acquisition unit, configured to measure the distance of the object in the machine room based on the second disparity map to obtain the distance information of the object in the machine room;

[0040] A second processing unit is used to start path search from the position of the visual robot and the target position at the same time according to the distance information of the object in the machine room, and generate a first obstacle avoidance path;

[0041] The inspection unit is used to inspect the object at the target position according to the inspection instruction when the visual robot reaches the target position along the first obstacle avoidance path.

[0042] In a third aspect, the present invention further provides a visual robot, comprising a processor and a transceiver, wherein the transceiver receives and sends data under the control of the processor, and the processor is used to perform the following operations:

[0043] Collecting the environment information in the computer room through a binocular camera to obtain a first disparity map;

[0044] performing optimization processing on the first disparity map to obtain a second disparity map;

[0045] Measuring the distance of the object in the machine room based on the second disparity map to obtain the distance information of the object in the machine room;

[0046] According to the distance information of the object in the machine room, path search is started simultaneously from the position of the visual robot and the target position to generate a first obstacle avoidance path;

[0047] When the visual robot reaches the target position along the first obstacle avoidance path, the object at the target position is inspected according to the inspection instruction.

[0048] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the computer room inspection method described in the first aspect above.

[0049] In a fifth aspect, the present invention further provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps in the computer room inspection method as described in the first aspect above.

[0050] The above technical solution of the present invention has at least the following beneficial effects:

[0051] In an embodiment of the present invention, environmental information in a machine room is collected by a binocular camera to obtain a first disparity map; the first disparity map is optimized to obtain a second disparity map; distances of objects in the machine room are measured based on the second disparity map to obtain distance information of the objects in the machine room; based on the distance information of the objects in the machine room, a path search is started simultaneously from the position of the visual robot and the target position to generate a first obstacle avoidance path; when the visual robot reaches the target position along the first obstacle avoidance path, the object at the target position is inspected according to the inspection instruction. In this way, based on the distance information of the objects in the machine room, a path search is started simultaneously from the position of the visual robot and the target position as starting points to generate the first obstacle avoidance path. The path planning efficiency is high and the real-time performance is strong. The subsequent visual robot can reach the target position along the first obstacle avoidance path to realize the inspection of the object at the target position. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A schematic diagram showing a flow chart of a computer room inspection method provided by an embodiment of the present invention;

[0053] Figure 2 A display diagram showing the relationship between parallax and depth by modeling three-dimensional space points on a two-dimensional plane;

[0054] Figure 3 A schematic diagram showing a multi-peak dynamic environment simulation;

[0055] Figure 4 A schematic diagram showing the simulation of optimal path search for a simple obstacle environment;

[0056] Figure 5 A schematic diagram showing a simulation of searching for an optimal path in a complex obstacle environment;

[0057] Figure 6 A schematic diagram showing an algorithm framework based on improved ORB-SLAM3 provided in an embodiment of the present invention;

[0058] Figure 7 One of the schematic diagrams showing the dense point cloud map construction process of the present invention;

[0059] Figure 8 The second schematic diagram of the dense point cloud map construction process of the present invention;

[0060] Fig. 9 A schematic diagram showing an application scenario of the method of the present invention;

[0061] Fig.10 A second flowchart of the method for inspecting a computer room provided by an embodiment of the present invention is shown;

[0062] Fig.11A schematic diagram showing an implementation scenario of the computer room inspection method of the present invention;

[0063] Fig.12 A schematic diagram showing a module of a machine room inspection device provided by an embodiment of the present invention;

[0064] Fig.13 A schematic diagram showing the hardware structure of a visual robot according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0066] At present, one method is to build a three-dimensional model of the binocular images, use the ant colony algorithm to plan the robot's motion path, and collect the internal environment information and server status of the computer room in real time. However, the ant colony algorithm has the problem of lengthy path planning and poor real-time performance.

[0067] Another method is to collect server status information through fixed cameras in the computer room to determine the server operation status. However, fixed cameras lack mobility and flexibility, and as the number of servers in the computer room increases, the number of cameras required will gradually increase, which will increase the cost.

[0068] Another method is to obtain server operation data based on the address of the baseboard management controller (BMC) of the server device. However, the premise is that the BMC is intact and can be logged in normally and the operation data can be obtained. Once the hardware failure causes the BMC to be unable to log in, the hardware failure cannot be remotely checked, and the inspection work cannot be completed.

[0069] In summary, the existing computer room inspection solutions have problems such as low path planning efficiency, lack of real-time performance, high cost, and inability to remotely troubleshoot hardware failures.

[0070] In order to solve the above technical problems, the present invention provides a machine room inspection method, device, equipment and medium. The method and the device are based on the same application concept. Since the method and the device solve the problem in a similar principle, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0071] like Figure 1 As shown, it is a flow chart of the machine room inspection method provided by the embodiment of the invention. The method is applied to a visual robot, that is, the method is executed by a visual robot. Among them, the method may specifically include:

[0072] Step 101, collecting the environment information in the computer room through a binocular camera to obtain a first disparity map;

[0073] The visual robot is a binocular robot, which is equipped with binocular cameras (left and right cameras). The visual robot uses the binocular cameras to collect environmental information in the machine room and obtain a first disparity map.

[0074] Step 102, optimizing the first disparity map to obtain a second disparity map;

[0075] Here, the first disparity map is optimized, such as anti-noise processing, occlusion point filling, smoothing filtering, etc., in order to obtain a high-precision second disparity map.

[0076] Step 103, measuring the distance of the objects in the machine room based on the second disparity map to obtain the distance information of the objects in the machine room;

[0077] It should be noted that the visual robot will plan its movement trajectory in advance when moving in the computer room. The premise of this is that the visual robot can perceive the distance of surrounding obstacles through vision (that is, measure the distance of objects in the computer room based on the second disparity map) and realize obstacle avoidance and path planning functions according to the obtained obstacle distance information. Therefore, the depth perception ability of the visual robot becomes particularly important.

[0078] Among them, the objects in the computer room include but are not limited to servers, cabinets, sundries, construction personnel, etc. Among them, generally, in addition to the objects involved in the visual robot and the target position (such as the target server), other servers, cabinets, sundries, construction personnel, etc. can be regarded as obstacles by the visual robot.

[0079] Step 104, based on the distance information of the object in the machine room, starting path search from the position of the visual robot and the target position at the same time, and generating a first obstacle avoidance path;

[0080] Based on the distance information of objects in the room, the preset path rule algorithm is used to start the path search from the position of the visual robot and the target position at the same time to generate the first obstacle avoidance path. That is, the first obstacle avoidance path is generated by starting the search from the position of the visual robot and the target position as the starting points (these two endpoints) at the same time, and its path planning efficiency is high and real-time performance is strong.

[0081] Optionally, the preset path rule algorithm is a combination of a bidirectional fast search random tree algorithm and a particle swarm optimization algorithm with an added environmental factor. The first obstacle avoidance path is the shortest path between the position of the visual robot and the target position.

[0082] Step 105: When the visual robot reaches the target position along the first obstacle avoidance path, the object at the target position is inspected according to the inspection instruction.

[0083] Here, the inspection instruction includes but is not limited to one of the following instructions: an inspection instruction for newly installed equipment; an inspection instruction for changed equipment; a safety inspection instruction for a target location.

[0084] Among them, if the inspection instruction is an inspection instruction for a newly installed device, the object at the target location is the newly installed device; if the inspection instruction is an inspection instruction for a changed server, the object at the target location is the inspection instruction for the changed server; if the inspection instruction is a security inspection instruction for the target location, the object at the target location is the environment around the target location.

[0085] Through the first obstacle avoidance path obtained efficiently and in real time, the visual robot can reach the target position along the first obstacle avoidance path and realize the inspection of the object at the target position.

[0086] In some embodiments, the above step 101, collecting the environment information in the machine room by using a binocular camera to obtain the first disparity map, may include:

[0087] Step 1011, obtaining a first environment image and a second environment image in the computer room captured by the binocular camera;

[0088] Step 1012, correcting the first environment image and the second environment image respectively according to the internal and external parameter information of the binocular camera to obtain a corrected first environment image and a corrected second environment image;

[0089] In order to ensure the accuracy of the captured image, it is inevitable to consider the distortion and correction of the binocular camera. The checkerboard calibration method can be used to obtain the internal and external parameter information of the binocular camera. Based on the internal and external parameter information of the binocular camera, the first environment image and the second environment image are corrected respectively to correct the image distortion and eliminate the image distortion.

[0090] Step 1013: Obtain the first disparity map by using a local binary pattern algorithm according to the grayscale image corresponding to the corrected first environment image and the grayscale image corresponding to the corrected second environment image.

[0091] Optionally, a region-based local dense matching algorithm is used for cost calculation.

[0092] Specifically, a difference grayscale image between a grayscale image corresponding to the corrected first environment image and a grayscale image corresponding to the corrected second environment image is calculated; a sliding search is performed on the difference grayscale image according to a first search window to obtain a matching cost; based on the obtained matching cost, a local binary pattern algorithm is used to perform disparity calculation to obtain a first disparity map.

[0093] The difference grayscale image expression is as follows:

[0094] I h (x,y)=|I l (x,y)-I r (x,y)| (1)

[0095] Among them, I l (x, y) represents the grayscale value of the left image (the first environment image after correction) at (x, y), I r (x, y) represents the gray value of the right image (the corrected second environment image) at (x, y), I h (x,y) represents the difference at (x,y).

[0096] The matching cost expression is as follows:

[0097]

[0098] Wherein, D(x, y, d) refers to the sum of the absolute values ​​of the pixels in the first search window at the difference grayscale image (x, y), d is the disparity, p and q are the offsets, and M is the neighborhood.

[0099] It should be understood that if M=3, it can be understood as a 3×3 grid, and the grid is in the neighborhood of the center, and the neighborhood is composed of pixel coordinates.

[0100] In view of the influence of light intensity on image parallax in complex environments (such as computer room environments), the present invention adopts a local binary pattern (LBP) algorithm to perform parallax calculation and obtain a first parallax map. The idea of ​​the LBP algorithm is to compare and distinguish only the grayscale values ​​symmetrical to the central pixel in the window. Specifically, the first and last pixels are symmetrically distinguished with the central pixel as the end point to form a binary code, and the cost value is obtained by XOR summation. This method only depends on the grayscale size relationship, so it has strong adaptability to light. The above-mentioned difference grayscale image combined with the LBP algorithm can improve the algorithm's adaptability to light.

[0101] The LBP algorithm expression is as follows:

[0102]

[0103] Among them, r represents the neighborhood radius, m represents the number of pixels within the radius, σ is the threshold, s is the sign function, and n i is the grayscale of pixels evenly distributed in the window, and i is the index.

[0104] In some embodiments or on the basis of the above embodiments, the above step 102, optimizing the first disparity map to obtain the second disparity map, may include:

[0105] Step 1021, performing anti-noise processing on the first disparity map to obtain a third disparity map;

[0106] In order to improve the noise resistance of the image, the Sober operator can be used to detect the edge of the first disparity map. The Sober operator also has the idea of ​​differentiation and Gaussian smoothing, and has good resistance to noise in the machine room. The specific expressions are the following formulas (4), (5), and (6):

[0107]

[0108] G=|d x *I(x,y)|+|d y *I(x,y)| (6)

[0109] Among them, d x d y Represents the matrix after the grayscale image is convolved by the two-dimensional coordinate system. G is the value of the convolution of the two sets of matrices with the grayscale image plane.

[0110] According to the size of G value, select a reasonable window size to protect the edge of the image. If the G value is large, a small window is used to protect the edge information, and if the G value is small, a large window is used to protect the edge of the image. The purpose of protecting the edge of the image is to ensure the clarity of the edge of the image.

[0111] Here, in order to further distinguish the image edge efficiently, the following large law discrimination can be used to determine the image edge. The expression corresponding to the large law discrimination is the following formula (7):

[0112]

[0113] Among them, f(x, y) represents the gray value judgment result. The result of 255 is judged as the edge area, and the result of 0 is judged as the non-edge area. Reasonable penalty coefficients are used to process the two areas respectively, and T is the ideal threshold.

[0114] Step 1022, identifying occlusion points in the third disparity map, and performing filling processing on the occlusion points to obtain a fourth disparity map;

[0115] In view of the occlusion problem existing in the complex environment of the computer room, the left-right consistency detection algorithm is used to identify the occlusion points in the third disparity map. The expression of the left-right consistency detection algorithm to identify the occlusion points is the following formula (8):

[0116] |h l (x,y)-h r [(x,y)-h l (x,y)]|>thr (8)

[0117] Among them, h l(x,y) is the estimated disparity of the left disparity map, and the matching point of this point is in [(x,y)-h l (x, y)] position, if the absolute value is greater than the threshold thr, then the point is an occlusion point.

[0118] It should be noted that the third disparity map includes a left disparity map and a right disparity map.

[0119] In some embodiments, in step 1022, filling the occluded points to obtain a fourth disparity map may include:

[0120] Based on the position of the occluding point, searching for the first non-occluding point on both sides of the occluding point simultaneously;

[0121] The non-occluded point with the smaller disparity value among the two non-occluded points is filled into the position of the occluded point to obtain the fourth disparity map.

[0122] In a binocular image, the occluded point only exists in one image. Therefore, according to the position of the occluded point, the first non-occluded point can be searched on both sides of the occluded point at the same time, and the occluded point can be filled with the smallest non-occluded point disparity. The specific expression is the following formula (9):

[0123] q′=min(q l ,q r ) (9)

[0124] Among them, q′ is the ideal disparity value of the occlusion point.

[0125] Step 1023: Filter the fourth disparity map to obtain the second disparity map.

[0126] The edge of the fourth disparity map may be filtered. Specifically, the fourth disparity map may be filtered using a guided filtering algorithm to obtain the second disparity map. The guided filtering algorithm is expressed as the following formula (10):

[0127]

[0128] Among them, μ τ is the grayscale mean, is the variance, ν is the regularization coefficient, and τ is the total number of pixels in the window.

[0129] Here, by optimizing the first disparity map, a high-precision second disparity map can be obtained.

[0130] It should be noted that the visual robot will plan its movement trajectory in advance when moving in the computer room. This is based on the premise that the visual robot can perceive the distance of surrounding obstacles through vision, so the depth perception ability of the visual robot becomes particularly important. The depth perception ability of the visual robot to objects in the computer room scene depends on the parallax value of the collected image. Accurate parallax value provides the prerequisite for the system's high-precision depth perception, that is, the more accurate the parallax, the more accurate the depth perception result. The relationship between the two can be demonstrated by building a two-dimensional plane model for three-dimensional space points, such as Figure 2 shown.

[0131] In the two-dimensional model, based on the similarity of triangles, we can infer that:

[0132]

[0133] Where B is the baseline, u l Indicates the horizontal coordinate of the left pixel, u r represents the horizontal coordinate of the right pixel, and the difference between the two is the disparity d, that is, disparity d = u l -u r , Z is the depth.

[0134] Simplify:

[0135]

[0136] According to the above matching algorithm, the size of the parallax d can be calculated, and the focal length f is the intrinsic value of the camera. According to the principle of triangulation, the depth information of objects in the scene can be calculated by the algorithm, and the parallax is inversely proportional to the depth.

[0137] In this way, based on the second disparity map, the distance of objects in the computer room (such as obstacles, construction personnel, cabinets, etc.) is determined by depth perception technology to obtain distance information of the objects in the computer room.

[0138] In some embodiments, the above step 104, based on the distance information of the object in the machine room, starts the path search from the position of the visual robot and the target position at the same time to generate the first obstacle avoidance path, including:

[0139] The position of the visual robot is taken as the first root node, and the target position is taken as the second root node, and the following steps are respectively performed until the latest node of the first path tree is the same as the latest node of the second path tree or the distance is less than a preset threshold, and the first obstacle avoidance path is obtained based on the first path tree and the second path tree, and the first path tree initially includes only the first root node, and the second path tree initially includes only the second root node:

[0140] Step 1041, in the computer room environment of the path to be searched, a first sampling point is selected, a first target node closest to the first sampling point in the first path tree is determined, the first target node is extended by a first preset step length along the direction of the line connecting the first target node and the first sampling point, the other end of the extension is used as the first new node, and according to the distance information of the object in the computer room, whether the line connecting the first target node and the first new node encounters an obstacle is determined, if no obstacle is encountered, the first new node is added to the first path tree, and the next sampling point is selected as the first sampling point; if an obstacle is encountered, the first sampling point and the first new node are discarded, and re-sampling is performed;

[0141] Step 1042: In the computer room environment of the path to be searched, a second sampling point is selected, a second target node closest to the second sampling point in the second path tree is determined, the second target node is extended by a second preset step length along the direction of the line connecting the second target node and the second sampling point, the other end of the extension is used as a second new node, and according to the distance information of the objects in the computer room, it is determined whether the line connecting the second target node and the second new node encounters an obstacle; if no obstacle is encountered, the second new node is added to the second path tree, and the next sampling point is selected as the second sampling point; if an obstacle is encountered, the second sampling point and the second new node are discarded and re-sampled; wherein the selection of the sampling point is determined by the environmental factors of the computer room environment.

[0142] Here, the generation formula of the new node (the first new node or the second new node) is as follows:

[0143]

[0144] Among them, S′ is the new node, S is the nearest node, q is the step size, and λ is the direction weight of the target point. is the unit vector in the direction of the target point, is the unit vector in the direction of the sampling point. When λ=1, the new node grows in a guided manner towards the target point; when λ=0, the growth of the new node is not guided.

[0145]

[0146] S 1 is the target node, S r are random sampling points.

[0147] It should be noted that accurate visual information provides a prerequisite for the movement of the visual robot. When the visual robot inspects the target machine in the computer room, it needs to plan the moving path reasonably and consider many factors, such as the movement direction of the visual robot, obstacles and path planning, etc., to pursue the shortest path and the least movement time. In the above path tree generation process, the selection of sampling points takes into account the environmental factors of the computer room environment. In the process of generating the first path tree and the second path tree, the particle swarm optimization algorithm is also introduced. The added environmental factors make it possible to adapt to the influence of the polymorphic environment during the search process, improve the ability to search for the optimal solution globally, and allow the random tree nodes to grow in a guided manner, solving the problem of too long growth paths and blind growth, reducing the blindness of random tree growth, and improving the efficiency of algorithm execution.

[0148] The polymorphic environment expression is the following formula (16):

[0149] f=asinacos(2a)-2asin(3a)(bsinb)cos(2b)-2bsin(3b) (16)

[0150] Among them, multi-peak dynamic environment simulation Figure 3 shown.

[0151] The above-mentioned combination algorithm of the bidirectional fast search random tree algorithm and the particle swarm optimization algorithm with added environmental factors can effectively avoid obstacles. In order to pursue the real-time performance of the visual robot path planning, in the process of generating the obstacle avoidance path using the above-mentioned combination algorithm, the minimum cost value of searching the shortest path is solved, and the expression is the following formula (17):

[0152] F(n)=G(n)+H(n) (17)

[0153] Among them, n is the node information, F(n) is the cost value; G(n) is the optimal cost up to the current node; H(n) is the optimal cost from the current node to the target node.

[0154] It should be noted that, based on the first path tree and the second path tree, each node for generating the first obstacle avoidance path is obtained, and then, along the direction from the position of the visual robot (end node) to the target position (end node) (or the opposite direction), each node is linearly connected in sequence:

[0155] Specifically, the Euclidean distance is used to represent the shortest distance between two coordinates in three-dimensional space:

[0156]

[0157] Finally, the path is smoothed using a B-spline curve to obtain the first obstacle avoidance path.

[0158] like Figure 4 As shown in FIG. 1 , it is a simulation diagram of searching for the optimal path in a simple obstacle environment; Figure 5 As shown, it is a simulation diagram of searching for the optimal path in a complex obstacle environment.

[0159] In some embodiments, in the above step 105, inspecting the object at the target location may include:

[0160] Step 1051, obtaining a frame image of the object through the binocular camera;

[0161] Step 1052, based on the feature points of the frame image, obtaining the point cloud coordinates corresponding to the frame image;

[0162] Optionally, step 1052 may include:

[0163] Performing optical flow tracking on the feature points in the frame image to obtain a first feature point;

[0164] Since the computer room environment may be dim and complex, feature points are easily lost, and the optical flow method (i.e., optical flow tracking) can be used to stably track feature points.

[0165] After filtering the first feature points using a preset approximate clustering algorithm, point cloud coordinates corresponding to the frame image are obtained.

[0166] Optionally, the preset approximate clustering algorithm is the prosac algorithm. If the first feature points are too many or even overlapped, the redundant feature points need to be removed. The prosac algorithm is used to filter and update the feature points, and the point cloud coordinates corresponding to the frame image are obtained based on the coordinates of the updated feature points.

[0167] Step 1053, generating a dense point cloud image and a sparse point cloud image related to the object according to the point cloud coordinates corresponding to the frame image;

[0168] Optionally, in step 1053, generating a dense point cloud image related to the object according to the point cloud coordinates corresponding to the frame image may include:

[0169] Perform point cloud stitching according to the point cloud coordinates corresponding to the frame image to obtain a point cloud image corresponding to the frame image;

[0170] Statistical filtering is performed on the point cloud image corresponding to the frame image to obtain a dense point cloud image related to the object.

[0171] Here, according to the point cloud coordinates corresponding to the frame image, the sparse point cloud image to be processed (i.e., local mapping) is obtained; then the sparse point cloud image to be processed is optimized, the processed key frame enters loop detection, and the global dense map is updated.

[0172] Step 1054 : Inspect the object at the target location based on the dense point cloud image and the sparse point cloud image.

[0173] Here, the present invention can be improved on the basis of the three major threads corresponding to the ORB-SLAM3 algorithm to realize the above-mentioned SLAM dense mapping, wherein the algorithm framework based on the improved ORB-SLAM3 (see Figure 6 ,)Construct a three-dimensional dense mapping algorithm, which can prevent safety accidents in the computer room and improve the safety of operation and maintenance.

[0174] See also Figure 6 The improved ORB-SLAM3 algorithm framework improves the tracking thread based on the original three threads of ORB-SLAM3, introduces the optical flow method and prosac algorithm, and adds a dense mapping thread after the tracking thread. The feature points optimized by the tracking thread are stitched and filtered, and the local mapping and loop detection continue to use the original thread of ORB-SLAM3.

[0175] A dim indoor environment similar to the complex environment inside the computer room is selected for simulation experiments. The TUM standard dataset freiburg2 desk sequence can be used. The dense point cloud map (image) construction process can be seen in Figure 7 (corresponding to the dense point cloud map construction process 1) as shown and Figure 8 (corresponding to the dense point cloud map construction process 2) as shown.

[0176] See also Fig. 9 , illustrating several application scenarios of the method of the present invention.

[0177] Application scenario 1: On-site acceptance

[0178] The visual robot can inspect the on-site equipment installation, wiring specifications, equipment labels, cable labels, asset labels, on-site cleanliness, etc. in the computer room. That is, if the inspection instruction is an inspection instruction for newly installed equipment, when the visual robot reaches the target position of the newly installed equipment through the planned path, based on the collected images of the newly installed equipment (such as server panels, cables and labels, etc.), it constructs dense point cloud images and sparse point cloud images, and inspects the newly installed equipment based on the dense point cloud images and sparse point cloud images. Optionally, the dense point cloud images and sparse point cloud images are sent to the sending end device of the inspection instruction (such as the cloud control platform), and the operation and maintenance personnel inspect the newly installed equipment based on the feedback dense point cloud images and sparse point cloud images.

[0179] Application scenario 2: Change inspection

[0180] The server hardware changes mainly include remote BMC upgrade, BIOS adjustment and Bond reconfiguration. The changes of the online equipment must shield the corresponding physical machine address. After shielding, the FOC alarm platform will not receive all the server alarms, which increases the risk of the change to a certain extent. The visual robot can inspect the server that is being changed. That is, if the inspection instruction is an inspection instruction for the changed server, when the visual robot reaches the target position of the changed server through the planned path, based on the collected image of the changed server, a dense point cloud image and a sparse point cloud image are constructed, and the changed server is inspected according to the dense point cloud image and the sparse point cloud image. Optionally, the dense point cloud image and the sparse point cloud image are sent to the sending end device of the inspection instruction (such as the cloud control platform), and the change personnel inspect the changed server based on the feedback dense point cloud image and the sparse point cloud image, such as observing the fault lights of the hard disk, motherboard, network card, fan, power supply and other fault lights on the front and rear panels of the server, timely monitoring the status of the server being changed, locating the hardware fault, and the change personnel immediately contact the monitoring group to follow the fault handling process when they find that the fault light is abnormal.

[0181] Application scenario three: safety testing

[0182] It should be noted that the visual robot not only integrates binocular cameras, but is also equipped with temperature, humidity and acceleration sensors. The inspection of the computer room can not only observe the internal situation of the computer room through vision, but also dynamically detect whether the internal temperature and humidity of the computer room are constant through the visual robot. When a fire occurs, the temperature sensor will prompt that the temperature detection is abnormal, and the robot will immediately issue an alarm message. In addition, the acceleration sensor can capture the ground vibration caused by seismic waves, and then issue an alarm to report to the mobile cloud console.

[0183] After the safety alarm information is triggered, the mobile cloud console sends a safety inspection instruction to the target location (which can be a preset location). When the visual robot reaches the target location through the planned path, it constructs a dense point cloud image and a sparse point cloud image based on the collected images of the surrounding environment of the target location (that is, the indoor SLAM dense mapping function is turned on), and inspects the surrounding environment of the target location based on the dense point cloud image and the sparse point cloud image. Optionally, the dense point cloud image and the sparse point cloud image are sent to the sending device of the inspection instruction (such as the cloud control platform), and the staff quickly locates the dangerous area based on the feedback of the dense point cloud image and the sparse point cloud image.

[0184] For example, when a fire occurs in the computer room, dry powder fire extinguishing and gas fire extinguishing are immediately triggered inside the computer room. Since SLAM mapping does not rely on prior information about the environment, it can establish a dense map in an environment with dense smoke and dust and dim vision, restore scene information, quickly locate the fire area, and feedback the on-site situation in the computer room, thus providing preparation for fire rescue. In addition, SLAM technology integrates the inertial navigation system and strengthens the visual anti-shake function, which can effectively establish a global dense map during an earthquake.

[0185] See below Fig.10 , an example is used to illustrate the specific implementation process of the computer room inspection method of the present invention.

[0186] Step 1001, the remote mobile cloud console sends an inspection instruction to the visual robot, and the visual robot can retrieve the server information and lock the server location according to the SN serial number.

[0187] Step 1002: The binocular camera collects the environment information in the machine room, performs grayscale processing on the collected environment image, and restores the undistorted image through the parallax optimization algorithm to obtain a high-precision parallax map.

[0188] Step 1003: The visual robot uses a high-precision disparity map and depth perception technology to measure the distance of objects in the computer room scene and perceive the distance information of the objects in the computer room.

[0189] Here, the distance information of objects in the machine room includes the distance between the surrounding objects and the visual robot.

[0190] Step 1004, planning an optimal obstacle avoidance path according to the distance information of objects in the machine room.

[0191] During the movement of the visual robot, in addition to itself and the target server, other servers, cabinets, debris, construction workers, etc. are regarded as obstacles by the visual robot. The visual robot plans the movement path according to the location information of the target server. The generated path will start searching from both ends of the location of the visual robot and the target server at the same time, generating a robot movement path with the shortest distance and the least time consumption, thereby improving the real-time performance of the system.

[0192] Step 1005, after the visual robot moves to the target server according to the planned path, it performs relevant inspections on the server according to the inspection instructions and the SLAM dense map obtained based on the collected images of the target server.

[0193] The implementation scenario of the above machine room inspection method can be found in Fig.11 shown.

[0194] like Fig.12As shown, an embodiment of the present invention further provides a machine room inspection device, which is applied to a visual robot. The device may include:

[0195] The first acquisition unit 1201 is used to collect the environment information in the computer room through a binocular camera to obtain a first disparity map;

[0196] A first processing unit 1202 is configured to optimize the first disparity map to obtain a second disparity map;

[0197] A second acquiring unit 1203 is configured to measure the distance of the object in the computer room based on the second disparity map to obtain the distance information of the object in the computer room;

[0198] The second processing unit 1204 is used to start path search from the position of the visual robot and the target position at the same time according to the distance information of the object in the machine room, and generate a first obstacle avoidance path;

[0199] The inspection unit 1205 is used to inspect the object at the target position according to the inspection instruction when the visual robot reaches the target position along the first obstacle avoidance path.

[0200] Optionally, the first acquiring unit 1201 is specifically configured to:

[0201] Acquire a first environment image and a second environment image in the computer room captured by the binocular camera;

[0202] Correcting the first environment image and the second environment image respectively according to the internal and external parameter information of the binocular camera to obtain a corrected first environment image and a corrected second environment image;

[0203] The first disparity map is obtained by using a local binary pattern algorithm according to the grayscale image corresponding to the corrected first environment image and the grayscale image corresponding to the corrected second environment image.

[0204] Optionally, the first processing unit 1202 is specifically configured to:

[0205] Performing anti-noise processing on the first disparity map to obtain a third disparity map;

[0206] Identifying occlusion points in the third disparity map, and performing filling processing on the occlusion points to obtain a fourth disparity map;

[0207] The fourth disparity map is filtered to obtain the second disparity map.

[0208] Optionally, the first processing unit 1202 is specifically configured to:

[0209] Based on the position of the occluding point, searching for the first non-occluding point on both sides of the occluding point simultaneously;

[0210] The non-occluded point with the smaller disparity value among the two non-occluded points is filled into the position of the occluded point to obtain the fourth disparity map.

[0211] Optionally, the second processing unit 1204 is specifically configured to:

[0212] The position of the visual robot is taken as the first root node, and the target position is taken as the second root node, and the following steps are respectively performed until the latest node of the first path tree is the same as the latest node of the second path tree or the distance is less than a preset threshold, and the first obstacle avoidance path is obtained based on the first path tree and the second path tree, and the first path tree initially includes only the first root node, and the second path tree initially includes only the second root node:

[0213] In a computer room environment of a path to be searched, a first sampling point is selected, a first target node closest to the first sampling point in a first path tree is determined, the first target node is extended by a first preset step length along a line connecting the first target node and the first sampling point, the other end of the extension is used as a first new node, and according to distance information of objects in the computer room, whether a line connecting the first target node and the first new node encounters an obstacle is determined, if no obstacle is encountered, the first new node is added to the first path tree, and the next sampling point is selected as the first sampling point; if an obstacle is encountered, the first sampling point and the first new node are discarded, and re-sampling is performed;

[0214] In a computer room environment of a path to be searched, a second sampling point is selected, a second target node closest to the second sampling point in a second path tree is determined, the second target node is extended by a second preset step length along a line connecting the second target node and the second sampling point, the other end of the extension is used as a second new node, and according to distance information of objects in the computer room, whether a line connecting the second target node and the second new node encounters an obstacle is determined; if no obstacle is encountered, the second new node is added to the second path tree, and the next sampling point is selected as the second sampling point; if an obstacle is encountered, the second sampling point and the second new node are discarded, and re-sampling is performed; wherein the selection of the sampling point is determined by environmental factors of the computer room environment.

[0215] Optionally, the inspection unit 1205 is specifically used for:

[0216] Acquire a frame image of the object through the binocular camera;

[0217] Based on the feature points of the frame image, obtaining point cloud coordinates corresponding to the frame image;

[0218] Generate a dense point cloud image and a sparse point cloud image related to the object according to the point cloud coordinates corresponding to the frame image;

[0219] An object at the target location is inspected based on the dense point cloud image and the sparse point cloud image.

[0220] Optionally, the inspection unit 1205 is specifically used for:

[0221] Performing optical flow tracking on the feature points in the frame image to obtain a first feature point;

[0222] After filtering the first feature points using a preset approximate clustering algorithm, point cloud coordinates corresponding to the frame image are obtained.

[0223] Optionally, the inspection unit 1205 is specifically used for:

[0224] Perform point cloud stitching according to the point cloud coordinates corresponding to the frame image to obtain a point cloud image corresponding to the frame image;

[0225] Statistical filtering is performed on the point cloud image corresponding to the frame image to obtain a dense point cloud image related to the object.

[0226] The computer room inspection device of the embodiment of the present invention collects environmental information in the computer room through a binocular camera to obtain a first disparity map; optimizes the first disparity map to obtain a second disparity map; measures the distance of objects in the computer room based on the second disparity map to obtain distance information of the objects in the computer room; starts path search from the position of the visual robot and the target position at the same time according to the distance information of the objects in the computer room to generate a first obstacle avoidance path; when the visual robot reaches the target position along the first obstacle avoidance path, the object at the target position is inspected according to the inspection instruction. In this way, based on the distance information of the objects in the computer room, the path search is started from the position of the visual robot and the target position as the starting points at the same time to generate the first obstacle avoidance path. The path planning efficiency is high and the real-time performance is strong. The subsequent visual robot can reach the target position along the first obstacle avoidance path to realize the inspection of the object at the target position.

[0227] In order to better achieve the above goals, Fig.13 As shown, an embodiment of the present invention further provides a visual robot, including a processor 1300 and a transceiver 1310, wherein the transceiver 1310 receives and sends data under the control of the processor 1300, and the processor 1300 is used to perform the following process:

[0228] Collecting the environment information in the computer room through a binocular camera to obtain a first disparity map;

[0229] performing optimization processing on the first disparity map to obtain a second disparity map;

[0230] Measuring the distance of the object in the machine room based on the second disparity map to obtain the distance information of the object in the machine room;

[0231] According to the distance information of the object in the machine room, path search is started simultaneously from the position of the visual robot and the target position to generate a first obstacle avoidance path;

[0232] When the visual robot reaches the target position along the first obstacle avoidance path, the object at the target position is inspected according to the inspection instruction.

[0233] In some embodiments, the processor 1300 is further configured to:

[0234] Acquire a first environment image and a second environment image in the computer room captured by the binocular camera;

[0235] Correcting the first environment image and the second environment image respectively according to the internal and external parameter information of the binocular camera to obtain a corrected first environment image and a corrected second environment image;

[0236] The first disparity map is obtained by using a local binary pattern algorithm according to the grayscale image corresponding to the corrected first environment image and the grayscale image corresponding to the corrected second environment image.

[0237] In some embodiments, the processor 1300 is further configured to:

[0238] Performing anti-noise processing on the first disparity map to obtain a third disparity map;

[0239] Identifying occlusion points in the third disparity map, and performing filling processing on the occlusion points to obtain a fourth disparity map;

[0240] The fourth disparity map is filtered to obtain the second disparity map.

[0241] In some embodiments, the processor 1300 is further configured to:

[0242] Based on the position of the occluding point, searching for the first non-occluding point on both sides of the occluding point simultaneously;

[0243] The non-occluded point with the smaller disparity value among the two non-occluded points is filled into the position of the occluded point to obtain the fourth disparity map.

[0244] In some embodiments, the processor 1300 is further configured to:

[0245] The position of the visual robot is taken as the first root node, and the target position is taken as the second root node, and the following steps are respectively performed until the latest node of the first path tree is the same as the latest node of the second path tree or the distance is less than a preset threshold, and the first obstacle avoidance path is obtained based on the first path tree and the second path tree, and the first path tree initially includes only the first root node, and the second path tree initially includes only the second root node:

[0246] In a computer room environment of a path to be searched, a first sampling point is selected, a first target node closest to the first sampling point in a first path tree is determined, the first target node is extended by a first preset step length along a line connecting the first target node and the first sampling point, the other end of the extension is used as a first new node, and according to distance information of objects in the computer room, whether a line connecting the first target node and the first new node encounters an obstacle is determined, if no obstacle is encountered, the first new node is added to the first path tree, and the next sampling point is selected as the first sampling point; if an obstacle is encountered, the first sampling point and the first new node are discarded, and re-sampling is performed;

[0247] In a computer room environment of a path to be searched, a second sampling point is selected, a second target node closest to the second sampling point in a second path tree is determined, the second target node is extended by a second preset step length along a line connecting the second target node and the second sampling point, the other end of the extension is used as a second new node, and according to distance information of objects in the computer room, whether a line connecting the second target node and the second new node encounters an obstacle is determined; if no obstacle is encountered, the second new node is added to the second path tree, and the next sampling point is selected as the second sampling point; if an obstacle is encountered, the second sampling point and the second new node are discarded, and re-sampling is performed; wherein the selection of the sampling point is determined by environmental factors of the computer room environment.

[0248] In some embodiments, the processor 1300 is further configured to:

[0249] Acquire a frame image of the object through the binocular camera;

[0250] Based on the feature points of the frame image, obtaining point cloud coordinates corresponding to the frame image;

[0251] Generate a dense point cloud image and a sparse point cloud image related to the object according to the point cloud coordinates corresponding to the frame image;

[0252] An object at the target location is inspected based on the dense point cloud image and the sparse point cloud image.

[0253] In some embodiments, the processor 1300 is further configured to:

[0254] Performing optical flow tracking on the feature points in the frame image to obtain a first feature point;

[0255] After filtering the first feature points using a preset approximate clustering algorithm, point cloud coordinates corresponding to the frame image are obtained.

[0256] In some embodiments, the processor 1300 is further configured to:

[0257] Perform point cloud stitching according to the point cloud coordinates corresponding to the frame image to obtain a point cloud image corresponding to the frame image;

[0258] Statistical filtering is performed on the point cloud image corresponding to the frame image to obtain a dense point cloud image related to the object.

[0259] The visual robot of the embodiment of the present invention collects environmental information in the machine room through a binocular camera to obtain a first disparity map; optimizes the first disparity map to obtain a second disparity map; measures the distance of objects in the machine room based on the second disparity map to obtain distance information of the objects in the machine room; starts path search from the position of the visual robot and the target position at the same time according to the distance information of the objects in the machine room to generate a first obstacle avoidance path; when the visual robot reaches the target position along the first obstacle avoidance path, the object at the target position is inspected according to the inspection instruction. In this way, based on the distance information of the objects in the machine room, the path search is started from the position of the visual robot and the target position as the starting points at the same time to generate the first obstacle avoidance path. The path planning efficiency is high and the real-time performance is strong. The subsequent visual robot can reach the target position along the first obstacle avoidance path to realize the inspection of the object at the target position.

[0260] An embodiment of the present invention also provides a visual robot, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the various processes in the embodiment of the computer room inspection method as described above are implemented, and the same technical effect can be achieved. To avoid repetition, they will not be described here.

[0261] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, each process in the above-mentioned computer room inspection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0262] The embodiment of the present invention also provides a computer program product, including computer instructions, which, when executed by a processor, implement the following Figure 1 The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0263] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0264] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A device that specifies functions in one or more processes and / or one or more blocks.

[0265] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable storage medium produce a paper product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0266] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that the computer or other programmable device executes a series of operating steps to produce a computer-implemented process, thereby providing instructions executed on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0267] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A machine room inspection method, applied to a visual robot, characterized in that: include: Collecting the environment information in the computer room through a binocular camera to obtain a first disparity map; performing optimization processing on the first disparity map to obtain a second disparity map; Measuring the distance of the object in the machine room based on the second disparity map to obtain the distance information of the object in the machine room; According to the distance information of the object in the machine room, path search is started simultaneously from the position of the visual robot and the target position to generate a first obstacle avoidance path; When the visual robot reaches the target position along the first obstacle avoidance path, the object at the target position is inspected according to the inspection instruction.

2. The method according to claim 1, characterized in that The step of collecting the environment information in the computer room by using a binocular camera to obtain a first disparity map includes: Acquire a first environment image and a second environment image in the computer room captured by the binocular camera; Correcting the first environment image and the second environment image respectively according to the internal and external parameter information of the binocular camera to obtain a corrected first environment image and a corrected second environment image; The first disparity map is obtained by using a local binary pattern algorithm according to the grayscale image corresponding to the corrected first environment image and the grayscale image corresponding to the corrected second environment image.

3. The method according to claim 1 or 2, characterized in that: The step of optimizing the first disparity map to obtain a second disparity map includes: Performing anti-noise processing on the first disparity map to obtain a third disparity map; Identifying occlusion points in the third disparity map, and performing filling processing on the occlusion points to obtain a fourth disparity map; The fourth disparity map is filtered to obtain the second disparity map.

4. The method according to claim 3, characterized in that: The filling process is performed on the occluded points to obtain a fourth disparity map, including: Based on the position of the occluding point, searching for the first non-occluding point on both sides of the occluding point simultaneously; The non-occluded point with the smaller disparity value among the two non-occluded points is filled into the position of the occluded point to obtain the fourth disparity map.

5. The method according to claim 1, characterized in that The method of simultaneously starting a path search from the position of the visual robot and the target position according to the distance information of the object in the machine room to generate a first obstacle avoidance path includes: The position of the visual robot is taken as the first root node, and the target position is taken as the second root node, and the following steps are respectively performed until the latest node of the first path tree is the same as the latest node of the second path tree or the distance is less than a preset threshold, and the first obstacle avoidance path is obtained based on the first path tree and the second path tree, and the first path tree initially includes only the first root node, and the second path tree initially includes only the second root node: In a computer room environment of a path to be searched, a first sampling point is selected, a first target node closest to the first sampling point in a first path tree is determined, the first target node is extended by a first preset step length along a line connecting the first target node and the first sampling point, the other end of the extension is used as a first new node, and according to distance information of objects in the computer room, whether a line connecting the first target node and the first new node encounters an obstacle is determined, if no obstacle is encountered, the first new node is added to the first path tree, and the next sampling point is selected as the first sampling point; if an obstacle is encountered, the first sampling point and the first new node are discarded, and re-sampling is performed; In a computer room environment of a path to be searched, a second sampling point is selected, a second target node closest to the second sampling point in a second path tree is determined, the second target node is extended by a second preset step length along a line connecting the second target node and the second sampling point, the other end of the extension is used as a second new node, and according to distance information of objects in the computer room, whether a line connecting the second target node and the second new node encounters an obstacle is determined; if no obstacle is encountered, the second new node is added to the second path tree, and the next sampling point is selected as the second sampling point; if an obstacle is encountered, the second sampling point and the second new node are discarded, and re-sampling is performed; wherein the selection of the sampling point is determined by environmental factors of the computer room environment.

6. The method according to claim 1, characterized in that The inspecting the object at the target location includes: Acquire a frame image of the object through the binocular camera; Based on the feature points of the frame image, obtaining point cloud coordinates corresponding to the frame image; Generate a dense point cloud image and a sparse point cloud image related to the object according to the point cloud coordinates corresponding to the frame image; An object at the target location is inspected based on the dense point cloud image and the sparse point cloud image.

7. The method according to claim 6, characterized in that The step of obtaining point cloud coordinates corresponding to the frame image based on the feature points of the frame image includes: Performing optical flow tracking on the feature points in the frame image to obtain a first feature point; After filtering the first feature points using a preset approximate clustering algorithm, point cloud coordinates corresponding to the frame image are obtained.

8. The method according to claim 6, characterized in that The step of generating a dense point cloud image related to the object according to the point cloud coordinates corresponding to the frame image includes: Perform point cloud stitching according to the point cloud coordinates corresponding to the frame image to obtain a point cloud image corresponding to the frame image; Statistical filtering is performed on the point cloud image corresponding to the frame image to obtain a dense point cloud image related to the object.

9. A machine room inspection device, applied to a visual robot, characterized in that: include: A first acquisition unit is used to collect environmental information in the computer room through a binocular camera to obtain a first disparity map; A first processing unit, configured to optimize the first disparity map to obtain a second disparity map; a second acquisition unit, configured to measure the distance of the object in the machine room based on the second disparity map to obtain the distance information of the object in the machine room; A second processing unit is used to start path search from the position of the visual robot and the target position at the same time according to the distance information of the object in the machine room, and generate a first obstacle avoidance path; The inspection unit is used to inspect the object at the target position according to the inspection instruction when the visual robot reaches the target position along the first obstacle avoidance path.

10. A visual robot, comprising a processor and a transceiver, wherein the transceiver receives and sends data under the control of the processor, characterized in that: The processor is configured to perform the following operations: Collecting the environment information in the computer room through a binocular camera to obtain a first disparity map; performing optimization processing on the first disparity map to obtain a second disparity map; Measuring the distance of the object in the machine room based on the second disparity map to obtain the distance information of the object in the machine room; According to the distance information of the object in the machine room, path search is started simultaneously from the position of the visual robot and the target position to generate a first obstacle avoidance path; When the visual robot reaches the target position along the first obstacle avoidance path, the object at the target position is inspected according to the inspection instruction.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the computer room inspection method according to any one of claims 1 to 8 are implemented.

12. A computer program product, characterized in that It comprises computer instructions, which, when executed by a processor, implement the steps in the computer room inspection method as claimed in any one of claims 1 to 8.

Images