Unmanned machine room inspection system based on machine vision

Abstract

The invention discloses an unmanned machine room inspection system based on machine vision, and relates to the technical field of unmanned machine room inspection, which comprises the following steps of: establishing a server panel indicator light feature library according to a fault library comprising light colors and change conditions of machine room server panel indicator lights; the server panel indicator lamp feature library is used for arranging all server panel graphic icons, indicator lamp colors and change condition information of a machine room, the fault solution is used for establishing a server panel problem knowledge base, a server fault problem solution knowledge base and a server panel knowledge introduction library, and the system provides a batch knowledge import function. And the image recognition module provides a data basis for fault problem retrieval and solution feedback, the image recognition module adopts a Hough circle detection algorithm to detect the circular contour of the server indicating lamp, the green light color is fault-free, and the red light color is abnormal.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned machine room inspection, and in particular to an unmanned machine room inspection system based on machine vision. Background Art

[0002] With the breakthrough development of technologies such as computers, machine vision and deep learning, and the use of big data, digital technology, the Internet of Things and 5G communication technology to upgrade server rooms, the market demand and application prospects of computer room inspections are constantly expanding. Considering that server systems are becoming more and more complex, and given the close and extensive use of server systems in various industries, server indicator lights are complex and diverse, with states of off, always on, flashing, and marked with different colors. The server failure status displayed by a single image information or several discontinuous images is unreliable because the system cannot distinguish between a flashing light and a faulty light. Applying machine vision technology to the field of server inspection allows computers to replace operation and maintenance personnel to understand various server panels, various icons on the running server panels, color changes and flashing states of indicator lights, and analyze the server health status displayed under the combined motion state of the server panel, so as to accurately locate the cause of the server failure and provide corresponding solutions.

[0003] The defects of the existing unmanned machine room inspection system are: 1. Patent document CN112325067A discloses an intelligent inspection system for an unmanned computer room, "including a truncated cone-shaped gravity seat, the lower end of the gravity seat is connected to a chassis, the upper end of the gravity seat is provided with a camera, the lower end of the camera is connected to a lifting column, the upper end of the camera is connected to a metal cap, and the lifting column is arranged in a circular plate groove opened on the top of the gravity seat. The present invention realizes the self-protection effect of a turtle in bionics, that is, if a person maliciously damages the camera, he will have to touch the arc rod, the arc rod will be offset, and the gas in the bellows will be discharged after the transmission triggers, so that the camera will directly retract into the gravity seat, and the gravity seat itself is large and too heavy to be carried, so the bad guys can only flee the computer room, and when the bad guys approach the camera, the camera will record and store the appearance information of the bad guys, and the security personnel can extract the information later", but the existing unmanned computer room inspection system cannot quickly and accurately identify server failures and provide solutions, and cannot push the failure information to relevant personnel in time, and cannot meet the requirements of unmanned inspection of the computer room; 2. Patent document CN117528020A discloses an intelligent bionic inspection system for an unmanned machine room, "including a bottom inspection component, an information collection component is fixedly installed above the bottom inspection component, a real-time monitoring component is arranged on the top of the information collection component, and the real-time monitoring component is adjusted by multiple adjustment components. The present invention can drive the square end plate to move by starting the control switch of the electric telescopic rod. Since the upper surface of the square end plate is movably connected to the side of the horizontal plate by the connecting diagonal rod, when the square end plate moves, the horizontal plate can be driven to rise and fall by the connecting diagonal rod. Since the horizontal plate is fixedly installed on the connecting bearing, the lifting of the horizontal plate can drive the connecting bearing, the polygonal sleeve and the real-time monitoring component to rise and fall as a whole, so that the monitoring height of the real-time monitoring camera can be adjusted, further improving the monitoring effect, so as to make the inspection effect better." However, the existing unmanned machine room inspection system sends periodic fault frame fragments to the frame processing module, and uses the VideoCapture class of OpenCV to operate the video frame; 3. Patent document CN115205997A discloses an unmanned inspection system and method for a HVAC room based on artificial intelligence, "including an inspection robot, a motion unit, a room inspection unit, a fault diagnosis unit and a fault alarm unit; the motion unit is used to realize the automatic driving of the inspection robot; the room inspection unit is used to detect the environmental information and equipment operation status in the HVAC room, and send it to the fault diagnosis unit; the fault diagnosis unit is used to receive the environmental information and equipment operation status detected by the room inspection unit, and compare the received environmental information with the preset environmental conditions, compare the received equipment operation status with the preset equipment operation status, and judge whether there is a fault in the HVAC room according to the comparison result. If there is a fault, the fault information is obtained and sent to the fault alarm unit; the fault alarm unit is used to alarm the fault information. The present invention can realize unmanned automatic inspection, reduce the workload of personnel, and improve the efficiency and accuracy of inspection work", but the efficiency of the existing unmanned room inspection system cannot be maximized, and early warning information cannot be issued immediately. Problems cannot be discovered and handled in time, increasing the potential risk of downtime; 4. Patent document CN116494201A discloses an integrated monitoring power room inspection robot and an unmanned inspection method. "It is used to conduct regular inspections of power rooms and promptly deal with specific abnormal conditions. The power room inspection robot includes: a mobile platform, a robotic arm system, a depth camera system, and a host computer. Among them, the mobile platform includes a positioning module and is equipped with at least one laser radar. The robotic arm system includes a lower computer, a robotic arm and an end actuator. The upper computer is electrically connected to the mobile platform, the lower computer of the robotic arm system, and the depth camera system. The upper computer runs a target recognition model, a target positioning model, and a space virtualization module. A navigation module based on the Navigation function package and a robotic arm trajectory planning module. The present invention solves the problems of low automation of existing power inspection robots, high difficulty in automating power inspection robots, and high implementation costs." However, the existing unmanned machine room inspection system cannot denoise and grayscale the original image, which reduces detection accuracy. Summary of the invention

[0004] The purpose of the present invention is to provide an unmanned computer room inspection system based on machine vision to solve the technical problems raised in the above background technology that server failures cannot be quickly and accurately identified and solutions cannot be provided, and fault information cannot be pushed to relevant personnel in a timely manner, and the requirements for unmanned inspection of the computer room cannot be met.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an unmanned machine room inspection system based on machine vision, comprising a fault library, a fault solution, a model verification module, a data acquisition module, a video analysis module, a frame image processing module, an image recognition module, an indicator light color feature extraction module, a diagnosis module, a solution retrieval module, and a terminal feedback module, wherein the fault library is written with a fault solution, and the fault library, the model verification module, the data acquisition module, the video analysis module, the frame image processing module, the image recognition module, the indicator light color feature extraction module, the diagnosis module, the solution retrieval module, and the terminal feedback module are electrically connected; The fault library includes the light color and change status of the indicator lights on the server panels in the computer room. A server panel indicator light feature library is established. The server panel indicator light feature library organizes all the graphic icons of the server panels in the computer room and the indicator light colors and change status information. The fault solution is to establish a server panel problem knowledge base, a server fault problem solution knowledge base and a server panel knowledge introduction base. The system provides a batch knowledge import function to provide a data basis for fault problem retrieval and feedback solutions. The image recognition module uses the Hough circle detection algorithm to detect the circular contour of the server indicator light. The green light color indicates no fault, and the red light color indicates an abnormality.

[0006] Preferably, the model verification module verifies the accuracy and coverage of the model through an image similarity algorithm, and calibrates the model by fine-tuning the model parameters to seek a suitable threshold for determining whether the image has been successfully matched, and the suitable threshold is 98% to 100%.

[0007] Preferably, the data acquisition module uses a CMOS camera to complete the collection of image information, and the model DS-2CD3146FWD-I Hikvision network camera is selected. This camera also supports real-time image transmission through the network IP address. The video is collected by the camera, and the collected video length is 3S~4S, and the video information is submitted to the video analysis module in real time.

[0008] Preferably, the operating status of the server is mainly identified based on the operating status of its panel indicator light. Different colors and states of the panel indicator light are used to indicate different working states of the server. The video analysis module identifies server faults based on the collected video information of the server panel indicator light. The collected video information of the server indicator light is analyzed through the Hough circle detection algorithm and the HSV color feature to determine the working status of the server panel indicator light, thereby completing the identification of the server fault.

[0009] Preferably, the frame image processing module analyzes the characteristics of the server panel graphics, icons, indicator light status and color for each frame of the video captured by the camera, compares the graphic icon indicator light status and color in the fault feature library, marks the fault characteristics of each frame, and feeds back the fault feature identification results of the video clip frame to the terminal feedback module.

[0010] Preferably, the indicator light color feature extraction module describes the features of the color panel indicator light in the HSV color space. In the HSV color space, the RGB image is first converted into the HSV color space, and then the non-color is divided according to the S and V components, and then the color is divided according to the H component, and finally the color classification result is output.

[0011] Preferably, the diagnostic module analyzes the key frame list, removes repeated frame features, retains the difference diagnosis between subsequent frames and the first frame, uses the differences of the same sub-components as the final diagnosis basis, matches them with the server fault feature library, and obtains a comprehensive diagnostic result.

[0012] Preferably, if the diagnosis result of the solution retrieval module is that there is no fault, the server panel introduction information is retrieved. If the diagnosis result is abnormal, the corresponding problem solution is retrieved in the server panel knowledge base based on the problem fed back by the comprehensive diagnosis result.

[0013] Preferably, the diagnosis result is fed back to the terminal feedback module in the form of a feature image with analysis and explanation content.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention is equipped with a device to collect the state image of the server fault indicator light, uses Hough transform and machine vision to detect the fault state of the server indicator light, establishes a server panel problem knowledge base, establishes a server fault problem solution knowledge base and a server panel knowledge introduction base, realizes fault detection and provides solutions through fault matching, realizes accurate positioning of the coordinates of the server indicator light and accurate identification of the color and provides solutions, and matches the server panel problem knowledge base, the server fault problem solution knowledge base and the server panel knowledge introduction base, so as to realize rapid and accurate identification of server faults and provide solutions, and push the fault information to relevant personnel in a timely manner, meeting the requirements of unmanned inspection of the computer room; 2. The present invention is installed with a device for reading the number of video frames and the length of the video, and the first frame is set as the key frame by default. By traversing each frame, the corresponding playback position of the video is obtained, thereby obtaining the screen image corresponding to each frame, and each frame image is matched for similarity between images. When the matched similar frame and the current frame are continuous frames, the continuous frames are counted. When the continuous frame count exceeds N, the value of N is a multiple of the frame rate. The frame rate is the number of pictures played per second by the video. The current setting of N is one times the frame rate and can be adjusted. It is considered that the current screen indicator light is in a single state, and the video is stopped from being parsed, and the key frame screen is returned; when the continuous frame similarity state changes, the current frame is counted as a key frame, and the next frame is read to continue matching until the frame similarity changes again and is consistent with the first key frame similarity, and the periodic fault frame fragments are sent to the frame processing module, and the VideoCapture class of OpenCV is used to implement the operation of the video frame; 3. The present invention is equipped with an image segmentation technology to first divide the whole into various sub-target blocks according to the regional technology; the sub-target blocks realize the identification of indicator lights, indicator light colors and brightness through the Imgproc class provided by OpenCV, which can complete large-scale data collection work in a short time, monitor and inspect the computer room 24 / 7 uninterruptedly, maximize efficiency, and issue early warning information immediately, so that problems can be discovered and handled in time, thereby reducing potential downtime risks; 4. The present invention is equipped with a device that collects video information that will be affected by camera vibration, on-site lighting environment, dust on the lens and other factors. When the inspection assistant collects the image data of the indicator light on the server panel, noise points that affect the image quality will be present in the collected indicator light video data. This will affect the accuracy of the panel indicator light position detection, shape detection and color recognition. Considering the speed at which the processor processes the image data, it is necessary to denoise and grayscale the original image to improve the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] In the description of the present invention, it should be noted that the terms "upper", "lower", "inner", "outer", "front end", "rear end", "two ends", "one end", "the other end" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0018] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, it can be understood according to the specific situation.

[0019] Example 1: Please refer to Figure 1The present invention provides an embodiment: an unmanned computer room inspection system based on machine vision, including a fault library, a fault solution, a model verification module, a data acquisition module, a video analysis module, a frame image processing module, an image recognition module, an indicator light color feature extraction module, a diagnosis module, a solution retrieval module, and a terminal feedback module. The fault solution is written into the fault library. The fault library, the model verification module, the data acquisition module, the video analysis module, the frame image processing module, the image recognition module, the indicator light color feature extraction module, the diagnosis module, the solution retrieval module, and the terminal feedback module are electrically connected. The fault library contains the light color and change of the indicator light on the server panel of the computer room. A server panel indicator light feature library is established. The server panel indicator light feature library is for organizing all server panel graphic icons and indicator light color and change information in the computer room. The fault solution is to establish a server panel problem knowledge base, and a server fault problem solution knowledge base and a server panel knowledge base are established. The system provides a batch knowledge import function to provide a data basis for fault problem retrieval and feedback solutions. The image recognition module uses the Hough circle detection algorithm to detect the circular outline of the server indicator light. The green light color indicates no fault, and the red light color indicates an abnormality. The image of the server fault indicator light status is collected. The Hough transform and machine vision are used to detect the fault status of the server indicator light. A server panel problem knowledge base, a server fault problem solution knowledge base and a server panel knowledge introduction base are established. Fault detection is achieved through fault matching and solutions are given. Accurate positioning of the server indicator light coordinates and accurate color recognition are achieved and solutions are given. Matching of the server panel problem knowledge base, the server fault problem solution knowledge base and the server panel knowledge introduction base can realize rapid and accurate identification of server faults and provide solutions, and push fault information to relevant personnel in a timely manner to meet the requirements of unmanned inspections in the computer room.

[0020] Example 2: Please refer to Figure 1, an embodiment provided by the present invention: : the model verification module verifies the accuracy and coverage of the model through an image similarity algorithm, fine-tunes the model parameters, calibrates the model, and seeks a suitable threshold value for judging whether the image has been successfully matched, and the suitable threshold value is 98%~100%. The data acquisition module uses a CMOS camera to complete the acquisition of image information, and selects the model DS-2CD3146FWD-I Hikvision network camera. This camera also supports real-time image transmission via network IP address, and collects video through the camera. The length of the collected video is 3S~4S, and the video information is submitted to the video analysis module in real time. The running status of the server is mainly identified according to the running status of its panel indicator light. The different colors and states of the panel indicator light are used to indicate different working states of the server. The video analysis module identifies the server fault according to the collected video information of the server panel indicator light, and the collected server indicator light video information is detected by the Hough circle detection algorithm and the HSV color feature. Analyze the working status of the indicator light on the server panel, complete the identification of server faults, read the number of video frames and the length of the video, and set the first frame as the key frame by default. By traversing each frame, the corresponding playback position of the video is obtained, so as to obtain the screen image corresponding to each frame. Each frame image is matched for similarity. When the matched similar frame is a continuous frame with the current frame, the continuous frames are counted. When the continuous frame count exceeds N, the value of N is a multiple of the frame rate, and the frame rate is the number of pictures played per second by the video. The current setting of N is one times the frame rate and can be adjusted. It is considered that the current screen indicator light is in a single state, stop parsing the video, and return to the key frame screen; when the continuous frame similarity state changes, the current frame is counted as a key frame, and the next frame is read to continue matching until the frame similarity changes again and is consistent with the first key frame similarity. The periodic fault frame fragment is sent to the frame processing module, and the VideoCapture class of OpenCV is used to operate the video frame.

[0021] Example 3: Please refer to Figure 1, an embodiment provided by the present invention: a frame image processing module analyzes the characteristics of server panel graphics, icons, indicator light status and color for each frame of video captured by a camera, compares the graphics icon indicator light status and color in a fault feature library, marks the fault characteristics of each frame, and feeds back the fault feature recognition result of the video clip frame to a terminal feedback module; an indicator light color feature extraction module describes the characteristics of the color panel indicator light in the HSV color space, first converts the RGB image into the HSV color space in the HSV color space, then divides the non-color by using the S and V components, and then divides the color by the H component, and finally outputs the color classification result, and uses the image segmentation technology to first divide the whole into various sub-target blocks according to the regional technology; the sub-target blocks use the Imgproc class provided by OpenCV to realize the identification of indicator lights, indicator light colors and brightness, can complete large-scale data collection work in a short time, monitor and inspect the computer room 24 / 7 uninterruptedly, achieve maximum efficiency, and can immediately issue warning information so that problems can be discovered and handled in time, thereby reducing potential downtime risks.

[0022] Example 4: Please refer to Figure 1 , an embodiment provided by the present invention: the diagnosis module analyzes the key frame list, removes repeated frame features, retains the difference diagnosis between the subsequent frames and the first frame, takes the difference of the same sub-component as the final diagnosis basis, matches it with the server fault feature library, and obtains a comprehensive diagnosis result. If the diagnosis result of the solution retrieval module is that there is no fault, the server panel introduction information is retrieved. If there is an abnormality in the diagnosis result, the corresponding problem solution is retrieved in the server panel knowledge base for the problem fed back by the comprehensive diagnosis result. The diagnosis result is fed back to the terminal feedback module in the form of a feature picture with analysis and description content. The collected video information will be interfered by factors such as camera vibration, on-site lighting environment and dust on the lens. When the inspection assistant collects the image data of the server panel indicator light, the collected indicator light video data will have noise points that affect the image quality, which will affect the accuracy of the panel indicator light position detection, shape detection and color recognition. At the same time, considering the speed of the processor processing image data, it is necessary to denoise and grayscale the original image to improve the detection accuracy.

[0023] Fault identification rate: 95% of abnormal situations are discovered and solved in the early stages and reported.

[0024] Fault response time: The average time from fault discovery to fault handling is shortened to 30 minutes.

[0025] Operation and maintenance costs: After the introduction of the computer room inspection assistant, labor costs were saved by 40% and operation and maintenance costs were reduced by 25%.

[0026] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. An unmanned machine room inspection system based on machine vision, characterized by: It includes a fault library, a fault solution, a model verification module, a data acquisition module, a video analysis module, a frame image processing module, an image recognition module, an indicator light color feature extraction module, a diagnosis module, a solution retrieval module, and a terminal feedback module. The fault library is written with a fault solution, and the fault library, the model verification module, the data acquisition module, the video analysis module, the frame image processing module, the image recognition module, the indicator light color feature extraction module, the diagnosis module, the solution retrieval module, and the terminal feedback module are electrically connected; The fault library includes the light color and change status of the indicator lights on the server panels in the computer room. A server panel indicator light feature library is established. The server panel indicator light feature library organizes all the graphic icons of the server panels in the computer room and the indicator light colors and change status information. The fault solution is to establish a server panel problem knowledge base, a server fault problem solution knowledge base and a server panel knowledge introduction base. The system provides a batch knowledge import function to provide a data basis for fault problem retrieval and feedback solutions. The image recognition module uses the Hough circle detection algorithm to detect the circular contour of the server indicator light. The green light color indicates no fault, and the red light color indicates an abnormality.

2. The unmanned machine room inspection system based on machine vision according to claim 1 is characterized in that: The model verification module verifies the accuracy and coverage of the model through an image similarity algorithm, and fine-tunes the model parameters, calibrates the model, and seeks a suitable threshold for determining whether the image has been successfully matched. The suitable threshold is 98% to 100%.

3. The unmanned machine room inspection system based on machine vision according to claim 2 is characterized in that: The data acquisition module uses a CMOS camera to complete the collection of image information. The model selected is the DS-2CD3146FWD-I Hikvision network camera. This camera also supports real-time image transmission via the network IP address. The video is collected through the camera, and the collected video length is 3S~4S. The video information is submitted to the video analysis module in real time.

4. The unmanned machine room inspection system based on machine vision according to claim 1 is characterized in that: The operating status of the server is mainly identified according to the operating status of its panel indicator light. The different colors and states of the panel indicator light are used to indicate different working states of the server. The video analysis module identifies the server fault according to the collected video information of the server panel indicator light. The collected video information of the server indicator light is analyzed through the Hough circle detection algorithm and the HSV color feature to identify the working status of the server panel indicator light, thereby completing the identification of the server fault.

5. The unmanned machine room inspection system based on machine vision according to claim 3 is characterized by: The frame image processing module analyzes the characteristics of the server panel graphics, icons, indicator light status and color for each frame of the video captured by the camera, compares the graphic icon indicator light status and color in the fault feature library, marks the fault characteristics of each frame, and feeds back the fault feature identification results of the video clip frame to the terminal feedback module.

6. The unmanned machine room inspection system based on machine vision according to claim 1 is characterized in that: The indicator light color feature extraction module describes the features of the color panel indicator light in the HSV color space. In the HSV color space, the RGB image is first converted to the HSV color space, and then the non-color is divided according to the S and V components, and the color is divided according to the H component, and finally the color classification result is output.

7. The unmanned machine room inspection system based on machine vision according to claim 1 is characterized by: The diagnosis module analyzes the key frame list, removes repeated frame features, retains the difference diagnosis between subsequent frames and the first frame, uses the difference of the same sub-components as the final diagnosis basis, matches it with the server fault feature library, and obtains a comprehensive diagnosis result.

8. The unmanned machine room inspection system based on machine vision according to claim 1 is characterized in that: If the diagnosis result of the solution retrieval module is that there is no fault, the server panel introduction information is retrieved. If the diagnosis result is abnormal, the corresponding problem solution is retrieved in the server panel knowledge base based on the problem fed back by the comprehensive diagnosis result.

9. The unmanned machine room inspection system based on machine vision according to claim 8 is characterized in that: The diagnosis result is fed back to the terminal feedback module in the form of a feature image with analysis and explanation content.

Images