Machine vision-based intelligent inspection method for machine room equipment

Abstract

The present application relates to the field of machine room equipment inspection, in particular to a machine room equipment intelligent inspection method based on machine vision, a machine room camera collects equipment images, obtains an equipment edge detection image, and obtains a guide image according to the equipment image and the equipment edge detection image; the change direction of each pixel point is obtained according to the Hessian matrix of each pixel point; the structure factor of each pixel point is obtained according to the gray distribution condition in the change direction of each pixel point; the guide filter adaptive window size of each pixel point is calculated, and then the guide filter adaptive optimization window size of each pixel point is calculated; a high-quality equipment image is obtained by combining the guide filter algorithm; the equipment state is detected through a neural network according to the high-quality equipment image, and the machine room equipment intelligent inspection is completed. Thus, the intelligent inspection of the machine room equipment is realized, and the inspection precision is high.

Description

Technical Field

[0001] This application relates to the field of data center equipment inspection, specifically to a machine vision-based intelligent inspection method for data center equipment. Background Technology

[0002] A data center is a crucial component, containing numerous servers, hardware devices, and other equipment. It stores vast amounts of information systems and critical data. The equipment in a data center is diverse, operating independently yet interconnected. As a big data processing center, not only is the data itself vital, but the equipment itself is also highly valuable. Therefore, proper inspection and maintenance of the data center equipment is paramount. To ensure the normal operation of the central data center, regular inspections are necessary to maintain stable equipment operation, extend equipment lifespan, and reduce the likelihood of equipment failure.

[0003] Currently, relying on manual, scheduled inspections of computer room equipment requires a significant investment of manpower and often results in missed inspections, making it difficult to promptly detect and address equipment malfunctions. Existing mobile inspection vehicles with cameras suffer from the following problems: The fixed cameras can only identify equipment models, indicator lights, and changes in displayed data status through oblique viewing angles. This can lead to blurred images or incomplete data capture, resulting in incomplete inspections and low accuracy.

[0004] In summary, this invention proposes a machine vision-based intelligent inspection method for computer room equipment. Several cameras are deployed in the computer room to acquire images of the equipment, the images are optimized to improve their quality, and the processed images are fed into a neural network to detect the equipment's operating status, thereby achieving intelligent inspection of the computer room equipment. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a machine vision-based intelligent inspection method for computer room equipment, thereby resolving existing issues.

[0006] The intelligent inspection method for computer room equipment based on machine vision of the present invention adopts the following technical solution:

[0007] One embodiment of the present invention provides a machine vision-based intelligent inspection method for data center equipment, the method comprising the following steps:

[0008] Acquire equipment images using cameras in the server room;

[0009] Obtain the device edge detection map from the device image, and then obtain the guide image based on the device image and the device edge detection map.

[0010] The guided filtering linear model is obtained based on the guided filtering algorithm. The linear parameters of the guided filtering linear model are obtained by combining the guided filtering linear model with linear regression. The Hessian matrix of each pixel in the device image is obtained. The direction of change of each pixel is obtained based on the Hessian matrix of each pixel. The structure factor of each pixel is obtained based on the gray-level distribution of each pixel in the direction of change. The guided filtering adaptive window size of each pixel is obtained based on the structure factor of each pixel.

[0011] The adaptive optimized window size for the guided filter is obtained for each pixel based on the adaptive window size and structure factor of the guided filter for each pixel.

[0012] The device image after adaptive guidance filtering is obtained by combining the guidance image, adaptively optimized window size, linear coefficients and guidance filtering algorithm, and is denoted as high-quality device image;

[0013] The system uses high-quality equipment images combined with neural networks to obtain equipment status and completes intelligent inspection of equipment in the computer room.

[0014] Preferably, the guide image obtained based on the device image and the device edge detection image is expressed as follows:

[0015]

[0016] In the formula, For guiding images, For device images, This is a map showing the edge detection of the device. This is the recombination coefficient.

[0017] Preferably, the guided filter linear model obtained according to the guided filter algorithm is expressed as follows:

[0018]

[0019] In the formula, For a rectangular window with size w centered at pixel k, These are the linear coefficients of the guided filter linear model. To guide the gray value of pixel i in the image, This is the filtered output value for pixel i.

[0020] Preferably, the linear parameters of the guided filter linear model are obtained by combining the guided filter linear model with linear regression, and the expression is:

[0021]

[0022]

[0023] In the formula, For a rectangular window with size w centered at pixel k, These are the linear coefficients of the guided filter linear model, and M is the total number of pixels within the window. Guide images In the window Mean and variance of grayscale values ​​of mid-pixel For device image I in window The average grayscale value of the middle pixel. These represent the grayscale values ​​of pixel i in the guide image and the device image, respectively. These are the normalization parameters.

[0024] Preferably, the Hessian matrix of each pixel in the obtained device image is expressed as:

[0025]

[0026] In the formula, Let i be the Hessian matrix of pixel i in the device image. Let be the second-order partial derivatives of pixel i in the device image in the x and y directions, respectively. and are the mixed partial derivatives of pixel i in the device image in the xy and yx directions, respectively, where .

[0027] Preferably, obtaining the direction of change of each pixel based on the Hessian matrix of each pixel specifically involves: calculating the eigenvalues ​​and eigenvectors of the Hessian matrix, and taking the eigenvector corresponding to the eigenvalue with the larger absolute value as the direction of change of the pixel.

[0028] Preferably, the structure factor of each pixel is obtained based on the grayscale distribution of pixels along the direction of pixel change, and the expression is:

[0029]

[0030]

[0031] In the formula, For user-defined functions, where , For the threshold, The structure factor of pixel s These represent the number of pixels selected from the left and right sides of pixel s along the direction of pixel s' change. Let u be the grayscale value of pixel u. Let v be the grayscale value of pixel v. Let be the grayscale value of pixel s.

[0032] Preferably, the adaptive window size for the guided filtering of each pixel is obtained based on the structure factor of each pixel, and the expression is:

[0033]

[0034] In the formula, Let be the adaptive window size for the guided filter at pixel s, which is also the side length of the guided filter window centered at pixel s. To Rounding function, Let be the normalized structure factor of pixel s. Parameters that define the window size.

[0035] Preferably, the adaptive optimized window size for the guided filter of each pixel is obtained based on the adaptive window size and structure factor of the guided filter for each pixel, and the expression is:

[0036]

[0037] In the formula, The window size is adaptively optimized for guided filtering of pixel s. The guided filter adaptive window size for pixel s. is the normalized structure factor for pixel s.

[0038] Preferably, the step of obtaining the device status based on the high-quality device image combined with a neural network specifically involves: the neural network being a target detection network, the network input being the high-quality device image, and the network output being the device bounding box category in the high-quality device image, where the device bounding box category includes: (Normal) and (Abnormal) The device bounding box category is used as the device state.

[0039] The present invention has at least the following beneficial effects:

[0040] This invention utilizes machine vision combined with the visual characteristics of data center equipment to detect the operational status of data center equipment, enabling intelligent inspection of various devices within the data center. By applying adaptive guided filtering to the equipment images, it solves the problem of noise affecting image quality during image acquisition, avoids issues such as unclear equipment in images, improves the accuracy of equipment information analysis, increases the reliability of equipment information, and thus improves the detection accuracy of data center equipment operational status.

[0041] To avoid the problem of poor image filtering effect caused by fixed windows in traditional guided filtering, this invention combines the analysis of structural factors of pixels in equipment images to construct a pixel-guided filtering adaptive window size calculation model, and optimizes the pixel-guided filtering adaptive window size to perform sliding filtering on pixels of equipment images. It can adaptively set the filtering window according to the local structural features of pixels, and can perform targeted filtering on pixels, improving the image processing effect and achieving higher equipment inspection accuracy. Attached Figure Description

[0042] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 The flowchart shows the intelligent inspection method for computer room equipment based on machine vision provided by this invention. Detailed Implementation

[0044] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the intelligent inspection method for data center equipment based on machine vision proposed by the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0046] The following description, in conjunction with the accompanying drawings, details the specific scheme of the intelligent inspection method for computer room equipment based on machine vision provided by this invention.

[0047] An embodiment of the present invention provides a machine vision-based intelligent inspection method for computer room equipment.

[0048] Specifically, the following machine vision-based intelligent inspection method for data center equipment is provided; please refer to [link / reference]. Figure 1 The method includes the following steps:

[0049] Step S001: The computer room camera captures images of the computer room equipment to obtain images of each device.

[0050] This embodiment mainly uses visual features to detect the status of equipment in the computer room, realizing the intelligent inspection process of the computer room equipment. Therefore, several cameras are deployed in the computer room to collect images of the equipment in the computer room. In order to ensure the integrity of each device image and avoid problems such as poor shooting angle, the implementer can install multiple cameras on the top and side walls of the computer room, and set the angle and shooting range of the cameras to ensure that each device in the computer room can be completely captured. The specific number of cameras, shooting range, angle and other deployments can be set by the implementer according to the actual situation.

[0051] At this point, images of the equipment in the computer room can be captured by the cameras in the room, and images of each device can be obtained for detection and analysis of the equipment status.

[0052] Step S002: Establish a device image processing model, optimize the device image, and obtain a high-quality device image.

[0053] Due to the inherent structural limitations of the camera sensor, during image acquisition, internal semiconductor devices generate discontinuous shot noise and low-frequency noise, resulting in significant noise in the acquired equipment images. Furthermore, the complex and enclosed environment of the computer room, coupled with unstable temperatures, makes the camera highly susceptible to external environmental influences, further contributing to noise in the acquired equipment images and affecting the detection of equipment operating status. Therefore, this embodiment optimizes the equipment images to improve image quality and prevent noise from affecting equipment information.

[0054] Traditional denoising processes mostly filter the entire image, which is computationally intensive and can lead to over-smoothing of non-noise pixels, resulting in low data accuracy. Therefore, for device images, this embodiment establishes a device image processing model to analyze the image and detect and extract noise, adaptively filtering out noise and avoiding its impact on device condition detection. The specific process of the image processing model is as follows:

[0055] Traditional filtering and denoising algorithms for device images are prone to problems such as loss of edge detail texture and excessive edge smoothing. This embodiment will use guided filtering to perform adaptive filtering on the device image. Guided filtering has the characteristic of preserving edge information and can retain the edge detail texture information in the device image. At the same time, the adaptive filtering process set in this embodiment can effectively filter noise in the device image, thereby improving image quality.

[0056] First, to improve filtering accuracy while preserving detailed texture information in the device image, this embodiment uses the Sobel operator to perform edge detection on the device image, obtaining a device edge detection map. It should be noted that the Sobel operator's edge detection process is a well-known existing technique and will not be described in detail in this embodiment. Based on the device image and the device edge detection map, the guiding image for the guiding filtering process is obtained:

[0057]

[0058] In the formula, For guiding images, For device images, This is a map showing the edge detection of the device. The recombination coefficient can be set by the implementer; in this embodiment, it is set to... ;

[0059] The guided filter linear model, derived from the guided filter algorithm, is expressed as follows:

[0060]

[0061] In the formula, For a rectangular window with size w centered at pixel k, These are the linear coefficients of the guided filter linear model. To guide the gray value of pixel i in the image, This is the filtered output value of pixel i;

[0062] The linear coefficients in the guided filter linear model can be obtained based on the guided filter linear model and the linear regression algorithm. The expressions are as follows:

[0063]

[0064]

[0065] In the formula, For a rectangular window with size w centered at pixel k, These are the linear coefficients of the guided filter linear model, and M is the total number of pixels within the window. Guide images In the window Mean and variance of grayscale values ​​of mid-pixel For device image I in window The average grayscale value of the middle pixel. These represent the grayscale values ​​of pixel i in the guide image and the device image, respectively. As a normalization parameter, the implementer can choose its own values; in this embodiment, it is set to... It should be noted that the linear parameter The specific calculation process is a well-known existing technology and will not be described in detail in this embodiment.

[0066] Traditionally, the window size in the guided filtering process is mostly set manually to a fixed value. When there are different texture regions in the image, if the guided filtering window size is set to be too large or too small, it will cause problems such as the edge regions in the device image being too blurry or the denoising effect being poor. Therefore, this embodiment will adaptively set the window size in the guided filtering process of the device image in order to achieve adaptive filtering processing of the device image, improve the filtering and denoising effect while preserving the edge detail texture information in the device image.

[0067] For smooth regions, where grayscale changes are not drastic, the guided filter window size can be appropriately enlarged to improve the filtering effect. For pixels at edge detail textures, i.e., edge regions, considering that pixel grayscale changes are more significant and the direction of change is also variable, setting the window too large will result in a high smoothing effect for edge detail texture pixels, leading to blurring. Therefore, the guided filter window size needs to be appropriately reduced. This embodiment will adaptively set the guided filter window size based on pixel characteristics. For each pixel in the device image, the Hessian matrix of each pixel is obtained, expressed as:

[0068]

[0069] In the formula, Let i be the Hessian matrix of pixel i in the device image. Let be the second-order partial derivatives of pixel i in the device image in the x and y directions, respectively. and are the mixed partial derivatives of pixel i in the device image in the xy and yx directions, respectively, where It should be noted that the specific calculation process of the pixel Hessian matrix is ​​a known prior art and is not within the scope of protection of this embodiment. Then, the eigenvalues ​​and eigenvectors of the Hessian matrix are calculated. The magnitude of the eigenvalues ​​of the pixel Hessian matrix is ​​used to characterize the degree of grayscale change of the pixel in the direction of the corresponding eigenvector. The eigenvector corresponding to the eigenvalue with the larger absolute value is taken as the direction of change of the pixel. It should be noted that the calculation process of the eigenvalues ​​and eigenvectors of the pixel Hessian matrix is ​​a known prior art and is not within the scope of protection of this embodiment; therefore, it will not be described in detail in this embodiment. Several pixels are selected from the left and right sides of the pixel along the direction of change. The structure factor of the pixel is calculated based on the grayscale distribution of the local neighborhood pixels on both sides of the direction of change. The specific expression for the structure factor of the pixel is:

[0070]

[0071]

[0072] In the formula, For user-defined functions, where , The threshold value can be selected by the implementer; in this embodiment, it is set to 10. These represent the number of pixels selected from the left and right sides of pixel s along the direction of pixel s' change. Let u be the grayscale value of pixel u. Let v be the grayscale value of pixel v. Let be the grayscale value of pixel s. The structure factor of pixel s is larger. The larger the structure factor, the more obvious the gray level difference between the two sides of the pixel change direction, the greater the gray level difference between the pixel and the pixels on both sides of the change direction, and the more drastic the local structure change of the pixel.

[0073] Repeat the above method to obtain the structure factor of each pixel in the device image, and normalize the structure factor to ensure that the structure factor is within (0,1). Further, adaptively set the guided filtering window for each pixel based on its structure factor. The adaptive window size for each pixel's guided filtering is obtained according to its structure factor. The specific expression for the adaptive window size is as follows:

[0074]

[0075] In the formula, Let be the adaptive window size for the guided filter at pixel s, which is also the side length of the guided filter window centered at pixel s. To Rounding function, Let be the normalized structure factor of pixel s, where is the normalized structure factor of pixel s. The implementer can set the value of the window size limitation parameter. In this embodiment, it will be set as follows: , ;

[0076] It should be noted that in this embodiment, the size of the pixel-guided filtering adaptive window is guaranteed to be an odd number to facilitate sliding filtering of the pixels in the device image. Therefore, the size of the pixel-guided filtering adaptive window will be optimized. The optimized adaptive window size for each pixel is obtained based on the adaptive window size of each pixel and the structure factor. The specific expression for optimizing the pixel-guided filtering adaptive window size is as follows:

[0077]

[0078] In the formula, The window size is adaptively optimized for guided filtering of pixel s. The guided filter adaptive window size for pixel s. is the normalized structure factor for pixel s.

[0079] Repeat the above method to obtain the adaptive optimization window size for the guided filter for each pixel in the device image;

[0080] This allows for adaptive setting of the window size during the guided filtering process of the device image. Based on the adaptively optimized window size and the guided filtering algorithm, the device image is filtered. By adaptively optimizing the filtering window size of the pixels in the device image, adaptive smoothing and noise reduction of the device image can be achieved, while preserving the edge detail texture information in the device image and avoiding the loss of detail information in the device image.

[0081] Thus, based on the device image processing model described above in this embodiment, the device image after adaptive guided filtering can be obtained, denoted as a high-quality device image. A high-quality device image can improve the accuracy of device information analysis, increase the reliability of device information, avoid the influence of noise, and accurately represent information such as the status of the device.

[0082] Step S003: Based on high-quality equipment images and combined with neural networks, the equipment status is detected to achieve intelligent inspection of equipment in the computer room.

[0083] High-quality equipment images can accurately represent equipment status and provide reliable information for equipment operation status detection. Based on these high-quality images, this embodiment will combine a neural network model to detect equipment operation status. The neural network is a target detection neural network that performs target detection on the equipment status lights in the high-quality equipment images, obtaining the bounding box information of the equipment status lights. During network training, the bounding box information of the equipment status lights is manually labeled. The bounding box information of the equipment status lights mainly includes: , Let w and h be the coordinates of the center point of the device status light bounding box, respectively. This refers to the device status light bounding box category. It should be noted that the bounding box category represents the operating status of the device within the bounding box. Device bounding box categories include: (Normal) and (Abnormal) Implementers can also set the device bounding box category as needed. The network input is a high-quality device image, and the network output is the device bounding box information in the high-quality device image. The loss function for network training is the mean squared error loss function. The object detection neural network structure and network training process are existing well-known technologies.

[0084] The system obtains the operating status of the equipment based on the equipment bounding box information. When the equipment is in an abnormal operating state, it promptly issues a buzzer warning to remind relevant management personnel to inspect and replace the equipment in the computer room as soon as possible, so as to prevent serious dangerous accidents caused by prolonged abnormal operation of the equipment.

[0085] In summary, this invention combines machine vision with the visual characteristics of data center equipment to detect the operational status of data center equipment, achieving intelligent inspection of various devices within the data center. By performing adaptive guided filtering on the equipment images, the impact of noise on image quality during image acquisition is resolved, avoiding problems such as unclear equipment in images, improving the accuracy of equipment information analysis, increasing the reliability of equipment information, and thus improving the detection accuracy of data center equipment operational status.

[0086] To avoid the problem of poor image filtering effect caused by fixed windows in traditional guided filtering, this embodiment of the invention combines the analysis of structural factors of pixels in the device image to construct a pixel-guided filtering adaptive window size calculation model, and optimizes the pixel-guided filtering adaptive window size to perform sliding filtering on the pixels of the device image. It can adaptively set the filtering window according to the local structural features of the pixels, and can perform targeted filtering on pixels, improve the image processing effect, and have high equipment inspection accuracy.

[0087] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0088] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0089] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them; modifications to the technical solutions described in the foregoing embodiments, or equivalent substitutions of some of the technical features, do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A machine vision-based intelligent inspection method for computer room equipment, characterized in that, The method includes the following steps: Acquire equipment images using cameras in the server room; Obtain the device edge detection map from the device image, and then obtain the guide image based on the device image and the device edge detection map. The guided filtering linear model is obtained based on the guided filtering algorithm. The linear parameters of the guided filtering linear model are obtained by combining the guided filtering linear model with linear regression. The Hessian matrix of each pixel in the device image is obtained. The direction of change of each pixel is obtained based on the Hessian matrix of each pixel. The structure factor of each pixel is obtained based on the gray-level distribution of each pixel in the direction of change. The guided filtering adaptive window size of each pixel is obtained based on the structure factor of each pixel. The adaptive optimized window size for the guided filter is obtained for each pixel based on the adaptive window size and structure factor of the guided filter for each pixel. The device image after adaptive guidance filtering is obtained by combining the guidance image, adaptively optimized window size, linear coefficients and guidance filtering algorithm, and is denoted as high-quality device image; The system obtains equipment status by combining high-quality equipment images with neural networks, and completes intelligent inspection of equipment in the computer room. The structure factor of each pixel is obtained based on the grayscale distribution of pixels along the direction of pixel change, and the expression is: In the formula, For user-defined functions, where , For the threshold, The structure factor of pixel s These represent the number of pixels selected from the left and right sides of pixel s along the direction of pixel s' change. Let u be the grayscale value of pixel u. Let v be the grayscale value of pixel v. Let be the grayscale value of pixel s; The adaptive window size for the guided filtering of each pixel is obtained based on the structure factor of each pixel, and the expression is as follows: In the formula, Let be the adaptive window size for the guided filter at pixel s, which is also the side length of the guided filter window centered at pixel s. To Rounding function Let be the normalized structure factor of pixel s. Define parameters for window size; The adaptive optimized window size for the guided filter is obtained for each pixel based on the adaptive window size and structure factor of the guided filter. The expression is as follows: In the formula, The window size is adaptively optimized for guided filtering of pixel s. The guided filter adaptive window size for pixel s. is the normalized structure factor for pixel s.

2. The intelligent inspection method for computer room equipment based on machine vision as described in claim 1, characterized in that, The guide image obtained based on the device image and the device edge detection image is expressed as follows: In the formula, For guiding images, For device images, This is a map showing the edge detection of the device. This is the recombination coefficient.

3. The intelligent inspection method for computer room equipment based on machine vision as described in claim 1, characterized in that, The guided filter linear model obtained according to the guided filter algorithm is expressed as follows: In the formula, For a rectangular window with size w centered at pixel k, These are the linear coefficients of the guided filter linear model. To guide the gray value of pixel i in the image, This is the filtered output value for pixel i.

4. The intelligent inspection method for computer room equipment based on machine vision as described in claim 1, characterized in that, The linear parameters of the guided filter linear model are obtained by combining the guided filter linear model with linear regression, and the expression is as follows: In the formula, For a rectangular window with size w centered at pixel k, These are the linear coefficients of the guided filter linear model, and M is the total number of pixels within the window. Guide images In the window Mean and variance of grayscale values ​​of mid-pixel For device image I in window The average grayscale value of the middle pixel. These represent the grayscale values ​​of pixel i in the guide image and the device image, respectively. These are the normalization parameters.

5. The intelligent inspection method for computer room equipment based on machine vision as described in claim 1, characterized in that, The Hessian matrix of each pixel in the acquired device image is expressed as follows: In the formula, Let i be the Hessian matrix of pixel i in the device image. Let be the second-order partial derivatives of pixel i in the device image in the x and y directions, respectively. and are the mixed partial derivatives of pixel i in the device image in the xy and yx directions, respectively, where .

6. The intelligent inspection method for computer room equipment based on machine vision as described in claim 1, characterized in that, The step of obtaining the direction of change of each pixel based on the Hessian matrix of each pixel is specifically as follows: calculate the eigenvalues ​​and eigenvectors of the Hessian matrix, and take the eigenvector corresponding to the eigenvalue with the larger absolute value as the direction of change of the pixel.

7. The intelligent inspection method for computer room equipment based on machine vision as described in claim 1, characterized in that, The process of obtaining the device status based on a high-quality device image combined with a neural network specifically involves: the neural network being a target detection network; the network input being a high-quality device image; and the network output being the device bounding box category within the high-quality device image. The device bounding box categories include: (Normal) and (Abnormal) The device bounding box category is used as the device state.

Images