Data center cabinet image splicing method and system, electronic device, and storage medium

Abstract

The application discloses a data center cabinet image splicing method and system, electronic equipment and a storage medium, and belongs to the technical field of image processing. The technical problem to be solved by the application is how to realize real-time online multi-image splicing, improve the accuracy of splicing, and the technical scheme is as follows: a sequence image data of a cabinet is acquired by a patrol robot in a data center; the sequence image data of the cabinet is preprocessed to acquire an edge feature region in the image; a local feature region is set in a reference image and a to-be-spliced image according to the edge feature region, and is used for extracting point features; key point features are used for image matching, a geometric relationship matrix is calculated, image registration is performed, and seamless image splicing fusion is realized. The system comprises a sequence image acquisition unit, an image preprocessing and edge feature acquisition unit, a key point feature extraction and matching unit and an image fusion and splicing unit.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a data center cabinet image stitching method, system, electronic device, and storage medium. Background Technology

[0002] Image stitching is a technique that combines multiple images acquired by a sensor into a larger image. During the shooting process, there are often overlapping areas between neighboring images. The purpose of image stitching is to simultaneously acquire complete scene information within a single, coherent image and accomplish related tasks. In some specific tasks, further image understanding, processing, detection, and classification are required within the stitched image.

[0003] When using inspection robots to inspect data center equipment, it is often necessary to inspect several components of the front of a complete server rack. Therefore, the robot's vision system needs to use a complete image of the front of the rack as the visual processing unit. However, in real-world scenarios, due to the installation and setup of the robot's vision sensors, the effective field of view, and the actual spatial limitations of the robot's movement, the vision sensors cannot capture a complete image of a server rack. This necessitates installing several vision sensors on the robot arm to simultaneously capture several sets of images. These sets of images contain a large number of overlapping areas, requiring the stitching of multiple images. Existing rack images are characterized by repetitive, sheet-like, fine mesh areas; the image information is rich but repetitive and monotonous, thus posing a significant challenge to accurate stitching.

[0004] Therefore, how to achieve real-time online stitching of multiple images, improve the accuracy of stitching, and overcome the shortcomings of image repetition and the limitations of image features in real-world scenarios are technical problems that urgently need to be solved. Summary of the Invention

[0005] The technical objective of this invention is to provide a data center rack image stitching method, system, electronic device, and storage medium to solve the problem of how to achieve real-time online stitching of multiple images and improve the accuracy of stitching.

[0006] The technical objective of this invention is achieved as follows: a method for stitching together images of a data center server rack, the specific method of which is as follows:

[0007] Using inspection robots to acquire sequential image data of server racks in data centers;

[0008] Preprocess the cabinet sequence image data to obtain edge feature regions in the image;

[0009] Local feature regions are set in the reference image and the image to be stitched based on edge feature regions to extract point features;

[0010] Image matching is performed using key point features, geometric relationship matrix is ​​calculated, and image registration is carried out to achieve seamless image stitching and fusion.

[0011] As a preferred method, the following is a specific approach to using inspection robots to acquire rack sequence image data in the data center:

[0012] The inspection robot has four CMOS cameras installed at fixed positions on the side of its body. The four CMOS cameras form a multi-camera sensor system, arranged from top to bottom, with a spacing between adjacent CMOS cameras.

[0013] When the inspection robot walks in a straight line along the data center aisle to the front of the rack, it simultaneously controls the shutters of four side CMOS cameras to open and acquire four RGB images containing different parts of the rack from top to bottom. The four images from top to bottom have overlapping areas and cover the front part of the rack.

[0014] The inspection robot continues to move a set distance, acquiring four images from top to bottom, covering part of the area behind the cabinet, thus encompassing a complete cabinet in eight images, with overlapping areas between each pair of the eight images.

[0015] Following its travel path, the inspection robot sequentially acquires multiple image sets from multiple cabinets.

[0016] As a preferred method, the cabinet sequence image data is preprocessed to obtain the edge feature regions in the image, as follows:

[0017] Image preprocessing and image enhancement: Homomorphic filtering is used to equalize and enhance the image;

[0018] Edge detection: Edge detection images are obtained using the Canny edge detection algorithm. The edge detection images clearly show the mesh-like areas, the areas formed by the edges of the cabinet doors, and the areas formed by the edges between cabinets.

[0019] Vertical projection: Project the pixel values ​​of the edge detection image vertically to obtain a projection curve of the sum of pixel values ​​at the horizontal axis of 800;

[0020] Output signal through wavelet transform and high-pass filtering: Perform Discrete Wavelet Transform (DWT) on the acquired projection curve;

[0021] Discrete wavelet transform (DWT) is a wavelet transform of discretely sampled wavelets. DWT can capture both frequency and temporal information. Different frequency filters within the DWT analyze signals of different frequencies. Wavelet basis functions and scaling functions are used to analyze high-frequency and low-frequency signals, i.e., high-pass and low-pass filters. The low-pass filter removes the high-frequency components of the input signal, outputting the low-frequency components, corresponding to an approximation of the original signal. The high-pass filter removes the low-frequency components, outputting the high-frequency components, corresponding to detailed information. Using the high-pass filtered output generated by the DWT as the signal to be used can be implemented using the `pywt.dwt` function in Python.

[0022] (cA,cD)=pywt.dwt(x,'db1');

[0023] Where x represents the aforementioned projection curve signal; db1 specifies the wavelet function type; the pywt.dwt function returns two arrays, where array cA represents the approximate value and array cD represents the detail coefficients; and expands the horizontal axis of the transformed output signal graph to 800;

[0024] Obtaining linear edge regions: Based on the signal map obtained by discrete wavelet transform, candidate edge regions are obtained. Within the candidate edge regions, the positions of the regions formed by the edges of the racks and the regions formed by the edges between racks are obtained based on the width threshold of the pixel values ​​and the prior physical dimensions.

[0025] More preferably, image preprocessing and image enhancement are as follows:

[0026] Homomorphic filtering removes multiplicative noise while increasing contrast and normalizing brightness, thereby achieving image enhancement. The process of homomorphic filtering image processing can be represented as follows:

[0027] f(x,y)→ln→FFT→H(u,v)→IFFT→exp→f′(x,y);

[0028] Where f(x,y) represents the original image; f′(x,y) represents the processed image; ln represents the logarithmic operation; FFT represents the Fast Fourier Transform; IFFT represents the Inverse Fast Fourier Transform; and exp represents the exponential operation.

[0029] The Canny edge detection algorithm is a multi-stage algorithm, specifically including the following stages:

[0030] (1) Noise reduction: Use a Gaussian filter to remove noise;

[0031] (2) Calculate the image intensity gradient;

[0032] (3) Nonmaximum suppression;

[0033] (4) Lag threshold: determines which edges are edges and which are not edges. That is, two thresholds are set: any edge with an intensity gradient greater than the larger threshold is an edge, while edges below the smaller threshold are non-edges and are discarded.

[0034] Preferably, local feature regions are set in both the reference image and the image to be stitched, based on edge feature regions, to extract point features, as follows:

[0035] Local feature regions: Based on the acquired edge regions, two local feature regions of different sizes are defined at the corresponding positions; the local feature regions include some edge features and some mesh features;

[0036] Key point feature extraction: The ORB feature operator is used to detect feature points in two local feature regions of different sizes, thereby obtaining key point features of the reference image and the image to be stitched together;

[0037] Key point matching: The Brute-Force matching method is used to match the feature point pairs detected in the reference image and the image to be stitched, and the Hamming distance is used as the similarity measure. At the same time, the Random Sampling Consensus (RANSAC) algorithm is used to remove abnormal matching values ​​in the initial matching results to obtain the best set of matching point pairs.

[0038] As a preferred method, key point features are used for image matching, geometric relationship matrix is ​​calculated, and image registration is performed to achieve seamless image stitching and fusion, as detailed below:

[0039] Based on the local feature regions, a set of matching point pairs is obtained. After obtaining the matching pairs of feature points, the geometric transformation relationship between the reference image and the image to be stitched is calculated: the homography matrix is ​​calculated using the findHomography() function of OpenCV.

[0040] Align the reference image and the image to be stitched together in the same coordinate system;

[0041] Image fusion is performed in the overlapping areas of image stitching, and adjustments are made based on edge feature areas to reduce discontinuities at the stitching points. Gradient blending or Poisson blending methods are used to achieve smooth transitions.

[0042] The aligned and blended images are then stitched together to generate the final stitched image.

[0043] A data center rack image stitching system includes a sequence image acquisition unit, an image preprocessing and edge feature acquisition unit, a key point feature extraction and matching unit, and an image fusion and stitching unit;

[0044] The sequence image acquisition unit is used to acquire sequence image data of server racks in the data center using an inspection robot;

[0045] The image preprocessing and edge feature acquisition unit is used to preprocess the cabinet sequence image data to obtain the edge feature regions in the image;

[0046] The key point feature extraction and matching unit is used to set local feature regions in the reference image and the image to be stitched according to the edge feature regions, respectively, for extracting point features;

[0047] The image fusion and stitching unit is used to perform image matching by utilizing key point features, calculate the geometric relationship matrix, perform image registration, and thus achieve seamless image stitching and fusion.

[0048] Preferably, the image preprocessing and edge feature acquisition unit includes an image preprocessing and enhancement module, an edge detection module, a vertical projection module, a signal output module, and a linear edge region acquisition module;

[0049] The image preprocessing and enhancement module is used to perform equalization and enhancement processing on images using homomorphic filtering;

[0050] The edge detection module is used to obtain edge detection images using the Canny edge detection algorithm. The edge detection images clearly show the mesh-like areas, the areas formed by the edges of the cabinet doors, and the areas formed by the edges between cabinets.

[0051] The vertical projection module is used to project the pixel values ​​of the edge detection image vertically, and obtain a projection curve of the sum of pixel values ​​with the horizontal axis at 800.

[0052] The signal output module performs discrete wavelet transform on the projected curve, using the high-pass filtered output of the discrete wavelet transform as the usable signal. The discrete wavelet transform is a wavelet transform where wavelets are discretely sampled. It captures both frequency and temporal information. Different frequency filters within the discrete wavelet transform analyze signals of different frequencies, using wavelet basis functions and scaling functions to analyze high-frequency and low-frequency signals, i.e., high-pass and low-pass filters. The low-pass filter removes the high-frequency components of the input signal, outputting the low-frequency components, corresponding to an approximation of the original signal. The high-pass filter removes the low-frequency components, outputting the high-frequency components, corresponding to detail information. Using the high-pass filtered output generated by the discrete wavelet transform as the usable signal can be implemented using the `pywt.dwt` function in Python.

[0053] (cA,cD)=pywt.dwt(x,'db1');

[0054] Where x represents the aforementioned projection curve signal; db1 specifies the wavelet function type; the pywt.dwt function returns two arrays, where array cA represents the approximate value and array cD represents the detail coefficients; and expands the horizontal axis of the transformed output signal graph to 800;

[0055] The linear edge region acquisition module is used to acquire candidate edge regions based on the signal map obtained by discrete wavelet transform, and within the candidate edge regions, based on the width threshold of pixel values ​​and prior physical dimensions, to acquire the positions of the regions formed by the edges of the cabinets and the regions formed by the edges between cabinets.

[0056] The key point feature extraction and matching unit includes a local feature region acquisition module, a key point feature extraction module, and a key point matching module;

[0057] The local feature region acquisition module is used to define two local feature regions of different sizes at corresponding positions based on the acquired edge regions; the local feature regions include some edge features and some mesh features;

[0058] The key point feature extraction module is used to perform feature point detection in two local feature regions of different sizes using the ORB feature operator, thereby obtaining key point features of the reference image and the image to be stitched together.

[0059] The key point matching module is used to match the feature point pairs detected in the reference image and the image to be stitched using the Brute-Force matching method, and uses the Hamming distance as a similarity measure; at the same time, the Random Sampling Consensus (RANSAC) algorithm is used to remove abnormal matching values ​​in the initial matching results to obtain the best set of matching point pairs.

[0060] An electronic device includes: a memory and at least one processor;

[0061] The memory contains computer programs;

[0062] The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the data center rack image stitching method as described above.

[0063] A computer-readable storage medium storing a computer program that can be executed by a processor to implement the data center rack image stitching method described above.

[0064] The data center rack image stitching method, system, electronic device, and storage medium of the present invention have the following advantages:

[0065] (i) This invention is based on a robot acquiring a sequence of cabinet images in a data center. According to the visual characteristics of the cabinet images, two sets of edge feature regions and local feature regions are obtained respectively. Key point features are extracted according to the defined local feature regions and matched accordingly, thereby completing the stitching of multiple images. This enables real-time online stitching of multiple images, which is beneficial to a series of subsequent image-related algorithms.

[0066] It should be noted that depending on the appearance of the server rack in a specific scenario, there will be some different processing procedures when stitching images. For example, if the original image has shape distortion, the original image needs to be corrected first, and then the series of operations of the method in this paper are performed.

[0067] (ii) This invention targets the visual features of an image, first obtains the edge feature region, defines the local feature region based on this, and extracts feature values ​​based on the local feature region; the defined local feature region retains a limited number of key points to facilitate effective image matching;

[0068] (III) This invention overcomes the shortcomings of image repetition and the limitations of image features in actual scenarios, and realizes real-time online stitching of multiple images, thereby improving the accuracy of stitching.

[0069] (iv) This invention is applicable to the stitching of multiple images that have similar appearance features to the cabinet image. Attached Figure Description

[0070] The invention will be further described below with reference to the accompanying drawings.

[0071] Appendix Figure 1 A flowchart of a data center rack image stitching method;

[0072] Appendix Figure 2 This is a schematic diagram illustrating the decomposition of the region feature extraction method for the left and right reference images and the image to be stitched together.

[0073] Appendix Figure 3 This is a schematic diagram illustrating the decomposition of the region feature extraction method for the two reference images and the image to be stitched together. Detailed Implementation

[0074] The data center cabinet image stitching method, system, electronic equipment, and storage medium of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0075] Example 1:

[0076] As attached Figure 1 As shown in the figure, this embodiment provides a method for stitching images of data center cabinets, which is as follows:

[0077] S1. Use inspection robots to acquire rack sequence image data in the data center;

[0078] S2. Preprocess the cabinet sequence image data to obtain edge feature regions in the image;

[0079] S3. Based on the edge feature regions, local feature regions are set in the reference image and the image to be stitched, respectively, to extract point features;

[0080] S4. Use key point features to perform image matching, calculate the geometric relationship matrix, perform image registration, and then achieve seamless image stitching and fusion.

[0081] In this embodiment, step S1, which involves using an inspection robot to acquire rack sequence image data in the data center, is specifically as follows:

[0082] S101, the inspection robot has four CMOS cameras installed at fixed positions on the side of the body. The four CMOS cameras form a multi-camera sensor system, arranged from top to bottom, and the spacing between adjacent CMOS cameras is set.

[0083] S102. When the inspection robot walks in a straight line along the data center aisle to the front of the rack, it simultaneously controls the shutters of the four side CMOS cameras to open and acquire four RGB images containing different parts of the rack from top to bottom. The four images from top to bottom have overlapping areas and cover the front part of the rack.

[0084] S103. The inspection robot continues to move a set distance and acquires four images from top to bottom, covering part of the area behind the cabinet. Thus, a complete cabinet is included in eight images, and there are overlapping areas between each pair of the eight images.

[0085] S104. Following the travel path, the inspection robot sequentially acquires multiple image groups from multiple cabinets.

[0086] During the camera shooting process, a series of image algorithms are completed simultaneously according to different engineering application requirements.

[0087] The images obtained through the above process also form an offline image dataset, which is used for algorithm testing.

[0088] In this embodiment, step S2 involves preprocessing the cabinet sequence image data to obtain edge feature regions in the image, as follows:

[0089] In addition to the large areas of repetitive mesh, the original image also has two significant edge regions: the region formed by the edges of the cabinet doors and the region formed by the edges between cabinets. These are the most important feature regions used in this embodiment.

[0090] To effectively extract these two regions, a series of preprocessing steps are first required on the original image, as follows:

[0091] S201. Image preprocessing and image enhancement: Homomorphic filtering is used to perform equalization and enhancement processing on the image;

[0092] S202, Edge Detection: Edge detection images are obtained using the Canny edge detection algorithm. The edge detection images clearly show the mesh-like areas, the areas formed by the edges of the cabinet doors, and the areas formed by the edges between cabinets.

[0093] S203, Vertical projection: The size of the image acquired in a specific scene varies depending on the visual sensor. The size of the image in this dataset is 800×603 pixels.

[0094] Projecting the pixel values ​​vertically onto the edge detection image yields a summation projection curve of pixel values ​​at the horizontal axis of 800.

[0095] S204. Output signal through wavelet transform and high-pass filtering: Perform discrete wavelet transform (DWT) on the acquired projection curve;

[0096] Discrete wavelet transform (DWT) is a wavelet transform of discretely sampled wavelets. DWT can capture both frequency and temporal information. Different frequency filters within the DWT analyze signals of different frequencies. Wavelet basis functions and scaling functions are used to analyze high-frequency and low-frequency signals, i.e., high-pass and low-pass filters. The low-pass filter removes the high-frequency components of the input signal, outputting the low-frequency components, corresponding to an approximation of the original signal. The high-pass filter removes the low-frequency components, outputting the high-frequency components, corresponding to detailed information. Using the high-pass filtered output generated by the DWT as the signal to be used can be implemented using the `pywt.dwt` function in Python.

[0097] (cA,cD)=pywt.dwt(x,'db1');

[0098] Where x represents the aforementioned projection curve signal; db1 specifies the wavelet function type; the pywt.dwt function returns two arrays, where array cA represents the approximate value and array cD represents the detail coefficients; and expands the horizontal axis of the transformed output signal graph to 800;

[0099] S205. Obtaining Linear Edge Regions: Based on the signal map obtained by discrete wavelet transform, candidate edge regions are obtained. Within these candidate edge regions, based on the pixel width threshold and prior physical dimensions, the locations of the regions formed by the edges of the racks and the regions formed by the edges between racks are obtained, as shown in the attached figure. Figure 2As shown in L10 and L11.

[0100] As attached Figure 2 As shown, based on prior knowledge, the edge areas of the cabinet doors and the edge areas between two cabinets are defined as edge feature regions. This definition of edge feature regions is based on their salience relative to the visual features of repetitive meshes within the entire image. This visual contrast facilitates the localization of specific regions in the image using edge feature regions as a benchmark.

[0101] It should be noted that the two edge regions mentioned above may differ depending on the actual shape of the server rack. This method uses images from the dataset as examples.

[0102] The image preprocessing and image enhancement in step S201 of this embodiment are as follows:

[0103] Homomorphic filtering removes multiplicative noise while increasing contrast and normalized brightness, thereby achieving image enhancement.

[0104] An image can be represented as the product of its illuminance and reflectance components. Although these two components are inseparable in the time domain, they can be linearly separated in the frequency domain through Fourier transform. Illuminance can be considered as ambient lighting, which varies relatively little and can be regarded as the low-frequency component of the image; while reflectance varies relatively much and can be regarded as the high-frequency component. By separately processing the effects of illuminance and reflectance on pixel gray values, usually through a high-pass filter, the illumination of the image is made more uniform, thereby enhancing the detail features of shadow areas.

[0105] The process of homomorphic filtering image processing is expressed as follows:

[0106] f(x,y)→ln→FFT→H(u,v)→IFFT→exp→f′(x,y);

[0107] Where f(x,y) represents the original image; f′(x,y) represents the processed image; ln represents the logarithmic operation; FFT represents the Fast Fourier Transform; IFFT represents the Inverse Fast Fourier Transform; and exp represents the exponential operation.

[0108] Homomorphic filtering also yields a grayscale image. The following steps are performed on the homomorphically filtered image.

[0109] The Canny edge detection algorithm in step S202 of this embodiment is a multi-stage algorithm, specifically including the following stages:

[0110] (1) Noise reduction: Use a Gaussian filter to remove noise;

[0111] (2) Calculate the image intensity gradient;

[0112] (3) Nonmaximum suppression;

[0113] (4) Lag threshold: determines which edges are edges and which are not edges. That is, two thresholds are set: any edge with an intensity gradient greater than the larger threshold is an edge, while edges below the smaller threshold are non-edges and are discarded.

[0114] In step S3 of this embodiment, local feature regions are set in the reference image and the image to be stitched according to the edge feature regions, respectively, for extracting point features, as follows:

[0115] S301, Local Feature Regions: Based on the acquired edge regions, two local feature regions of different sizes are defined at the corresponding positions; the local feature regions include some edge features and some mesh features;

[0116] Based on the edge regions obtained above, two local feature regions of different sizes are defined at certain locations. These local feature regions contain some edge features and some mesh features. (See attached image.) Figure 2 As shown, the areas A10, A11 and A20, A21 enclosed by the dashed borders in the image are the defined local feature regions. They are called local because they are relative to the entire image.

[0117] It should be noted that the size of the local feature region is adjusted based on the number of key point feature values ​​extracted in subsequent processes.

[0118] Corresponding local feature regions in the two images have similar structural features.

[0119] The corresponding local feature regions in the two images are located in the possible overlapping regions. The left image is used as the reference image and the right image is used as the image to be matched.

[0120] S302, Key Point Feature Extraction: See attached... Figure 2 As shown, the left image is used as the reference image and the right image is used as the image to be matched. The ORB feature operator is used to detect feature points in local feature regions of two different sizes, thereby obtaining the key point features of the reference image and the image to be stitched.

[0121] S303. Key point matching: The Brute-Force matching method is used to match the feature point pairs detected in the reference image and the image to be stitched, and the Hamming distance is used as the similarity measure. At the same time, the Random Sampling Consensus (RANSAC) algorithm is used to remove abnormal matching values ​​in the initial matching results to obtain the best set of matching point pairs.

[0122] In this embodiment, step S4 involves using key point features for image matching, calculating the geometric relationship matrix, performing image registration, and ultimately achieving seamless image stitching and fusion, as detailed below:

[0123] S401. Obtain a set of matching point pairs based on local feature regions. After obtaining the matching pairs of feature points, calculate the geometric transformation relationship between the reference image and the image to be stitched: use OpenCV's findHomography() function to calculate the homography matrix.

[0124] S402. Align the reference image and the image to be stitched in the same coordinate system;

[0125] S403. Perform image fusion in the overlapping area of ​​image stitching and adjust it according to the edge feature area to reduce the discontinuity at the stitching point, and refer to the method of gradient blending or Poisson blending to achieve a smooth transition.

[0126] S404. The aligned and blended images are stitched together to generate the final stitched image.

[0127] It should be noted that, as shown in the attached document... Figure 3 As shown, for the case of two adjacent images positioned one above the other, because the four cameras are mounted vertically, the misalignment of the two adjacent images in the left-right direction is limited to a very small scale. The stitching process is the same as that for stitching two left-right images.

[0128] Follow the above process to complete the overall stitching of the 8 images.

[0129] Example 2:

[0130] This embodiment provides a data center rack image stitching system, which includes a sequence image acquisition unit, an image preprocessing and edge feature acquisition unit, a key point feature extraction and matching unit, and an image fusion and stitching unit;

[0131] The sequence image acquisition unit is used to acquire sequence image data of server racks in the data center using an inspection robot;

[0132] The image preprocessing and edge feature acquisition unit is used to preprocess the cabinet sequence image data to obtain the edge feature regions in the image;

[0133] The key point feature extraction and matching unit is used to set local feature regions in the reference image and the image to be stitched according to the edge feature regions, respectively, for extracting point features;

[0134] The image fusion and stitching unit is used to perform image matching by utilizing key point features, calculate the geometric relationship matrix, perform image registration, and thus achieve seamless image stitching and fusion.

[0135] The image preprocessing and edge feature acquisition unit in this embodiment includes an image preprocessing and enhancement module, an edge detection module, a vertical projection module, a signal output module, and a linear edge region acquisition module;

[0136] The image preprocessing and enhancement module is used to perform equalization and enhancement processing on images using homomorphic filtering;

[0137] The edge detection module is used to obtain edge detection images using the Canny edge detection algorithm. The edge detection images clearly show the mesh-like areas, the areas formed by the edges of the cabinet doors, and the areas formed by the edges between cabinets.

[0138] The vertical projection module is used to project the pixel values ​​of the edge detection image vertically, and obtain a projection curve of the sum of pixel values ​​with the horizontal axis at 800.

[0139] The signal output module performs discrete wavelet transform on the projected curve, using the high-pass filtered output of the discrete wavelet transform as the usable signal. The discrete wavelet transform is a wavelet transform where wavelets are discretely sampled. It captures both frequency and temporal information. Different frequency filters within the discrete wavelet transform analyze signals of different frequencies, using wavelet basis functions and scaling functions to analyze high-frequency and low-frequency signals, i.e., high-pass and low-pass filters. The low-pass filter removes the high-frequency components of the input signal, outputting the low-frequency components, corresponding to an approximation of the original signal. The high-pass filter removes the low-frequency components, outputting the high-frequency components, corresponding to detail information. Using the high-pass filtered output generated by the discrete wavelet transform as the usable signal can be implemented using the `pywt.dwt` function in Python.

[0140] (cA,cD)=pywt.dwt(x,'db1');

[0141] Where x represents the aforementioned projection curve signal; db1 specifies the wavelet function type; the pywt.dwt function returns two arrays, where array cA represents the approximate value and array cD represents the detail coefficients; and expands the horizontal axis of the transformed output signal graph to 800;

[0142] The linear edge region acquisition module is used to obtain candidate edge regions based on the signal map obtained by discrete wavelet transform, and within the candidate edge regions, based on the width threshold of pixel values ​​and prior physical dimensions, to obtain the positions of the regions formed by the edges of the cabinets and the regions formed by the edges between cabinets.

[0143] The key point feature extraction and matching unit in this embodiment includes a local feature region acquisition module, a key point feature extraction module, and a key point matching module.

[0144] The local feature region acquisition module is used to define two local feature regions of different sizes at corresponding positions based on the acquired edge regions; the local feature regions include some edge features and some mesh features;

[0145] The key point feature extraction module is used to perform feature point detection in two local feature regions of different sizes using the ORB feature operator, thereby obtaining key point features of the reference image and the image to be stitched together.

[0146] The key point matching module is used to match the feature point pairs detected in the reference image and the image to be stitched using the Brute-Force matching method, and uses the Hamming distance as a similarity measure; at the same time, the Random Sampling Consensus (RANSAC) algorithm is used to remove abnormal matching values ​​in the initial matching results to obtain the best set of matching point pairs.

[0147] Example 3:

[0148] This invention also provides an electronic device, including: a memory and a processor;

[0149] The memory stores the instructions executed by the computer.

[0150] The processor executes computer execution instructions stored in the memory, causing the processor to perform the data center rack image stitching method in any embodiment of the present invention.

[0151] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can be a microprocessor or any conventional processor.

[0152] Memory is used to store computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function, etc.; the data storage area can store data created based on the use of the terminal, etc. In addition, memory can also include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart memory cards (SMC), secure digital cards (SD cards), flash memory cards, at least one disk storage device, flash memory devices, or other volatile solid-state storage devices.

[0153] Example 4:

[0154] This invention also provides a computer-readable storage medium storing multiple instructions, which are loaded by a processor to cause the processor to execute the data center rack image stitching method according to any embodiment of this invention. Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the above embodiments is stored, and the computer (or CPU or MPU) of the system or apparatus can read and execute the program code stored in the storage medium.

[0155] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0156] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0157] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0158] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for stitching images of a data center server rack, characterized in that, The method is as follows: Using inspection robots to acquire sequential image data of server racks in data centers; Preprocess the cabinet sequence image data to obtain edge feature regions in the image; Local feature regions are set in the reference image and the image to be stitched based on edge feature regions to extract point features; Image matching is performed using key point features, geometric relationship matrices are calculated, and image registration is carried out to achieve seamless image stitching and fusion. Specifically, the preprocessing of the cabinet sequence image data to obtain edge feature regions in the images is as follows: Image preprocessing and image enhancement: Homomorphic filtering is used to equalize and enhance the image; Edge detection: Edge detection images are obtained using the Canny edge detection algorithm. The edge detection images clearly show the mesh-like areas, the areas formed by the edges of the cabinet doors, and the areas formed by the edges between cabinets. Vertical projection: Project the pixel values ​​of the edge detection image vertically to obtain a projection curve of the sum of pixel values ​​at the horizontal axis of 800; The output signal is obtained through wavelet transform and high-pass filtering: A discrete wavelet transform is performed on the acquired projection curve; the high-pass filtered output generated by the discrete wavelet transform is used as the signal, which can be achieved using the Python function `pywt.dwt`. ； Where x represents the aforementioned projection curve signal; db1 specifies the wavelet function type; the pywt.dwt function returns two arrays, where array cA represents the approximate value and array cD represents the detail coefficients; and expands the horizontal axis of the transformed output signal graph to 800; Obtaining linear edge regions: Based on the signal map obtained by discrete wavelet transform, candidate edge regions are obtained, and within the candidate edge regions, the positions of the regions formed by the edges of the racks and the regions formed by the edges between racks are obtained based on the width threshold of the pixel values ​​and the prior physical dimensions. Local feature regions are set in both the reference image and the image to be stitched based on edge feature regions, and point features are extracted as follows: Local feature regions: Based on the acquired edge regions, two local feature regions of different sizes are defined at the corresponding positions; the local feature regions include some edge features and some mesh features; Key point feature extraction: The ORB feature operator is used to detect feature points in two local feature regions of different sizes to obtain key point features of the reference image and the image to be stitched. Key point matching: The Brute-Force matching method is used to match the feature point pairs detected in the reference image and the image to be stitched, and the Hamming distance is used as the similarity measure. At the same time, the Random Sampling Consensus (RANSAC) algorithm is used to remove abnormal matching values ​​in the initial matching results to obtain the best set of matching point pairs.

2. The data center rack image stitching method according to claim 1, characterized in that, The specific steps for using inspection robots to acquire sequential image data of server racks in a data center are as follows: The inspection robot has four CMOS cameras installed at fixed positions on the side of its body. The four CMOS cameras form a multi-camera sensor system, arranged from top to bottom, with a spacing between adjacent CMOS cameras. When the inspection robot walks in a straight line along the data center aisle to the front of the rack, it simultaneously controls the shutters of four side CMOS cameras to open and acquire four RGB images containing different parts of the rack from top to bottom. The four images from top to bottom have overlapping areas and cover the front part of the rack. The inspection robot continues to move a set distance, acquiring four images from top to bottom, covering part of the area behind the cabinet, thus encompassing a complete cabinet in eight images, with overlapping areas between each pair of the eight images. Following its travel path, the inspection robot sequentially acquires multiple image sets from multiple cabinets.

3. The data center rack image stitching method according to claim 1, characterized in that, Image preprocessing and image enhancement are detailed below: Homomorphic filtering removes multiplicative noise while increasing contrast and normalizing brightness, thereby achieving image enhancement. The process of homomorphic filtering image processing can be represented as follows: ； in, Represents the original image; This represents the processed image; Represents logarithmic operations; Represents the Fast Fourier Transform; This represents the inverse fast Fourier transform. Indicates exponentiation; The Canny edge detection algorithm is a multi-stage algorithm, specifically including the following stages: (1) Noise reduction: Use a Gaussian filter to remove noise; (2) Calculate the image intensity gradient; (3) Non-maximum suppression; (4) Lag threshold: determines which edges are edges and which are not edges. That is, two thresholds are set: any edge with an intensity gradient greater than the larger threshold is an edge, while edges below the smaller threshold are non-edges and are discarded.

4. The data center rack image stitching method according to claim 1, characterized in that, Image matching is performed using key point features, a geometric relationship matrix is ​​calculated, and image registration is carried out to achieve seamless image stitching and fusion, as detailed below: After obtaining the matching pair set of feature points, the geometric transformation relationship between the reference image and the image to be stitched is calculated; Based on the local feature regions, a set of matching point pairs is obtained. After obtaining the matching pairs of feature points, the geometric transformation relationship between the reference image and the image to be stitched is calculated: the homography matrix is ​​calculated using the findHomography() function of OpenCV. Align the reference image and the image to be stitched together in the same coordinate system; Image fusion is performed in the overlapping areas of image stitching, and adjustments are made based on edge feature areas to reduce discontinuities at the stitching points. Gradient blending or Poisson blending methods are used to achieve smooth transitions. The aligned and blended images are then stitched together to generate the final stitched image.

5. A data center rack image stitching system, characterized in that, The system includes a sequence image acquisition unit, an image preprocessing and edge feature acquisition unit, a key point feature extraction and matching unit, and an image fusion and stitching unit; The sequence image acquisition unit is used to acquire sequence image data of server racks in the data center using an inspection robot; The image preprocessing and edge feature acquisition unit is used to preprocess the cabinet sequence image data to obtain the edge feature regions in the image; The key point feature extraction and matching unit is used to set local feature regions in the reference image and the image to be stitched according to the edge feature regions, respectively, for extracting point features; The image fusion and stitching unit is used to perform image matching by utilizing key point features, calculate the geometric relationship matrix, perform image registration, and thus achieve seamless image stitching and fusion. The image preprocessing and edge feature acquisition unit includes an image preprocessing and enhancement module, an edge detection module, a vertical projection module, a signal output module, and a linear edge region acquisition module. The image preprocessing and enhancement module is used to perform equalization and enhancement processing on images using homomorphic filtering; The edge detection module is used to obtain edge detection images using the Canny edge detection algorithm. The edge detection images clearly show the mesh-like areas, the areas formed by the edges of the cabinet doors, and the areas formed by the edges between cabinets. The vertical projection module is used to project the pixel values ​​of the edge detection image vertically, and obtain a projection curve of the sum of pixel values ​​with the horizontal axis at 800. The signal output module performs discrete wavelet transform on the projected curve, using the high-pass filtered output of the discrete wavelet transform as the usable signal. The use of the high-pass filtered output of the discrete wavelet transform as the usable signal can be implemented using the Python function `pywt.dwt`. ； Where x represents the aforementioned projection curve signal; db1 specifies the wavelet function type; the pywt.dwt function returns two arrays, where array cA represents the approximate value and array cD represents the detail coefficients; and expands the horizontal axis of the transformed output signal graph to 800; The linear edge region acquisition module is used to acquire candidate edge regions based on the signal map obtained by discrete wavelet transform, and within the candidate edge regions, based on the width threshold of pixel values ​​and prior physical dimensions, to acquire the positions of the regions formed by the edges of the cabinets and the regions formed by the edges between cabinets. The key point feature extraction and matching unit includes a local feature region acquisition module, a key point feature extraction module, and a key point matching module; The local feature region acquisition module is used to define two local feature regions of different sizes at corresponding positions based on the acquired edge regions; the local feature regions include some edge features and some mesh features; The key point feature extraction module is used to perform feature point detection in two local feature regions of different sizes using the ORB feature operator, thereby obtaining key point features of the reference image and the image to be stitched together. The key point matching module is used to match the feature point pairs detected in the reference image and the image to be stitched using the Brute-Force matching method, and uses the Hamming distance as a similarity measure; at the same time, the Random Sampling Consensus (RANSAC) algorithm is used to remove abnormal matching values ​​in the initial matching results to obtain the best set of matching point pairs.

6. An electronic device, characterized in that, include: Memory and at least one processor; The memory contains computer programs; The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the data center rack image stitching method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by a processor to implement the data center rack image stitching method as described in any one of claims 1 to 4.

Images