A method for acquiring rack status information, a storage medium, and electronic equipment.

Abstract

This invention relates to the field of rack inspection technology, and in particular to a method, storage medium, and electronic device for acquiring rack status information. It includes: generating primary displacement information between the current position of the camera device and the initial target acquisition point based on the difference between the world coordinates corresponding to the center pixel position of each auxiliary positioning patch and the world coordinates corresponding to a preset center position; controlling the movement of the camera device based on the primary and secondary displacement information, and acquiring image information corresponding to the current position upon completion of each displacement information step. In this invention, by setting the abnormal target acquisition point corresponding to the image area where the indicator light is obscured, near the rack door, the camera device can be brought close to the observation hole to capture a specific area. This avoids obstruction from honeycomb metal in the camera device's field of view, thus allowing for clearer acquisition of the image information of the obscured indicator light.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cabinet inspection, in particular to a cabinet state information acquisition method, a storage medium and an electronic device. BACKGROUND

[0002] Cabinet inspection is an important part of data center or server room maintenance, aiming to ensure that all hardware devices are running normally, the environmental conditions are suitable, and potential problems are found and solved in time. During the inspection process, the display content of the indicator light or display screen corresponding to each condition (such as the temperature and humidity sensor indicator light and the various signal indicator lights set on each service in the cabinet) needs to be observed to prevent and solve potential problems and ensure the stable operation of the data center.

[0003] In the prior art, an automatic inspection terminal is used to automatically inspect the equipment in the cabinet, which is mainly an intelligent monitoring method realized by using autonomous navigation, automatic control technology and image technology. Currently, the product usually adopts a combination of a self-moving platform, a multi-axis adjustment assembly and a camera for image acquisition. The self-moving platform is usually a base in the form of wheels, four legs or tracks. First, the automatic inspection terminal travels to the vicinity of the cabinet along the preset route through the self-moving platform; then the camera acquires images of the equipment, usually at a position far from the cabinet door in order to capture all the indicator lights in one image; finally, the device state value and the indicator light state are analyzed by image recognition technology.

[0004] However, there are many cabinet doors that are not transparent glass but honeycomb-shaped metal doors. In this case, the metal mesh structure of the cabinet door will cause strong shielding effect on the line of sight, so that small indicator lights (such as dot-shaped indicator lights) or digital displays set at some preset positions are blocked, and the distance between the camera device and the cabinet door further reduces the image size of the indicator lights, which leads to the fact that the camera cannot acquire clear images of some indicator lights, and the device state value and the indicator light state cannot be analyzed by image recognition technology. SUMMARY

[0005] To solve one of the above technical problems, the technical solution adopted by the present application is as follows:

[0006] According to one aspect of the present application, a cabinet state information acquisition method is provided, which comprises the following steps:

[0007] According to the target cabinet closed door image collected by the camera device in the inspection area, the world coordinates corresponding to each auxiliary positioning sticker center pixel position in the target cabinet closed door image are obtained; the image of one auxiliary positioning sticker includes a two-dimensional code, and the two-dimensional code stores the world coordinates corresponding to each auxiliary positioning sticker preset center position and the secondary displacement information corresponding to two adjacent nodes in the preset camera inspection path; the nodes in the preset camera inspection path include the initial target collection point of the camera device and the abnormal target collection point corresponding to the abnormal area in the initial collection image; the lens of the camera device located at the abnormal target collection point is located at the corresponding observation hole on the cabinet door, and the distance between the abnormal target collection point and the plane of the cabinet door is less than the distance between the initial target collection point and the plane of the cabinet door; the abnormal target collection point is used to collect the local state information image of the cabinet; the initial target collection point is used to collect the state information image of the cabinet; the abnormal area includes the image area of the indicator light being shielded;

[0008] According to the difference between the world coordinates corresponding to each auxiliary positioning sticker center pixel position and the world coordinates corresponding to the preset center position, the primary displacement information between the current position of the camera device and the initial target collection point is generated; the primary displacement information and the secondary displacement information are used to determine the adjustment increment of each position adjustment axis in the multi-axis adjustment assembly corresponding to the camera device;

[0009] According to the order of the nodes to be reached by the primary displacement information and the secondary displacement information in the preset camera inspection path, the camera device is controlled to move in turn, and the state information image of the corresponding area of the cabinet is collected after each displacement information is executed.

[0010] Further, the abnormal target collection point corresponding to the abnormal area in the initial collection image is obtained according to the following steps:

[0011] The camera device is controlled at the initial target collection point to respectively acquire the open door state image and the closed door state image of the target cabinet;

[0012] The open door state binary image and the closed door state binary image are generated by respectively performing image region segmentation processing on the open door state image and the closed door state image; the pixel value of the target image area corresponding to the indicator light in the binary image is 0, and the pixel value of the background image area is 255;

[0013] The open door state binary image is used as a mask for the closed door state image to obtain the filling coefficient corresponding to each indicator light mask area in the open door state binary image; wherein the filling coefficient Gn corresponding to the nth indicator light mask area satisfies the following conditions:

[0014] Gn=G z n / G m n ;

[0015] wherein, G z n is the area of the indicator light image region in the nth indicator light mask region in the closed door state image; G m n is the image area of the nth indicator light mask region in the open door state binary image;

[0016] If the filling coefficient corresponding to any one of the indicator light mask regions in the open door state binary image is less than the filling threshold, the world coordinate corresponding to the center of the indicator light mask region is taken as the coordinate of the first type of abnormal region;

[0017] The center coordinate of the observation hole closest to the coordinate of the first type of abnormal region in the equivalent observation plane of the cabinet door is taken as the coordinate corresponding to the first type of abnormal target acquisition point. The equivalent observation plane of the cabinet door is parallel and spaced apart from the cabinet door.

[0018] Further, the abnormal region also includes a false indicator light image region formed by scattered light leakage;

[0019] After generating the open door state binary image and the closed door state binary image, the method further comprises:

[0020] Using the closed door state binary image as a mask for the open door state image, if there is no indicator light image in any one of the indicator light mask regions in the closed door state binary image, the world coordinate corresponding to the center of the indicator light image closest to the indicator light mask region in the open door state image is taken as the coordinate of the second type of abnormal region;

[0021] The center coordinate of the observation hole closest to the coordinate of the second type of abnormal region in the equivalent observation plane of the cabinet door is taken as the coordinate corresponding to the second type of abnormal target acquisition point.

[0022] Further, the two-dimensional code also stores the position coordinates corresponding to the initial target acquisition point and the abnormal target acquisition point, and the method further comprises:

[0023] Based on the nearest neighbor algorithm, the initial target acquisition point and the abnormal target acquisition point are subjected to path planning processing to generate a preset camera inspection path.

[0024] Further, the two-dimensional code also stores an updated closed door state binary image corresponding to a previous update period of the cabinet, and the method further comprises:

[0025] If an update instruction is obtained, the camera device is controlled to acquire a target cabinet closed door image at the initial target acquisition point at the current update time;

[0026] perform image region segmentation processing on the target cabinet closed door image collected at the current update time to generate an updated closed door state binary image corresponding to the current update time; the pixel value of the target image region corresponding to the indicator light in the updated closed door state binary image corresponding to the current update time is 0, and the pixel value of the background image region is 255;

[0027] perform pixel comparison processing on the updated closed door state binary image corresponding to the current update time and the updated closed door state binary image corresponding to the previous update period to generate at least one position change image region in which the updated closed door state binary image corresponding to the current update time and the updated closed door state binary image corresponding to the previous update period exist pixel differences;

[0028] take the center coordinate of the observation hole in the cabinet door equivalent observation plane closest to the center coordinate of the position change image region as the coordinate corresponding to the third type of abnormal target acquisition point.

[0029] Further, the pixel comparison processing includes:

[0030] align the updated closed door state binary image corresponding to the current update time and the updated closed door state binary image corresponding to the previous update period;

[0031] perform pixel-by-pixel comparison on the two images to generate a position change image region; the pixel-by-pixel comparison satisfies the following conditions:

[0032]

[0033] wherein, diff (x,y) is the pixel value after comparing two pixels at pixel coordinates (x, y); imgD (x,y) is the pixel value of the pixel at pixel coordinates (x, y) in the updated closed door state binary image corresponding to the current update time; imgS (x,y) is the pixel value of the pixel at pixel coordinates (x, y) in the updated closed door state binary image corresponding to the previous update period.

[0034] Further, after generating the coordinate corresponding to the third type of abnormal target acquisition point, the method further includes:

[0035] if the minimum distance between the third type of abnormal target acquisition point and the initial target acquisition point or the abnormal target acquisition point is greater than the field of view radius of the camera device, the third type of abnormal target acquisition point is determined as a new abnormal target acquisition point;

[0036] The field of view radius L of the camera device satisfies the following condition:

[0037] H is the distance between the lens of the camera device and the equivalent plane of the internal indicator light of the cabinet, and FOV is the field of view angle of the lens of the camera device.

[0038] According to the updated initial target collection point and the abnormal target collection point, a new preset camera inspection path is generated.

[0039] Further, after generating the coordinates corresponding to the third type of abnormal target collection point, the method further comprises:

[0040] If the minimum distance between the third type of abnormal target collection point and the initial target collection point or the abnormal target collection point is less than the field of view radius of the camera device, the third type of abnormal target collection point is deleted.

[0041] According to a second aspect of the present application, a non-transitory computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the cabinet state information acquisition method.

[0042] According to a third aspect of the present application, an electronic device is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the cabinet state information acquisition method.

[0043] The present application has at least one of the following beneficial effects:

[0044] In the present application, the abnormal target collection point corresponding to the image area of the indicator light being shielded is arranged at a position close to the cabinet door, so that the lens of the camera device located at the abnormal target collection point is located at the corresponding observation hole of the cabinet door. That is, the camera device can be close to the observation hole to take a picture of a certain local part. At this time, since the camera device can avoid the shielding of the honeycomb metal in the field of view, the image information of the shielded indicator light can be obtained more clearly, so as to identify the related state information of the cabinet through the visible light display information of the indicator light subsequently.

[0045] Meanwhile, in the present application, the way of sticking multiple auxiliary positioning stickers on the cabinet door is also adopted to position the initial target collection point of the camera device each time of inspection. In this way, the position of the initial collection point of each subsequent inspection is ensured to be exactly the same as the position of the initial collection point set in the previous preparation stage, and thus the visual angle range of the image captured each time of subsequent inspection can be ensured to be the same as the visual angle range specified in the previous stage. Since the positions of the devices in the cabinet basically do not change frequently, the positions of the indicator lights that will appear to be blocked when shooting at the initial collection point are also stable and unchanged. Thus, the initial target collection point and the abnormal target collection point can be used to form a preset camera inspection path, and the corresponding displacement information can be stored in the two-dimensional code. The initial target collection point is used to collect the full image of the state information of the cabinet, and the abnormal target collection point is used to obtain the local image of the local state information. When the camera device performs inspection, the information in the two-dimensional code can be directly used to position the initial target collection point, and the camera device can move and shoot according to the preset camera inspection path, and thus the image information of each state indicator light in the cabinet can be clearly and comprehensively collected.

[0046] In addition, in the present application, the secondary displacement information is directly stored in the two-dimensional code, so that when performing subsequent inspection, the displacement adjustment system of the camera device can directly adjust the corresponding position adjustment shaft according to the secondary displacement information each time, so that the entire inspection path can be traversed, and the displacement information does not need to be calculated during inspection, and thus the inspection efficiency can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0048] Figure 1 A flowchart of a cabinet state information acquisition method provided by an embodiment of the present application is shown in the figure.

[0049] Figure 2 A working scene diagram of a camera device when inspecting a cabinet is shown in the figure.

[0050] Figure 3 A diagram showing a false indicator light image area is shown in the figure.

[0051] Figure 4 A cabinet state information working diagram of an inspection robot when inspecting is shown in the figure. DETAILED DESCRIPTION

[0052] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments of the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0053] As a possible embodiment of the present application, as shown in Figure 1 , a cabinet state information acquisition method is provided, which comprises the following steps:

[0054] S100: According to the target cabinet closing door image currently collected by the camera device 1, the world coordinates corresponding to the center pixel position of each auxiliary positioning sticker in the target cabinet closing door image are obtained. The camera device 1 can refer to the camera device 1 in Figure 2 .

[0055] The image of one of the auxiliary positioning stickers includes a two-dimensional code, which stores the world coordinates corresponding to the preset center position of each auxiliary positioning sticker, and the secondary displacement information corresponding to any two adjacent nodes in the preset camera inspection path. The nodes in the preset camera inspection path include the initial target collection point of the camera device 1 and the abnormal target collection point corresponding to the abnormal area in the initial collection image. The lens of the camera device 1 located at the abnormal target collection point is located at the corresponding observation hole on the cabinet door. Specifically, the two-dimensional code also stores the position coordinates corresponding to the initial target collection point and the abnormal target collection point. The initial target collection point and the abnormal target collection point can be processed by path planning based on the nearest neighbor algorithm to generate a preset camera inspection path, and the preset camera inspection path is the shortest running path.

[0056] In this embodiment, the terms such as primary displacement information, secondary displacement information, initial target collection point, initial collection image and abnormal target collection point are described as follows to make the meaning of the terms in this embodiment more clear and explicit.

[0057] The primary displacement information and the secondary displacement information are displacement information, and are used to control the adjustment variation of each position adjustment shaft in the multi-axis adjustment assembly. Each element in the displacement information is a displacement increment (i.e., an adjustment increment of the corresponding position adjustment shaft) in the prior art. Taking a three-axis adjustment assembly as an example, if a displacement information is (ΔX, ΔY, ΔZ), ΔX represents that the X-axis moves a distance of ΔX from the current position in the preset positive direction, and ΔY and ΔZ represent that the Y-axis and the Z-axis respectively move a distance of ΔY and ΔZ from the current position in the respective preset positive direction. In the embodiment, an initial inspection parking position (i.e., an initial target collection point) of the camera device 1 is arranged in front of each cabinet, but due to the positioning error of the moving platform of the inspection device, the camera device 1 may not be parked at the initial inspection parking position. Therefore, in the embodiment, the primary displacement information is used to adjust the camera device 1 from the current inaccurate parking position to the position corresponding to the initial target collection point. The camera device 1 can collect the state information full map of the cabinet at the initial target collection point.

[0058] Then, due to the fact that some areas in the state information full map of the cabinet are the indicator light blocked areas or the false indicator light image areas 3, these areas are abnormal areas, and the collection points of the local state information maps corresponding to these areas are abnormal target collection points. The plurality of abnormal target collection points can constitute part of the preset camera inspection path, and the secondary displacement information is used to control the movement of each shaft in the multi-axis adjustment assembly, so as to drive the camera device 1 to move from the current abnormal target collection point to the next abnormal target collection point, thereby completing the collection of the local state information map of the cabinet at each abnormal target collection point.

[0059] Before this step, some corresponding preparation work needs to be done, which can include the following contents:

[0060] A plurality of auxiliary positioning stickers are pasted on each cabinet door, such as three rectangular auxiliary positioning stickers, which are pasted on the upper left, lower left and upper right corners of the cabinet to provide positioning assistance. The auxiliary positioning stickers are distributed on the corners of the shooting area, which can provide a global reference framework for the entire scene. Through the positional relationship of these markers, the coordinate system of the scene (such as the origin, the X-axis, the Y-axis, etc.) can be clearly defined, so that the absolute position of the target object can be calculated more conveniently. In the present application, the auxiliary positioning sticker on the upper left corner can be set as the coordinate origin, the center line between the auxiliary positioning stickers on the upper left and the upper right can be set as the X-axis, and the center line between the auxiliary positioning stickers on the upper left and the lower left can be set as the Y-axis. Thus, the world coordinates of each position point can be determined, and the specific coordinate values or displacement information can be stored in the two-dimensional code as needed.

[0061] An initial target acquisition point, i.e., a fixed inspection point, is determined, at which the camera device 1 can generally capture image information of all indicator lights in the cabinet, i.e., the state information of the cabinet can be acquired in full at the initial target acquisition point. However, due to the limitation of the shooting position, some indicator lights can be blocked and the full image of the indicator light cannot be acquired, or due to the multiple diffuse reflection of the metal cabinet door and the outer wall of the equipment in the cabinet on the light, some light leaks out of the honeycomb observation hole without an indicator light, and then a false indicator light image is formed in the area when the camera device 1 images. The image area corresponding to the above two cases is an abnormal area in this embodiment.

[0062] Specifically, the abnormal area in which the indicator light is blocked in the initial acquisition image is acquired according to the following steps at the corresponding abnormal target acquisition point.

[0063] S101: Control the camera device 1 to acquire an open-door state image and a closed-door state image of the target cabinet at the initial target acquisition point, respectively.

[0064] In this step, the position of the camera device 1 is kept unchanged, and images in the open-door and closed-door states of the cabinet are shot. In addition, in order to find all abnormal areas more comprehensively, all indicator lights in the cabinet can be set to the working state.

[0065] S102: Perform image region segmentation processing on the open-door state image and the closed-door state image to generate an open-door state binary image and a closed-door state binary image. The pixel value of the target image area corresponding to the indicator light in the binary image is 0, and the pixel value of the background image area is 255.

[0066] In this step, due to the color and shape of the indicator light in the cabinet, which are greatly different from the remaining background parts (such as the equipment outer wall panel and the mesh cabinet door), the binary image composed of the indicator light and other backgrounds can be better segmented through image region segmentation processing. In this embodiment, the target image area corresponding to the indicator light includes the whole indicator light (such as the green circular indicator light 21 in Figure 2 ) and the image area corresponding to the digital display part of the display screen (such as the red digital part 22 on the display screen in Figure 2 ).

[0067] S103: Use the open-door state binary image as a mask for the closed-door state image to acquire a filling coefficient corresponding to each indicator light mask area in the open-door state binary image. The filling coefficient Gn corresponding to the nth indicator light mask area satisfies the following condition:

[0068] Gn=G z n / G m n .

[0069] wherein G z n is the area of the indicator light image region in the nth indicator light mask region in the open door state binary image. m n is the image area of the nth indicator light mask region in the open door state binary image.

[0070] S104: If the filling coefficient corresponding to any one of the indicator light mask regions in the open door state binary image is less than the filling threshold, the world coordinate corresponding to the center of the indicator light mask region is taken as the coordinate of the first type of abnormal region.

[0071] In the embodiment, the first type of abnormal region mainly refers to the abnormality caused by the cabinet door shielding some small point light sources and screen display numbers, and thus failing to obtain all the indicator light images. Since the point light sources and screen display numbers in the embodiment are relatively small indicator lights, even if the mesh wall of the mesh cabinet door is thin, the shielding ratio of the above-mentioned types of indicator lights is still relatively large. Therefore, the abnormal region caused by the shielding is determined based on the filling coefficient.

[0072] S105: The center coordinate of the observation hole closest to the coordinate of the first type of abnormal region in the equivalent observation plane of the cabinet door is taken as the coordinate corresponding to the first type of abnormal target acquisition point.

[0073] Since the coordinate corresponding to the abnormal target acquisition point determined finally is used to determine the position of the camera device 1, in order to ensure that the camera device 1 can be close to the shooting position without interfering with the cabinet door, the equivalent observation plane of the cabinet door is arranged in parallel and spaced apart from the cabinet door. The coordinate corresponding to the abnormal target acquisition point in the embodiment can be the point to which the lens center position of the camera device 1 is controlled to reach.

[0074] In addition, the abnormal region also includes a false indicator light image region formed by scattered light leakage, such as Figure 3 the false indicator light image region 3 in FIG. 1.

[0075] After the open door state binary image and the closed door state binary image are generated, the method for acquiring the abnormal target acquisition point further includes:

[0076] S111: The closed door state binary image is used as a mask for the open door state image. If there is no indicator light image in any one of the indicator light mask regions in the closed door state binary image, the world coordinate corresponding to the center of the indicator light image closest to the indicator light mask region in the open door state image is taken as the coordinate of the second type of abnormal region.

[0077] S112: The observation hole center coordinate closest to the second type of abnormal region coordinate in the equivalent observation plane of the cabinet door is taken as the coordinate corresponding to the second type of abnormal target acquisition point.

[0078] For the false indicator light image area 3 formed by scattered light leakage, the false indicator light image caused by the leakage observation hole only exists in the closed state binary image, so if there is no indicator light image in any indicator light mask area in the closed state binary image, it can be determined that the position is the false indicator light image area 3. Generally, the false indicator light image is formed by the light emitted by the real indicator light due to multiple diffuse reflection and even the mixing of light diffraction and scattering, so the false indicator light image only appears in the area close to the real indicator light. Therefore, the world coordinate corresponding to the indicator light image center closest to the indicator light mask area in the open state image is taken as the second type of abnormal region coordinate.

[0079] S200: According to the difference between the world coordinate corresponding to each auxiliary positioning sticker center pixel position and the world coordinate corresponding to the preset center position, the primary displacement information between the current position of the camera device 1 and the initial target acquisition point is generated. The primary displacement information and the secondary displacement information are used to determine the adjustment increment of each position adjustment axis in the multi-axis adjustment assembly corresponding to the camera device 1.

[0080] When the camera device 1 obtains the state image of the cabinet door at the current position, the position of the auxiliary positioning sticker can be detected using a computer vision algorithm (such as edge detection or template matching), and the center pixel position coordinates of each auxiliary positioning sticker can be extracted. Then, the pixel coordinates are converted into real world coordinates through inverse projection through the camera projection model (usually pinhole camera model).

[0081] After obtaining the world coordinates corresponding to the center position of each auxiliary positioning sticker at the current position, the adjustment increment of each adjustment axis can be determined according to the average value of the difference between the world coordinate corresponding to each auxiliary positioning sticker center pixel position and the world coordinate corresponding to the preset center position.

[0082] For example, there are three auxiliary positioning stickers, and the adjustment increment ΔX of the X-axis direction satisfies the following conditions: ΔX = (X1-X1 target +X2-X2 target +X3-X3 target ) / 3, where X1 target , X2 target , X3 target are the world coordinates corresponding to the preset center positions of the three auxiliary positioning stickers, and X1, X2, X3 are the world coordinates corresponding to the center pixel position coordinates of the three auxiliary positioning stickers in the current image.

[0083] S300: According to the order of the primary displacement information and the secondary displacement information to reach the node in the preset camera inspection path, the camera device 1 is controlled to move in turn, and the state information map of the corresponding area of the cabinet is collected after each displacement information is executed.

[0084] In the embodiment, the abnormal target collection point corresponding to the image area of the indicator light shielded is arranged at a position close to the cabinet door, so that the lens of the camera device 1 located at the abnormal target collection point is located at the corresponding observation hole of the cabinet door. That is, the camera device 1 can be close to the observation hole to take a picture of a certain local part. At this time, since the camera device 1 can avoid the shielding of the honeycomb metal in the field of view, the image information of the shielded indicator light can be more clearly obtained, so that the related state information of the cabinet can be identified through the visible light display information of the indicator light.

[0085] Meanwhile, in the embodiment, the initial target collection point of the camera device 1 during each inspection is positioned by pasting a plurality of auxiliary positioning stickers on the cabinet door. In this way, the position of the initial collection point during each subsequent inspection is ensured to be exactly the same as the position of the initial collection point set in the previous preparation stage, so that the angle of view range of the image taken during each subsequent inspection is ensured to be the same as the angle of view range specified in the previous stage. Since the positions of the devices in the cabinet do not change frequently, the positions of the indicator lights that may be shielded when taking pictures at the initial collection point are also stable and unchanged. Therefore, the initial target collection point and the abnormal target collection point can be used to form a preset camera inspection path, and the corresponding displacement information can be stored in the two-dimensional code. The initial target collection point is used to collect a full map of the state information of the cabinet, and the abnormal target collection point is used to obtain a local map of the local state information. When the camera device 1 performs inspection, the initial target collection point can be directly positioned according to the information in the two-dimensional code, and the camera device 1 can be moved and photographed according to the preset camera inspection path, so that the image information of each state indicator light in the cabinet can be clearly and comprehensively collected.

[0086] Since the devices in the server cabinet will inevitably be maintained and inspected in the long-term use process, after the devices are repositioned in the cabinet after the artificial maintenance is completed, the position of the devices may be slightly displaced. In this case, the indicator light after displacement may be abnormal, such as being shielded or causing a false indicator light image. Therefore, the coordinates corresponding to the abnormal target collection point need to be updated.

[0087] In another embodiment of the application, the two-dimensional code also stores an updated closed-door state binary map corresponding to an update period of the cabinet, and the cabinet state information acquisition method further comprises:

[0088] S400: If the update instruction is obtained, the camera device 1 is controlled to acquire the target cabinet closed door image at the initial target acquisition point at the current update time.

[0089] The update instruction in this step can be an update instruction triggered according to a preset period, or an update instruction manually triggered.

[0090] S500: The target cabinet closed door image acquired at the current update time is subjected to image region segmentation processing to generate an update closed door state binary image corresponding to the current update time. The pixel value of the target image region corresponding to the indicator light in the update closed door state binary image corresponding to the current update time is 0, and the pixel value corresponding to the background image region is 255.

[0091] S600: The update closed door state binary image corresponding to the current update time of the cabinet and the update closed door state binary image corresponding to the previous update period are subjected to pixel comparison processing to generate at least one position change image region in which the update closed door state binary image corresponding to the current update time and the update closed door state binary image corresponding to the previous update period exist pixel differences.

[0092] Specifically, the pixel comparison processing includes:

[0093] S601: The update closed door state binary image corresponding to the current update time and the update closed door state binary image corresponding to the previous update period are subjected to image alignment.

[0094] S602: The two images are compared pixel by pixel to generate a position change image region. The pixel-by-pixel comparison satisfies the following conditions:

[0095]

[0096] wherein, diff (x,y) is the pixel value after comparing the two pixels at the pixel coordinates (x, y). imgD (x,y) is the pixel value of the pixel at the pixel coordinates (x, y) in the update closed door state binary image corresponding to the current update time. imgS (x,y) is the pixel value of the pixel at the pixel coordinates (x, y) in the update closed door state binary image corresponding to the previous update period.

[0097] S700: The center coordinates of the observation hole closest to the center coordinates of the position change image region in the equivalent observation plane of the cabinet door are taken as the coordinates corresponding to the third type of abnormal target acquisition point.

[0098] In the embodiment, the current acquired update closed door state binary image and the update closed door state binary image obtained in the last update cycle are subjected to pixel comparison processing. Specifically, if the pixel value of the current acquired update closed door state binary image is the same as the pixel value of the corresponding position in the update closed door state binary image obtained in the last update cycle, the pixel value result after comparison is 255; if different, the pixel value result after comparison is 0. In the embodiment, the image region with a pixel value of 0 can be determined as a position change image region.

[0099] After the pixel comparison processing, the positions with different pixel values before and after the movement of the device are determined as abnormal regions, such as the indicator light region before the movement and the image region corresponding to the background region after the movement, or the background region before the movement and the image region corresponding to the indicator light region after the movement. These position change image regions are basically distributed in the range of the movement of the device. That is, the coordinates corresponding to the third type of abnormal target collection points of these position change image regions can circumscribe the region where the device has changed position. In the movement region, new abnormal situations caused by new indicator light occlusion or new false indicator light images caused by new light leakage may occur. Therefore, the close-up observation of the change region is required. Specifically, these third type of abnormal target collection points formed by the position change image regions can be used as new abnormal target collection points to control the camera device 1 to perform close-up observation at these points to obtain more clear partial images at the local change positions.

[0100] Further, after the coordinates corresponding to the third type of abnormal target collection points are generated, the method further includes:

[0101] S810: If the minimum distance between the third type of abnormal target collection point and the initial target collection point or the abnormal target collection point is greater than the field of view radius of the camera device 1, the third type of abnormal target collection point is determined as a new abnormal target collection point.

[0102] The field of view radius L of the camera device 1 satisfies the following condition:

[0103] Wherein, H is the distance between the lens of the camera device 1 and the indicator light inside the cabinet according to the equivalent plane, and FOV is the field of view angle of the lens of the camera device 1.

[0104] S900: Generate a new preset camera inspection path according to the updated initial target collection point and abnormal target collection point.

[0105] S820: If the minimum distance between the third type of abnormal target collection point and the initial target collection point or the abnormal target collection point is less than the field of view radius of the camera device 1, the third type of abnormal target collection point is deleted.

[0106] After the region of the device movement is determined through the coordinates corresponding to the third type of abnormal target collection point, the abnormal region may change before and after the movement, so the position region needs to be observed,

[0107] In steps S810, S900 and S820, whether the field of view range of the camera device 1 at the existing initial target collection point or abnormal target collection point can cover the position change image region, to further screen the third type of abnormal target collection point. Specifically, if the minimum distance between the third type of abnormal target collection point and the initial target collection point or the abnormal target collection point is greater than the field of view radius of the camera device 1, the third type of abnormal target collection point is determined as a new abnormal target collection point, otherwise the third type of abnormal target collection point is deleted. Thus, if the existing close-up shooting point can cover the new abnormal region, the abnormal target collection point corresponding to the abnormal region is not added, and if the existing close-up shooting point cannot cover the new abnormal region, the abnormal target collection point corresponding to the abnormal region is added. Further, the number of repeated shooting of the same abnormal local region in the image collected in the later inspection can be reduced.

[0108] In addition, since the placement shelves of the devices in the cabinet are basically fixed, the slight displacement of the device position caused by manual maintenance will only occur in the horizontal direction. In this case, in order to further reduce the number of repeated shooting of the same abnormal local region, the following scheme can be used to determine the third type of abnormal target collection point from the plurality of position change image regions generated in S600.

[0109] The specific steps are as follows:

[0110] According to the maximum distance D between the center positions of the plurality of position change image regions max and L, the number W of the third type of abnormal target collection point is determined. W satisfies the following conditions:

[0111] The line between the center positions of the two position change image regions with the farthest distance is divided into W-1 segments, and the end points of each segment are taken as the displacement target collection points.

[0112] The center coordinate of the observation hole closest to the coordinate distance of the displacement target collection point in the equivalent observation plane of the cabinet door is taken as the coordinate corresponding to the third type of abnormal target collection point.

[0113] Due to the slight misalignment in the horizontal direction (such as 1-2 CM), the indicator light position will be offset from the original position, and after the pixel comparison process, a relatively dense transverse arrangement of position variation image areas will be generated in the displacement area. If each position variation image area is determined as a third type of abnormal target collection point, it will inevitably cause the abnormal target collection points to be too dense, and further cause repeated image collection of the same area. In the embodiment, by means of a larger field of view radius L, the maximum displacement of the entire offset interval is divided into several segments, and the endpoints of each segment are taken as the displacement target collection points. Therefore, not only can the number of abnormal target collection points be greatly reduced, but also the distance between any two adjacent third type of abnormal target collection points can be ensured to be less than the field of view radius L of the camera device 1, thereby ensuring the comprehensiveness of the image collection of the position variation area.

[0114] In addition, although the various steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. In addition or alternatively, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be divided into multiple steps, etc.

[0115] From the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) execute the method according to the embodiments of the present disclosure.

[0116] In the example embodiments of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0117] Those skilled in the art can understand that each aspect of the present disclosure can be implemented as a system, a method or a program product. Therefore, each aspect of the present disclosure can be embodied in the form of a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.

[0118] The electronic device according to this embodiment of the present disclosure. The electronic device is only an example, and should not limit the function and use range of the embodiments of the present disclosure.

[0119] The electronic device is in the form of a general purpose computing device. Components of the electronic device can include, but are not limited to, the at least one processor described above, the at least one memory described above, a bus that connects the various system components including the memory and the processor.

[0120] The memory stores a program code that can be executed by the processor, such that the processor performs the steps described in the above "Exemplary Methods" section of the specification according to various exemplary embodiments of the present application.

[0121] The memory can include a readable medium in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and can further include read only memory (ROM).

[0122] The memory can also include a program / utility having a set (at least one) of program modules that include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each or a combination thereof, which may

[0123] The bus can be representative of one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or local bus using any of a variety of bus structures.

[0124] The electronic device can also communicate with one or more external devices (e.g., a keyboard or a pointing device, etc.) that can be coupled thereto, and / or one or more devices that enable a user to interact with the electronic device (e.g., a display, a remote control device, a mouse, a keypad, a microphone, etc.), and / or one or more devices that enable the electronic device to communicate with one or more other computing devices. Such communication can occur via an input / output (I / O) interface. Still yet, the electronic device can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or the Internet) through a network adapter. It should be appreciated that the network adapter can also be utilized to enable the electronic device to communicate with other electronic devices or devices of the system 100 that are not explicitly shown, which can include through any appropriate connectivity and wirelessly (such as via wireless networking, TTY, etc.). The network adapter can be implemented as part of the electronic device or as a separate component that is coupled thereto. Further, the electronic device can include an infrared port (not shown) in electrical communication with the bus. The infrared port can be used to communicate to / from the electronic device with any infrared-enabled devices that are within a range of the infrared port.

[0125] Those skilled in the art can clearly understand, through the description of the above embodiments, that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes a plurality of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to perform the method according to the embodiments of the present disclosure.

[0126] In the example embodiments of the present disclosure, a computer-readable storage medium having a program product stored thereon capable of implementing the above-mentioned method of the present disclosure is also provided. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code for causing a terminal device to perform the steps according to various example embodiments of the present disclosure described in the above-mentioned “example method” section of the present disclosure when the program product is run on the terminal device.

[0127] The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0128] The computer-readable signal medium can include a data signal propagated in a baseband or as a part of a carrier wave, in which readable program code is borne. Such a propagated data signal can take on multiple forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable signal medium can also be any readable medium other than the readable storage medium, which can send, propagate or transmit a program for use by or in connection with an instruction execution system, device or component.

[0129] The program code contained on the readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0130] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0131] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0132] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0133] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method of acquiring cabinet status information, characterized by, The method comprises the following steps: According to the target cabinet closed door image collected by the camera device in the inspection area, the world coordinates corresponding to each auxiliary positioning sticker center pixel position in the target cabinet closed door image are obtained; the image of one auxiliary positioning sticker includes a two-dimensional code, and the two-dimensional code stores the world coordinates corresponding to the preset center position of each auxiliary positioning sticker and the secondary displacement information corresponding to two adjacent nodes in the preset camera inspection path; The nodes in the preset camera inspection path include the initial target acquisition point of the camera device and the abnormal target acquisition point corresponding to the abnormal area in the initial acquisition image; The lens of the camera device located at the abnormal target acquisition point is located at the corresponding observation hole on the cabinet door, and the distance between the abnormal target acquisition point and the plane of the cabinet door is less than the distance between the initial target acquisition point and the plane of the cabinet door; the abnormal target acquisition point is used to acquire the local state information image of the cabinet; the initial target acquisition point is used to acquire the state information full image of the cabinet; the abnormal area includes the image area of the indicator light being shielded; According to the difference between the world coordinates corresponding to each auxiliary positioning sticker center pixel position and the world coordinates corresponding to the preset center position, the primary displacement information between the current position of the camera device and the initial target acquisition point is generated; The primary displacement information and the secondary displacement information are used to determine the adjustment increment of each position adjustment axis in the multi-axis adjustment assembly corresponding to the camera device; According to the order of the nodes to be reached in the preset camera inspection path respectively according to the primary displacement information and the secondary displacement information, the camera device is controlled to move in turn, and the state information image of the corresponding area of the cabinet is acquired after each displacement information is executed; The abnormal target acquisition point corresponding to the abnormal area in the initial acquisition image is obtained according to the following steps: Control the camera device at the initial target acquisition point to acquire the open door state image and the closed door state image of the target cabinet respectively; Perform image region segmentation processing on the open door state image and the closed door state image respectively to generate open door state binary image and closed door state binary image; the pixel value of the target image area corresponding to the indicator light in the binary image is 0, and the pixel value of the background image area is 255; Use the open door state binary image as the mask of the closed door state image to obtain the filling coefficient corresponding to each indicator light mask area in the open door state binary image; wherein the filling coefficient Gn corresponding to the nth indicator light mask area satisfies the following conditions: Gn = G z n / G m n ; wherein G z n is the area of the indicator light image region in the nth indicator light mask region of the open door state binary image; G m n is the image area of the nth indicator light mask region in the open door state binary image; If there is any indicator light mask area in the open door state binary image whose filling coefficient is less than the filling threshold, the world coordinates corresponding to the center of the indicator light mask area are taken as the coordinates of the first type of abnormal area; The center coordinates of the observation hole in the equivalent observation plane of the cabinet door closest to the first type of abnormal area coordinates are taken as the coordinates corresponding to the first type of abnormal target acquisition point; the equivalent observation plane of the cabinet door is parallel and spaced apart from the cabinet door.

2. The method of claim 1, wherein, The abnormal area also includes the false indicator light image area formed by scattered light leakage; After generating the open door state binary image and the closed door state binary image, the method further comprises: The world coordinates corresponding to the center of the indicator light image closest to the indicator light mask area in the open door state image are taken as the coordinates of the second type of abnormal area if there is no indicator light image in any indicator light mask area in the closed door state binary image; The coordinates of the third type of abnormal target collection point are taken as the coordinates of the center of the observation hole closest to the second type of abnormal area coordinates in the equivalent observation plane of the cabinet door.

3. The method of claim 1, wherein, The coordinates of the third type of abnormal target collection point are taken as the coordinates of the center of the observation hole closest to the second type of abnormal area coordinates in the equivalent observation plane of the cabinet door. The method further comprises:

4. The method of claim 3, wherein, The initial target collection point and the abnormal target collection point are subjected to path planning processing based on the nearest neighbor algorithm to generate the preset camera inspection path. The method further comprises: If an update instruction is obtained, the camera device is controlled to collect the target cabinet closed door image at the initial target collection point at the current update time; The target cabinet closed door image collected at the current update time is subjected to image region segmentation processing to generate the update closed door state binary image corresponding to the current update time; the pixel value of the target image region corresponding to the indicator light in the update closed door state binary image corresponding to the current update time is 0, and the pixel value of the background image region is 255; The update closed door state binary image corresponding to the current update time of the cabinet is subjected to pixel comparison processing with the update closed door state binary image corresponding to the last update period to generate at least one position change image region in which the update closed door state binary image corresponding to the current update time and the update closed door state binary image corresponding to the last update period exist pixel differences.

5. The method of claim 4, wherein, The coordinates of the third type of abnormal target collection point are taken as the coordinates of the center of the observation hole closest to the second type of abnormal area coordinates in the equivalent observation plane of the cabinet door. The pixel comparison processing comprises: The update closed door state binary image corresponding to the current update time and the update closed door state binary image corresponding to the last update period are subjected to image alignment; ； wherein, diff （x,y） is the pixel value of the pixel with pixel coordinate (x, y) in the updated closed door state binary image corresponding to the current update moment; imgD （x,y） is the pixel value of the pixel with pixel coordinate (x, y) in the updated closed door state binary image corresponding to the current update moment; imgD （x,y） is the pixel value of the pixel with pixel coordinate (x, y) in the updated closed door state binary image corresponding to the current update moment; imgD 6. The method of claim 4, wherein, The position change image region is generated by comparing the pixels of the two images; the pixel-by-pixel comparison satisfies the following conditions: After the coordinates of the third type of abnormal target collection point are generated, the method further comprises: The field of view radius L of the camera device satisfies the following condition: ; If the minimum distance between the third type of abnormal target collection point and the initial target collection point or the abnormal target collection point is greater than the field of view radius of the camera device, the third type of abnormal target collection point is determined as a new abnormal target collection point. Wherein, H is the distance between the lens of the camera device and the indicator light in the cabinet according to the equivalent plane, and FOV is the field of view angle of the lens of the camera device; 7. The method of claim 6, wherein, A new preset camera inspection path is generated according to the updated initial target collection point and the abnormal target collection point. After the coordinates of the third type of abnormal target collection point are generated, the method further comprises: If the minimum distance between the third type of abnormal target collection point and the initial target collection point or the abnormal target collection point is less than the field of view radius of the camera device, the third type of abnormal target collection point is deleted. 8.A non-transitory computer-readable storage medium storing a computer program, the computer program comprising instructions configured to cause a processor to perform the method according to any one of claims 1 to 7. The computer program, when executed by a processor, implements the cabinet state information acquisition method according to any one of claims 1 to 7.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the cabinet state information acquisition method according to any one of claims 1 to 7.

Images