Equipment panel inspection method and system

Abstract

The application discloses a device panel inspection method and system, and belongs to the technical field of device inspection. The technical problem to be solved is how to detect and analyze information modules on a device panel to infer a device running state. The method comprises the following steps: obtaining a device panel image to obtain a standard image; performing image preprocessing to obtain an enhanced standard image; extracting a plurality of minimum markable units from the enhanced standard image, marking the minimum markable units based on prior knowledge, and constructing a data model; obtaining a to-be-detected device panel image as a real-time image, selecting a region of interest from the target image based on the data model, and obtaining a real-time standard image corresponding to the region of interest; taking the real-time standard image as a target image, performing image preprocessing to obtain an enhanced real-time standard image; correlating and comparing each information module of the enhanced real-time standard image based on the data model, and forming a report based on running state information of each information module.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment inspection, in particular to an inspection method and system for an equipment panel. Background Art

[0002] Inspection and analysis of equipment panels is a common task in computer rooms and data centers. For example, various devices, such as server front panels and distribution box front panels, often feature visual or interactive information such as indicator lights, small displays, buttons, and meters. Extracting information from these panels is a key capability for computer room inspection robots. Typically, the robot moves around the room, capturing the panel contents with a camera and transmitting the images or video data to the backend for analysis. Analysis algorithms leverage images and prior knowledge to identify valuable inspection information and generate inspection reports for use by operations and maintenance personnel. Automated detection of various information modules, such as indicator lights, meters, buttons, and screens, is challenging given the complexity and diversity of these panels. Once these modules are detected, intelligently extracting information from each module and analyzing the device's operating status presents a formidable challenge.

[0003] How to detect and analyze the information modules on the equipment panel to infer the equipment's operating status is a technical problem that needs to be solved. Summary of the Invention

[0004] The technical task of the present invention is to address the above shortcomings and provide a method and system for inspecting a device panel to solve the technical problem of how to detect and analyze information modules on the device panel.

[0005] In a first aspect, a method for inspecting a device panel of the present invention comprises the following steps:

[0006] The device panel image is obtained as the original image, the original image is used as the target image, a region of interest is selected from the target image, and rectangular correction is performed on the region of interest to obtain a standard image corresponding to the region of interest;

[0007] Taking the standard image as the target image, performing image preprocessing on the target image to obtain an enhanced standard image;

[0008] Taking the information module in the device panel as the minimum annotable unit, extracting multiple minimum annotable units from the enhanced standard image, annotating the minimum annotable units based on prior knowledge, and constructing a data model based on the minimum annotable units and the corresponding annotation information;

[0009] Acquire an image of the panel of the device to be inspected as a real-time image, use the real-time image as a target image, select a region of interest from the target image based on a data model, and perform rectangular correction on the region of interest to obtain a real-time standard image corresponding to the region of interest, wherein the real-time standard image has the same size as the standard image;

[0010] Taking the real-time standard image as the target image, performing image preprocessing on the target image to obtain an enhanced real-time standard image;

[0011] Based on the data model, correlation comparison is performed on each information module of the enhanced real-time standard image to obtain operation status information of each information module, and a report is generated based on the operation status information of each information module.

[0012] Preferably, selecting a region of interest from a target image and performing rectangular correction on the region of interest comprises the following steps:

[0013] Marking four intersection points in the target image, wherein the four intersection points form a quadrilateral, and the quadrilateral can cover all relevant information modules of the device panel;

[0014] Correcting the image content covered by the quadrilateral to a rectangle, and mapping each of the four diagonal corners of the quadrilateral to the corresponding diagonal corner of the rectangle to obtain a homography matrix for transforming the quadrilateral to the rectangle;

[0015] Transform a quadrilateral image region into a rectangular image region based on a homography matrix.

[0016] Preferably, the target image is preprocessed by performing histogram equalization on the target image, and after the preprocessing, the brightness and color of the target image meet predetermined conditions.

[0017] Preferably, the annotation information includes perception information and device operation knowledge, wherein the perception information includes number, type and location, and the device operation knowledge is the association information between the external characteristics of the device and the device operation status;

[0018] The number is a unique identification code corresponding to the smallest marking unit;

[0019] The position is the position of the smallest annotable unit in the corresponding image;

[0020] Said types include indicators, buttons, screens, pointers and dials;

[0021] For the smallest labelable unit of indicator light type, the device operation knowledge includes shape, color, on / off, flashing, and status meaning;

[0022] For the smallest annotable unit of button type, the device operation knowledge includes shape and state;

[0023] For the smallest annotable unit of type screen, the device operation knowledge includes a value range and anomaly definition;

[0024] For the smallest labelable unit of the dial type, the device operation knowledge includes a numerical range.

[0025] Preferably, performing correlation comparison on each information module of the real-time standard image based on the data model comprises the following steps:

[0026] Traverse the annotation information in the data model;

[0027] For each annotation information, the area corresponding to the annotation information is extracted from the enhanced real-time standard image as the target area according to the position information, and the corresponding perception information in the target area is extracted according to the different types in the annotation information. The perception information is then correlated and compared with the equipment operation knowledge in the data model to obtain the corresponding operation status information.

[0028] In a second aspect, the present invention provides a device panel inspection system, configured to execute the device panel inspection method according to any one of the first aspects, the system comprising:

[0029] An image acquisition module, which is used to photograph the device panel used as a template to obtain an original image, and to photograph the device panel to be inspected to obtain a real-time image;

[0030] An original image correction module is used to select a region of interest from the original image and perform rectangular correction on the region of interest to obtain a standard image corresponding to the region of interest;

[0031] An original image preprocessing module, configured to perform image preprocessing on the standard image to obtain an enhanced standard image;

[0032] A data model construction module, the data model construction module is used to extract multiple minimum labelable units from the enhanced standard image using the information module in the device panel as the minimum labelable unit, label the minimum labelable units based on prior knowledge, and construct a data model based on the minimum labelable units and the corresponding labeling information;

[0033] A real-time image correction module, configured to select a region of interest from a real-time image based on a data model, and perform rectangular correction on the region of interest to obtain a real-time standard image corresponding to the region of interest, wherein the real-time standard image has the same size as the standard image;

[0034] A real-time image preprocessing module, configured to perform image preprocessing on the real-time standard image to obtain an enhanced real-time standard image;

[0035] A comparison and analysis module is used to compare the information modules of the enhanced real-time standard image based on the data model, obtain the operating status information of each information module, and generate a report based on the operating status information of each information module.

[0036] Preferably, the original image correction module and the real-time image correction module are both used to perform rectangular correction on the region of interest through the following operations:

[0037] Marking four intersection points in the target image, wherein the four intersection points form a quadrilateral, and the quadrilateral can cover all relevant information modules of the device panel;

[0038] Correcting the image content covered by the quadrilateral to a rectangle, and mapping each of the four diagonal corners of the quadrilateral to the corresponding diagonal corner of the rectangle to obtain a homography matrix for transforming the quadrilateral to the rectangle;

[0039] Transform a quadrilateral image region into a rectangular image region based on a homography matrix.

[0040] Preferably, the original image preprocessing module and the real-time image preprocessing module are both used to preprocess the image by means of histogram equalization, and the brightness and color of the preprocessed image meet predetermined conditions.

[0041] Preferably, the annotation information includes perception information and device operation knowledge, wherein the perception information includes number, type and location, and the device operation knowledge is the association information between the external characteristics of the device and the device operation status;

[0042] The number is a unique identification code corresponding to the smallest marking unit;

[0043] The position is the position of the smallest annotable unit in the corresponding image;

[0044] Said types include indicators, buttons, screens, pointers and dials;

[0045] For the smallest labelable unit of indicator light type, the device operation knowledge includes shape, color, on / off, flashing, and status meaning;

[0046] For the smallest annotable unit of button type, the device operation knowledge includes shape and state;

[0047] For the smallest annotable unit of type screen, the device operation knowledge includes a value range and anomaly definition;

[0048] For the smallest labelable unit of the dial type, the device operation knowledge includes a numerical range.

[0049] Preferably, the comparison and analysis module is configured to perform correlation comparison on each information module of the real-time standard image based on the data model through the following steps:

[0050] Traverse the annotation information in the data model;

[0051] For each annotation information, the area corresponding to the annotation information is extracted from the enhanced real-time standard image as the target area according to the position information, and the corresponding perception information in the target area is extracted according to the different types in the annotation information. The perception information is then correlated and compared with the equipment operation knowledge in the data model to obtain the corresponding operation status information.

[0052] The device panel inspection method and system of the present invention have the following advantages:

[0053] 1. Capture the device panel to obtain an original image, extract the region of interest from the original image, and correct the region of interest to obtain a standard image. Extract the minimum annotable unit from the standard image and annotate it based on prior knowledge. Build a data model based on each minimum annotable unit and the annotation information. Use this data model as a template to perform correlation analysis on the real-time images corresponding to the device panel to be inspected. This enables batch inspection of related device panels through a single data model, improving inspection speed and quality.

[0054] 2. Image preprocessing is performed on both original images and real-time images to improve precision and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] The present invention will be further described below with reference to the accompanying drawings.

[0057] Figure 1 This is a flowchart of a device panel inspection method according to Example 1;

[0058] Figure 2 This is a flow chart of constructing a data model in a device panel inspection method according to Example 1;

[0059] Figure 3A flowchart of a process of correlation analysis on real-time images based on a data model in a device panel inspection method of Embodiment 1. DETAILED DESCRIPTION

[0060] The present application will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present application and implement it, but the embodiments are not limiting to the present application, and the technical features in the embodiments and the embodiments can be combined with each other without conflict.

[0061] Embodiments of the present application provide a device panel inspection method and system, which are used to solve the technical problem of how to detect and analyze information modules on a device panel to infer the running state of the device.

[0062] Embodiment 1:

[0063] The device panel inspection method of the present application comprises the following steps:

[0064] S100, obtaining a device panel image as an original image, taking the original image as a target image, selecting a region of interest from the target image and performing rectangular correction on the region of interest to obtain a standard image corresponding to the region of interest;

[0065] S200, taking the standard image as a target image, performing image preprocessing on the target image to obtain an enhanced standard image;

[0066] S300, taking an information module in the device panel as a smallest labelable unit, extracting a plurality of smallest labelable units from the enhanced standard image, labeling the smallest labelable units based on prior knowledge, and constructing a data model based on the smallest labelable units and the corresponding labeling information;

[0067] S400, obtaining a to-be-detected device panel image as a real-time image, taking the real-time image as a target image, selecting a region of interest from the target image based on the data model, and performing rectangular correction on the region of interest to obtain a real-time standard image corresponding to the region of interest, the real-time standard image being consistent in size with the standard image;

[0068] S500, taking the real-time standard image as a target image, performing image preprocessing on the target image to obtain an enhanced real-time standard image;

[0069] S600, based on the data model, correlating and comparing each information module of the real-time standard image to obtain running state information of each information module, and forming a report based on the running state information of each information module.

[0070] Step S100: For a certain type of equipment that needs to be inspected, the panel is photographed to obtain a panel image, and the panel image is used as the original image. The original image is subjected to rectangular correction. As a specific implementation, it includes the following steps:

[0071] (1) Mark four intersection points in the target image, where the four intersection points form a quadrilateral that can cover all relevant information modules on the device panel;

[0072] (2) Correcting the image content covered by the quadrilateral to a rectangle, and mapping each of the four diagonal corners of the quadrilateral to the corresponding diagonal corner of the rectangle to obtain a homography matrix for transforming the quadrilateral to the rectangle;

[0073] (3) The quadrilateral image region is transformed into a rectangular image region based on the homography matrix.

[0074] That is, in this step, for a certain type of equipment that needs to be inspected, an image of its panel is captured, and four corner points are manually marked in the image. These four pixel points are selected so that the quadrilateral formed by these four points covers all inspection equipment as much as possible, and irrelevant content is isolated outside the quadrilateral as much as possible. Next, the image content covered by this quadrilateral is corrected to a rectangle, and the upper left corner of the quadrilateral is mapped to the upper left corner of the rectangle, and so on for the other three corners. This results in a homography matrix that transforms the quadrilateral to a rectangle. The homography matrix is ​​then used to transform the quadrilateral image area to a rectangle. At this point, a standard image of the region of interest (ROI) is obtained.

[0075] Step S200 is to pre-process the obtained standard image to ensure that the brightness and color of the image meet the requirements for subsequent labeling. As a specific implementation, the standard image is pre-processed by a histogram equalization method.

[0076] In step S300, the annotation information includes perception information and device operation knowledge. The perception information includes number, type and location. The device operation knowledge is the association information between the external features of the device and the device operation status. The number is the unique identification code of the corresponding minimum annotation unit. The location is the location of the minimum annotable unit in the corresponding image. The types include indicator lights, buttons, screens, pointers and dials. For the minimum annotable unit of the indicator type, the device operation knowledge includes shape, color, on and off, flashing and status meaning. For the minimum annotable unit of the button type, the device operation knowledge includes shape and status. For the minimum annotable unit of the screen type, the device operation knowledge includes value range and abnormality definition. For the minimum annotable unit of the dial type, the device operation knowledge includes value range.

[0077] In this step, the enhanced standard image is labeled, and the minimum labelable unit is artificially selected in the enhanced standard image. The purpose is to quickly construct the labeling information of the entire standard image by simply moving, copying, arranging, and combining these minimum labelable units. One or more same or different minimum labelable units constitute the labeling information of the entire standard image. The information labeled by each minimum labelable unit can include a label type (such as an indicator light, a button, a screen, a pointer, a dial, etc.), a position, a number, etc., and can even label personalized labeling content for different types of labeling units, such as labeling the actual meaning of different states such as shape, color, and on / off of an indicator light. By labeling the minimum unit and copying and expanding, the ROI template of the entire standard image is obtained, which contains a large amount of prior information, including two types, one is position, type, etc., and the other is the association information between the external characteristics of these modules and the overall running state of the device. The former is objective and perceptible perceptual information, and the latter is device operation knowledge, collectively referred to as the digital model of the device.

[0078] The steps S100-S300 cooperate to construct a data model of the device panel based on the selected device panel. As shown in Figure 2 The template image is photographed, the corner points are selected and the ROI is determined. The ROI is corrected, and one or more appropriate minimum labelable units are selected and labeled. Finally, the labels are arranged and combined to obtain the labeling template of the ROI. If the types of devices photographed between multiple task points are similar and the pictures are similar, the template can be labeled only once. Otherwise, a corresponding template is labeled for each different task point. The templates are persistently stored.

[0079] The construction of the data model is realized based on the above steps, and the device panel to be detected can be detected based on the above data model.

[0080] In step S400, for the type of device panel, the real-time image of the device running is obtained by photographing the image each time the inspection is performed.

[0081] In step S500, the real-time image obtained is corrected and the ROI region is extracted by positioning the four corner points of the real-time image, to obtain an image consistent in size with the ROI template. The sequential steps are as follows:

[0082] (1) Labeling four intersection points in the target image, the four intersection points forming a quadrilateral, the quadrilateral being able to cover all relevant information modules of the device panel;

[0083] (2) correcting the image content covered by the quadrilateral to a rectangle, and for the four diagonals of the quadrilateral, each diagonal is corresponded to a related diagonal of the rectangle, to obtain a homography matrix from the quadrilateral to the rectangle;

[0084] (3) The quadrilateral image region is transformed into a rectangular image region based on the homography matrix.

[0085] After obtaining the real-time standard image corresponding to the real-time image, step S500 is executed to perform image preprocessing on the real-time standard image. As a specific implementation, the preprocessing is performed by a histogram equalization processing method to obtain an enhanced real-time standard image.

[0086] In step S600, based on the data model, the information modules of the enhanced real-time standard image are correlated and compared, including the following steps: traversing the annotation information in the data model; for each annotation information, extracting the area corresponding to the annotation information from the enhanced real-time standard image as the target area according to the position information, and extracting the corresponding perception information in the target area according to the different types in the annotation information, and correlating and comparing the perception information with the equipment operation knowledge in the data model to obtain the corresponding operation status information.

[0087] Steps S400 to S600 are to associate templates with each task point of the inspection robot. The inspection robot needs to take an image at each task point and then compare it with the associated template to obtain the equipment operation status, as shown in the attached figure. Figure 2 shown.

[0088] Example 2:

[0089] The present invention provides an inspection system for equipment panels, including an image acquisition module, an original image correction module, an original image preprocessing module, a data model construction module, a real-time image correction module, a real-time image preprocessing module and a comparison and analysis module. The system can execute the method disclosed in Example 1 to inspect the equipment panel.

[0090] The image acquisition module is used to photograph the device panel used as a template to obtain an original image, and is used to photograph the device panel to be inspected to obtain a real-time image.

[0091] For the equipment panels that need to be inspected, the image acquisition module calls the camera to shoot the equipment panel and obtains the equipment panel image as the original image. Subsequently, for each equipment panel that needs to be inspected, the image acquisition module calls the camera to shoot the equipment panel to be inspected and obtains the equipment panel image as the real-time image.

[0092] The original image correction module is used to select the region of interest from the original image and perform rectangular correction on the region of interest to obtain the standard image corresponding to the region of interest. The specific operations of this module are:

[0093] (1) Mark four intersection points in the target image, where the four intersection points form a quadrilateral that can cover all relevant information modules on the device panel;

[0094] (2) correcting the image content covered by the quadrangle to a rectangle, for the four opposite corners of the quadrangle, corresponding each opposite corner to the relevant opposite corner of the rectangle, to obtain a homography matrix of the quadrangle to the rectangle;

[0095] (3) transforming the quadrangle image region to a rectangle image region based on the homography matrix.

[0096] For some kind of equipment that needs to be inspected, the image acquisition module and the original image correction module cooperate to shoot the panel image, and mark four corner points in the image. The selection method of the four pixel points is to make the quadrangle formed by the four points cover all the inspection equipment as much as possible, and to isolate irrelevant content outside the quadrangle as much as possible. Next, the image content covered by the quadrangle is corrected to a rectangle, the upper left corner of the quadrangle is corresponded to the upper left corner of the rectangle, and the other three corners are corresponded in the same way, so as to obtain a homography matrix of the quadrangle to the rectangle. Then, the quadrangle image region is transformed to a rectangle by using the homography matrix. Thus, a standard image of a region of interest (ROI) is obtained.

[0097] The original image preprocessing module is used for image preprocessing of the standard image to obtain an enhanced standard image. The brightness and color of the image are ensured to meet the conditions, so as to facilitate subsequent labeling. As a specific implementation, the original image preprocessing module is configured with a histogram equalization processing method, and the module pre-processes the standard image by the histogram equalization processing method.

[0098] The data model construction module is used for taking the information module in the device panel as the smallest labelable unit, extracting a plurality of smallest labelable units from the enhanced standard image, labeling the smallest labelable units based on prior knowledge, and constructing a data model based on the smallest labelable units and the corresponding labeling information.

[0099] In this embodiment, the labeling information includes perception information and device operation knowledge. The perception information includes number, type and position, and the device operation knowledge is the associated information between the external characteristics of the device and the running state of the device. The number is a unique identification code of the smallest labelable unit. The position is the position of the smallest labelable unit in the corresponding image. The type includes indicator light, button, screen, pointer and dial. For the smallest labelable unit of the type of indicator light, the device operation knowledge includes shape, color, on-off, flicker and state meaning. For the smallest labelable unit of the type of button, the device operation knowledge includes shape and state. For the smallest labelable unit of the type of screen, the device operation knowledge includes numerical range and abnormal definition. For the smallest labelable unit of the type of dial, the device operation knowledge includes numerical range.

[0100] The real-time image correction module is used to select a region of interest (ROI) from the real-time image based on the data model and perform rectangular correction on the ROI to obtain a real-time standard image corresponding to the ROI. The real-time standard image is the same size as the standard image. The real-time image correction module performs rectangular correction on the ROI in the same manner as the original image correction module.

[0101] The real-time image preprocessing module is used to preprocess the real-time standard image to obtain an enhanced real-time standard image. The image preprocessing method used by the real-time image preprocessing module is consistent with that used by the original image preprocessing module. In practical applications, the real-time image preprocessing module and the original image preprocessing module can be used as a single image preprocessing module to preprocess the standard image and the real-time standard image, respectively.

[0102] The comparative analysis module is used to compare and correlate the various information modules of the enhanced real-time standard image based on the data model, obtain the operating status information of each information module, and generate reports based on the operating status information of each information module. The specific operations of this module are as follows: traversing the annotated information in the data model; for each annotated information, extracting the area corresponding to the annotated information from the enhanced real-time standard image based on the location information as the target area, and extracting the corresponding perception information in the target area based on the type of the annotated information. This perception information is then compared and correlated with the device operation knowledge in the data model to obtain the corresponding operating status information.

[0103] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the scope of protection of the present invention.

Claims

1. A method for inspecting a device panel, characterized in that The steps include: The device panel image is obtained as the original image, the original image is used as the target image, a region of interest is selected from the target image, and rectangular correction is performed on the region of interest to obtain a standard image corresponding to the region of interest; Taking the standard image as the target image, performing image preprocessing on the target image to obtain an enhanced standard image; Taking the information module in the device panel as the minimum labelable unit, extracting multiple minimum labelable units from the enhanced standard image, labeling the minimum labelable units based on prior knowledge, and building a data model based on the minimum labelable units and the corresponding labeling information; Acquire an image of the panel of the device to be inspected as a real-time image, use the real-time image as a target image, select a region of interest from the target image based on a data model, and perform rectangular correction on the region of interest to obtain a real-time standard image corresponding to the region of interest, wherein the real-time standard image has the same size as the standard image; Taking the real-time standard image as the target image, performing image preprocessing on the target image to obtain an enhanced real-time standard image; Based on the data model, correlation comparison is performed on each information module of the enhanced real-time standard image to obtain operation status information of each information module, and a report is generated based on the operation status information of each information module.

2. A device panel inspection method according to claim 1, characterized in that Selecting a region of interest from the target image and performing rectangular correction on the region of interest includes the following steps: Marking four intersection points in the target image, wherein the four intersection points form a quadrilateral, and the quadrilateral can cover all relevant information modules of the device panel; Correcting the image content covered by the quadrilateral to a rectangle, and mapping each of the four diagonal corners of the quadrilateral to the corresponding diagonal corner of the rectangle to obtain a homography matrix for transforming the quadrilateral to the rectangle; Transform a quadrilateral image region into a rectangular image region based on a homography matrix.

3. The inspection method for a device panel according to claim 1, characterized in that The target image is preprocessed by performing histogram equalization on the target image, and the brightness and color of the target image after preprocessing meet predetermined conditions.

4. A device panel inspection method according to any one of claims 1 to 3, characterized in that The annotation information includes perception information and device operation knowledge, wherein the perception information includes number, type and location, and the device operation knowledge is the association information between the external characteristics of the device and the device operation status; The number is a unique identification code corresponding to the smallest marking unit; The position is the position of the smallest annotable unit in the corresponding image; Said types include indicators, buttons, screens, pointers and dials; For the smallest labelable unit of indicator light type, the device operation knowledge includes shape, color, on / off, flashing, and status meaning; For a minimum annotable unit of type button, the device operation knowledge includes shape and state; For the smallest annotable unit of type screen, the device operation knowledge includes a value range and anomaly definition; For the smallest labelable unit of the dial type, the device operation knowledge includes a numerical range.

5. A device panel inspection method according to any one of claim 4, characterized in that Based on the data model, correlation comparison is performed on each information module of the real-time standard image, including the following steps: Traverse the annotation information in the data model; For each piece of annotation information, the area corresponding to the annotation information is extracted from the enhanced real-time standard image as the target area according to the position information, and the corresponding perception information in the target area is extracted according to the different types in the annotation information. The perception information is then correlated and compared with the equipment operation knowledge in the data model to obtain the corresponding operation status information.

6. A device panel inspection system, characterized in that A system for performing an inspection method for a device panel according to any one of claims 1 to 5, the system comprising: An image acquisition module, which is used to photograph the device panel used as a template to obtain an original image, and to photograph the device panel to be inspected to obtain a real-time image; An original image correction module is used to select a region of interest from the original image and perform rectangular correction on the region of interest to obtain a standard image corresponding to the region of interest; An original image preprocessing module, configured to perform image preprocessing on the standard image to obtain an enhanced standard image; a data model construction module, the data model construction module being configured to extract multiple minimum labelable units from the enhanced standard image using the information module in the device panel as the minimum labelable unit, label the minimum labelable units based on prior knowledge, and construct a data model based on the minimum labelable units and the corresponding labeling information; A real-time image correction module, configured to select a region of interest from a real-time image based on a data model, and perform rectangular correction on the region of interest to obtain a real-time standard image corresponding to the region of interest, wherein the real-time standard image has the same size as the standard image; A real-time image preprocessing module, configured to perform image preprocessing on the real-time standard image to obtain an enhanced real-time standard image; A comparison and analysis module is used to compare and correlate the various information modules of the enhanced real-time standard image based on the data model, obtain the operating status information of each information module, and form a report based on the operating status information of each information module.

7. The inspection system for equipment panels according to claim 6, characterized in that The original image correction module and the real-time image correction module are both used to perform rectangular correction on the region of interest through the following operations: Marking four intersection points in the target image, wherein the four intersection points form a quadrilateral, and the quadrilateral can cover all relevant information modules of the device panel; Correcting the image content covered by the quadrilateral to a rectangle, and mapping each of the four diagonal corners of the quadrilateral to the corresponding diagonal corner of the rectangle to obtain a homography matrix for transforming the quadrilateral to the rectangle; Transform a quadrilateral image region into a rectangular image region based on a homography matrix.

8. The inspection system for equipment panels according to claim 6, characterized in that The original image preprocessing module and the real-time image preprocessing module are both used to preprocess the image by means of histogram equalization processing, and the brightness and color of the preprocessed image meet predetermined conditions.

9. The inspection system for equipment panels according to any one of claims 6 to 8, characterized in that The annotation information includes perception information and device operation knowledge, wherein the perception information includes number, type and location, and the device operation knowledge is the association information between the external characteristics of the device and the device operation status; The number is a unique identification code corresponding to the smallest marking unit; The position is the position of the smallest annotable unit in the corresponding image; Said types include indicators, buttons, screens, pointers and dials; For the smallest labelable unit of indicator light type, the device operation knowledge includes shape, color, on / off, flashing, and status meaning; For a minimum annotable unit of type button, the device operation knowledge includes shape and state; For the smallest annotable unit of type screen, the device operation knowledge includes a value range and anomaly definition; For the smallest labelable unit of the dial type, the device operation knowledge includes a numerical range.

10. The inspection system for equipment panels according to claim 9, characterized in that The comparison and analysis module is used to perform correlation comparison on each information module of the real-time standard image based on the data model through the following steps: Traverse the annotation information in the data model; For each piece of annotation information, the area corresponding to the annotation information is extracted from the enhanced real-time standard image as the target area according to the position information, and the corresponding perception information in the target area is extracted according to the different types in the annotation information. The perception information is then correlated and compared with the equipment operation knowledge in the data model to obtain the corresponding operation status information.

Images