Inspection method and data center inspection method

Abstract

The embodiment of the invention provides an inspection method and a data center inspection method. The inspection method comprises the following steps: acquiring a target inspection image of a target inspection point; analyzing a plurality of target inspection areas and inspection information respectively corresponding to the plurality of target inspection areas from the target inspection image; the multiple target inspection areas are matched with multiple target template areas in the target template image, the corresponding relation between the target inspection areas and the target template areas is determined, and the target template areas are in one-to-one correspondence with the to-be-inspected objects; and determining the inspection information of the to-be-inspected object according to the inspection information and the corresponding relationship. By matching the target inspection area and the plurality of target template areas, automatic inspection is realized from the perspective of the area, the situation of large deviation between the target inspection image and the target template image can be dealt with on the basis of improving the inspection efficiency, and the anti-interference capability and the inspection accuracy of the inspection are further improved.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and particularly to an inspection method. Background Art

[0002] With the development of computer technology, equipment inspection has gradually become more and more important in work and life. The purpose of equipment inspection is to master the operating conditions of equipment and the changes in the surrounding environment, discover defects in facilities and potential safety hazards, take effective measures in a timely manner, reduce the chance of sudden failures, keep the equipment in good operating condition, and ensure the safety of the equipment and the stability of the system.

[0003] Currently, the traditional inspection method is mainly manual inspection, which is achieved by inspection personnel walking back and forth in the inspection area to observe. However, this method consumes labor costs and cannot be implemented through manual inspection in some special scenarios such as high temperature, lack of oxygen, and high risk. Moreover, due to the vast number of inspection points, manual inspection may introduce some uncertain factors, resulting in poor accuracy of inspection results. Therefore, there is an urgent need for an inspection scheme with high accuracy. Summary of the Invention

[0004] In view of this, the embodiments of this specification provide an inspection method. One or more embodiments of this specification simultaneously relate to a data center inspection method, an inspection device, a data center inspection device, a computing device, a computer-readable storage medium, and a computer program to solve the technical defects existing in the prior art.

[0005] According to the first aspect of the embodiments of this specification, an inspection method is provided, including:

[0006] Obtain a target inspection image of a target inspection point;

[0007] Parse out multiple target inspection areas and the inspection information corresponding to the multiple target inspection areas respectively from the target inspection image;

[0008] Match the multiple target inspection areas with the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection areas and the target template areas, where the target template areas correspond to the objects to be inspected one by one;

[0009] Determine the inspection information of the objects to be inspected according to the inspection information and the corresponding relationship.

[0010] According to the second aspect of the embodiments of this specification, a data center inspection method is provided, including:

[0011] Obtain a target inspection image of a target inspection point in the data center;

[0012] Parse out multiple target inspection areas and the inspection information corresponding to the multiple target inspection areas from the target inspection image;

[0013] Match the multiple target inspection areas with the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection areas and the target template areas, where the target template areas correspond to the data devices to be inspected one by one;

[0014] Determine the inspection information of the data devices to be inspected according to the inspection information and the corresponding relationship.

[0015] According to the third aspect of the embodiments of the present specification, there is provided an inspection device, including:

[0016] The first acquisition module is configured to acquire the target inspection image of the target inspection point;

[0017] The first parsing module is configured to parse out multiple target inspection areas and the inspection information corresponding to the multiple target inspection areas from the target inspection image;

[0018] The first matching module is configured to match the multiple target inspection areas with the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection areas and the target template areas, where the target template areas correspond to the objects to be inspected one by one;

[0019] The first determination module is configured to determine the inspection information of the objects to be inspected according to the inspection information and the corresponding relationship.

[0020] According to the fourth aspect of the embodiments of the present specification, there is provided a data center inspection device, including:

[0021] The second acquisition module is configured to acquire the target inspection image of the target inspection point in the data center;

[0022] The second parsing module is configured to parse out multiple target inspection areas and the inspection information corresponding to the multiple target inspection areas from the target inspection image;

[0023] The second matching module is configured to match the multiple target inspection areas with the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection areas and the target template areas, where the target template areas correspond to the data devices to be inspected one by one;

[0024] The second determination module is configured to determine the inspection information of the data devices to be inspected according to the inspection information and the corresponding relationship.

[0025] According to the fifth aspect of the embodiments of the present specification, there is provided a computing device, including:

[0026] Memory and processor;

[0027] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method provided in the above first aspect or second aspect are implemented.

[0028] According to a sixth aspect of the embodiments of the present specification, a computer-readable storage medium is provided, which stores computer-executable instructions. When the instructions are executed by a processor, the steps of the method provided in the above first aspect or second aspect are implemented.

[0029] According to a seventh aspect of the embodiments of the present specification, a computer program is provided. When the computer program is executed in a computer, the computer is made to execute the steps of the method provided in the above first aspect or second aspect.

[0030] The inspection method provided by an embodiment of the present specification includes: obtaining a target inspection image of a target inspection point; parsing out a plurality of target inspection areas and inspection information respectively corresponding to the plurality of target inspection areas from the target inspection image; matching the plurality of target inspection areas with a plurality of target template areas in a target template image to determine the corresponding relationship between the target inspection areas and the target template areas, where the target template areas correspond one-to-one with the objects to be inspected; and determining the inspection information of the objects to be inspected according to the inspection information and the corresponding relationship. By matching the target inspection areas with the plurality of target template areas, automatic inspection is realized from the perspective of areas, which can cope with the situation where there is a large deviation between the target inspection image and the target template image on the basis of improving the inspection efficiency, and further improves the anti-interference ability and inspection accuracy of the inspection. Description of the Drawings

[0031] Figure 1 is an architecture diagram of an inspection system provided by an embodiment of the present specification;

[0032] Figure 2 is a flowchart of an inspection method provided by an embodiment of the present specification;

[0033] Figure 3 is a schematic diagram for solving the corresponding relationship in an inspection method provided by an embodiment of the present specification;

[0034] Figure 4 is a flowchart of a data center inspection method provided by an embodiment of the present specification;

[0035] Figure 5 is a processing procedure flowchart of an inspection method provided by an embodiment of the present specification;

[0036] Figure 6It is a schematic structural diagram of an inspection device provided by an embodiment of this specification;

[0037] Figure 7 It is a schematic structural diagram of a data center inspection device provided by an embodiment of this specification;

[0038] Figure 8 It is a structural block diagram of a computing device provided by an embodiment of this specification. Specific implementation manners

[0039] In the following description, numerous specific details are set forth in order to provide a thorough understanding of this specification. However, this specification can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the connotation of this specification. Therefore, this specification is not limited by the specific implementations disclosed below.

[0040] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a", "the", and "said" used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more of the associated listed items.

[0041] It should be understood that although the terms first, second, etc. may be used in one or more embodiments of this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first can also be referred to as the second, and similarly, the second can also be referred to as the first. Depending on the context, the word "if" as used herein can be interpreted as "when" or "while" or "in response to determining".

[0042] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of the relevant data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entrances are provided for the user to select authorization or rejection.

[0043] First, the noun terms involved in one or more embodiments of this specification are explained.

[0044] Object Detection: Object detection refers to the task of classifying and locating a variable number of objects in an image.

[0045] YOLO: YOLO (You Only Look Once) is a real-time object detection algorithm that simplifies the object detection problem into a regression problem, enabling the simultaneous prediction of multiple objects, their bounding boxes, and class labels in an image.

[0046] Semantic Segmentation: Semantic segmentation refers to the task of classifying each pixel of an image into each instance (each instance corresponds to a class).

[0047] Optical Character Recognition: Optical Character Recognition (OCR) refers to the task of converting a text image into a machine-readable text format. OCR typically identifies characters by detecting the edges and textures of characters in an image and then converts them into readable text data.

[0048] Intersection over Union: Intersection over Union (IoU) is a metric for measuring the accuracy of object detection. It represents the degree of overlap between two regions, with a value range from 0 to 1, where 1 indicates complete overlap and 0 indicates no overlap.

[0049] Monte Carlo Method: The Monte Carlo Method (MCM), also known as the statistical simulation method or random sampling technique, is a stochastic simulation method based on probability and statistical theory.

[0050] Euclidean Distance: Euclidean Distance (EUC) refers to the distance between any two points in an M-dimensional space. It is one of the most commonly used distance metrics in Euclidean geometry and has wide applications in fields such as computer science and physics.

[0051] KM Algorithm: The KM Algorithm (Kuhn-Munkres Algorithm) is a combinatorial optimization algorithm for solving the weighted bipartite matching problem based on the Hungarian algorithm.

[0052] Gradient Descent Algorithm: The Gradient Descent Algorithm is commonly used for parameter estimation and function optimization in machine learning. Its basic principle is to iterate along the gradient direction of the loss function to minimize the loss function.

[0053] Common inspection robots on the market delimit areas based on template images to match inspection targets and template targets. Specifically, when deploying inspection points, an image is taken as the template image, and the position information of a certain target to be recognized in the template image is marked with a rectangular frame. During inspection, the inspection robot moves to the same position as the inspection point during deployment, takes an inspection image, and directly uses the position information of the target to be recognized recorded in the template image as the position information of the corresponding target to be recognized in the inspection image.

[0054] However, the above method assumes that the values of the position information of the target to be recognized in the inspection image and the template image are very close. If the deviation between the two is large, there may be no target to be recognized or only part of the target to be recognized in the corresponding rectangular frame in the inspection image, resulting in an incorrect matching result. Therefore, the greater the deviation between the inspection image and the template image, the lower the accuracy of the recognized target. Moreover, due to the complex inspection scenario in the data center and the narrow channels, it is easy to interfere with the positioning of the inspection robot, resulting in a large deviation between the inspection image and the template image, and the matching accuracy using the above method is relatively low. In addition, in the above method, a template image must be configured for each inspection point. For a data center, the number of inspection points for the inspection robot is extremely large. For example, there are hundreds of inspection points in a power distribution room, and configuring a template image for each inspection point during deployment results in a long total deployment time for the inspection points.

[0055] To solve the above problems, the embodiments of this specification propose a solution for matching the inspection area and the template area based on the object detection algorithm and the template matching algorithm, changing the factor determining the matching error from the positioning error to the errors of the object detection algorithm accuracy and the matching algorithm accuracy. Through algorithm optimization, high accuracy of the object detection algorithm and the matching algorithm can still be maintained in the case of large positioning errors, which can handle the situation where the deviation between the inspection image and the template image is large, and improve the anti-interference ability and accuracy of the inspection robot during inspection.

[0056] Specifically, the embodiments of this specification propose an inspection method, which includes obtaining the target inspection image of the target inspection point; parsing out multiple target inspection areas and the inspection information corresponding to the multiple target inspection areas from the target inspection image; matching the multiple target inspection areas with the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection area and the target template area, where the target template area corresponds to the object to be inspected one by one; and determining the inspection information of the object to be inspected according to the inspection information and the corresponding relationship.

[0057] In this specification, an inspection method is provided. This specification also relates to a data center inspection method, an inspection device, a data center inspection device, a computing device, and a computer-readable storage medium, which will be described in detail one by one in the following embodiments.

[0058] See Figure 1 , Figure 1 which shows an architecture diagram of an inspection system provided by an embodiment of this specification. The inspection system may include a client 100 and a server 200;

[0059] The client 100 is configured to send a target inspection image of a target inspection point to the server 200;

[0060] The server 200 is configured to parse, from the target inspection image, a plurality of target inspection regions and inspection information respectively corresponding to the plurality of target inspection regions; match the plurality of target inspection regions with a plurality of target template regions in a target template image to determine a correspondence between the target inspection regions and the target template regions, where each target template region corresponds to an object to be inspected one by one; determine the inspection information of the object to be inspected according to the inspection information and the correspondence; and send the inspection information of the object to be inspected to the client 100;

[0061] The client 100 is further configured to receive the inspection information of the object to be inspected sent by the server 200.

[0062] By applying the solution of the embodiment of this specification, automatic inspection is realized from the perspective of regions by matching the target inspection regions with a plurality of target template regions. On the basis of improving the inspection efficiency, it can cope with the situation where there is a large deviation between the target inspection image and the target template image, further improving the anti-interference ability and inspection accuracy of the inspection.

[0063] In practical applications, the inspection system may include a plurality of clients 100 and a server 200. Among them, the client 100 may include end-side devices, and the server 200 may include cloud-side devices. A communication connection can be established between the plurality of clients 100 through the server 200. In an inspection scenario, the server 200 is used to provide inspection services between the plurality of clients 100. The plurality of clients 100 can be used as sending ends or receiving ends respectively to achieve communication through the server 200.

[0064] The user can interact with the server 200 through the client 100 to receive data sent by other clients 100, or send data to other clients 100, etc. In an inspection scenario, it may be that the user publishes a data stream to the server 200 through the client 100, and the server 200 generates an inspection result according to the data stream and pushes the inspection result to other clients that have established communication.

[0065] Among them, a connection is established between the client 100 and the server 200 through a network. The network provides a medium for the communication link between the client 100 and the server 200. The network can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc. The data transmitted by the client 100 may need to be processed such as encoded, transcoded, compressed, etc. before being published to the server 200.

[0066] The client 100 can be a browser, an APP (Application), or a web application such as an H5 (HyperText Markup Language 5) application, or a light application (also known as a mini-program, a lightweight application program), or a cloud application, etc. The client 100 can be developed based on the software development kit (SDK) of the corresponding service provided by the server 200, such as developed based on the real-time communication (RTC) SDK. The client 100 can be deployed in an electronic device and needs to rely on the device or certain APPs in the device to run, etc. The electronic device can, for example, have a display screen and support information browsing, etc., such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. Various other types of applications can usually be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0067] The server 200 can include servers that provide various services, such as a server that provides communication services for multiple clients, or a server for background training that provides support for the models used on the client, or a server that processes the data sent by the client, etc. It should be noted that the server 200 can be implemented as a distributed server cluster composed of multiple servers, or can be implemented as a single server. The server can also be a server of a distributed system, or a server combined with a blockchain. The server can also be a cloud server of basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms, or an intelligent cloud computing server or an intelligent cloud host with artificial intelligence technology.

[0068] It should be noted that the inspection method provided in the embodiments of this specification is generally executed by the server. However, in other embodiments of this specification, the client can also have a similar function to the server, so as to execute the inspection method provided in the embodiments of this specification. In other embodiments, the inspection method provided in the embodiments of this specification can also be jointly executed by the client and the server.

[0069] See Figure 2 , Figure 2 which shows the flowchart of an inspection method provided by an embodiment of this specification, specifically including the following steps:

[0070] Step 202: Obtain the target inspection image of the target inspection point.

[0071] In one or more embodiments of this specification, at the beginning of the inspection, the target inspection image of the target inspection point can be obtained to perform the inspection based on the target inspection image and obtain the inspection result.

[0072] Specifically, the target inspection point refers to the target position where the inspection device moves. The inspection scenario can include one inspection point or multiple inspection points. The number of inspection points is specifically set according to the actual situation, and this specification does not make any limitations on this. The target inspection image refers to the image taken by the inspection device at the target inspection point.

[0073] In practical applications, there are various ways to obtain the target inspection image of the target inspection point, which are specifically selected according to the actual situation, and this specification does not make any limitations on this. In one possible implementation of this specification, the target inspection image of the target inspection point sent by the user through the client can be received. In another possible implementation of this specification, the target inspection image of the target inspection point can be read from other data acquisition devices or databases. In still another possible implementation of this specification, the inspection device can be scheduled to move to the target inspection point to take a real-time shot to obtain the target inspection image.

[0074] In an optional embodiment of this specification, the above-mentioned obtaining the target inspection image of the target inspection point may include the following steps:

[0075] Obtain the inspection device status information corresponding to the target inspection point, where the inspection device status information includes at least one of device position information, device attitude information, and device shooting parameters;

[0076] According to the inspection device status information, schedule the inspection device to move to the target inspection point for shooting to obtain the target inspection image.

[0077] Specifically, the inspection device status information is used to describe the position, attitude, and shooting parameters of the inspection device at the target inspection point. The device position information includes, but is not limited to, the device position coordinates and the device position area. The device attitude information includes, but is not limited to, the device shooting angle and the device shooting direction. The device shooting parameters include, but are not limited to, the camera fill light intensity and the camera focal length. The inspection device includes, but is not limited to, inspection cameras, inspection scanning robots, and so on.

[0078] It should be noted that there are various ways to obtain the inspection device status information corresponding to the target inspection point, which is specifically selected according to the actual situation, and the embodiments of this specification do not make any limitations in this regard. In one possible implementation manner of this specification, the inspection device status information corresponding to the target inspection point sent by the user through the client can be received. In another possible implementation manner of this specification, the inspection device status information corresponding to the target inspection point can be read from other data acquisition devices or databases.

[0079] In practical applications, there are various ways to dispatch the inspection device to move to the target inspection point for shooting according to the inspection device status information to obtain the target inspection image, which is specifically selected according to the actual situation, and the embodiments of this specification do not make any limitations in this regard. In one possible implementation manner of this specification, an inspection instruction can be sent to the inspection device, where the inspection instruction carries the inspection device status information, and the inspection device moves to the target inspection point for shooting based on the inspection device status information in response to the inspection instruction to obtain the target inspection image. In another possible implementation manner of this specification, a shorter inspection path can be generated based on the inspection device status information, and an inspection instruction is sent to the inspection device, where the inspection instruction carries the inspection path and the inspection device status information, and the inspection device moves to the target inspection point based on the inspection path in response to the inspection instruction and adjusts the camera for shooting based on the inspection device status information to obtain the target inspection image.

[0080] Applying the solution of the embodiments of this specification, the inspection device status information corresponding to the target inspection point is obtained, and according to the inspection device status information, the inspection device is dispatched to move to the target inspection point for shooting to obtain the target inspection image. By dispatching the inspection device to take real-time shots to obtain the target inspection image, the timeliness of the target inspection image is ensured.

[0081] Step 204: Parse out multiple target inspection areas and the inspection information respectively corresponding to the multiple target inspection areas from the target inspection image.

[0082] In one or more embodiments of this specification, after obtaining the target inspection image of the target inspection point, further, multiple target inspection areas and the inspection information respectively corresponding to the multiple target inspection areas can be parsed out from the target inspection image.

[0083] Specifically, the target inspection area refers to the area in the target inspection image that includes inspection information. The inspection information includes, but is not limited to, status information and digital information. Taking the inspection image of the distribution room as an example of the target inspection image, multiple target inspection areas may include the areas where multiple distribution boxes are located. The inspection information respectively corresponding to the multiple target inspection areas may be voltage, indicator light color, and so on.

[0084] In practical applications, there are various ways to parse out multiple target inspection areas and the inspection information respectively corresponding to the multiple target inspection areas from the target inspection image, which are specifically selected according to the actual situation, and the embodiments of this specification do not make any limitations in this regard. In one possible implementation manner of this specification, a preset area division rule may be obtained. For example, the target inspection image is evenly divided into N inspection areas, and information recognition is respectively performed on these N inspection areas, and the inspection areas that do not include inspection information are discarded to obtain the target inspection areas and the inspection information respectively corresponding to the multiple target inspection areas, where N is the number of inspection areas and is a positive integer, and the preset area division rule is specifically set according to the actual situation. In another possible implementation manner of this specification, a region detection model may be used to parse out multiple target inspection areas from the target inspection image, and an identification strategy is used to identify the inspection information respectively corresponding to the multiple target inspection areas.

[0085] In an optional embodiment of this specification, since the inspection information in multiple target inspection areas may be different. For example, the inspection information in the target inspection area may be the status information of whether the indicator light is on, or the voltage value on the distribution box, or the gear position pointed by the status knob. Therefore, a corresponding target identification strategy can be selected according to the area type of each target inspection area, and the target identification strategy is used to obtain the inspection information corresponding to the target inspection area. That is, the above-mentioned process of parsing out multiple target inspection areas and the inspection information respectively corresponding to the multiple target inspection areas from the target inspection image may include the following steps:

[0086] Input the target inspection image into the region detection model to obtain multiple target inspection areas and the area types respectively corresponding to the multiple target inspection areas;

[0087] Select the target identification strategy corresponding to the area type from multiple pre-set identification strategies;

[0088] Use the target identification strategy to perform information recognition on multiple target inspection areas to obtain the inspection information respectively corresponding to the multiple target inspection areas.

[0089] Specifically, the area detection model can be understood as an object detection algorithm model. The area detection model can be built based on the YOLO framework and trained with a large number of sample inspection images and the sample labels corresponding to each sample inspection image, so as to achieve a high accuracy. Among them, the sample labels include sample inspection area labels and sample area type labels. The area detection model can detect the input target inspection image and output the area coordinates and area type of the target inspection area in the target inspection image. Among them, the area coordinates can only include the coordinates of the upper left corner and the lower right corner of the area. The multiple recognition strategies include, but are not limited to, multiple recognition algorithms such as image classification, semantic segmentation, keypoint detection, OCR, and image processing algorithms.

[0090] Further, after obtaining the area types corresponding to the multiple target inspection areas respectively, the recognition strategies whose corresponding types are the same as the area types can be screened out from the multiple recognition strategies, and the recognition strategy can be determined as the target recognition strategy. For example, if the area type is a text area, then the recognition strategy corresponding to the text area is screened out from the multiple recognition strategies as the OCR strategy.

[0091] Applying the solution of the embodiment of this specification, for the multiple detected target inspection areas, according to the area types corresponding to the multiple target inspection areas respectively, the inspection information is determined through the target recognition strategy corresponding to the area type, and the recognition result of the detection target is obtained. Since the accuracy of the area detection model and the recognition strategy is relatively high, the accuracy of the multiple target inspection areas and the inspection information is further improved.

[0092] In a possible implementation manner of this specification, since the target inspection image may include blank areas, be too large in size and poor in clarity, therefore, the target inspection image can be only cropped or super-resolution processed. Further, the target inspection image can be first cropped, and then the cropped target inspection image can be super-resolution processed to obtain an updated target inspection image. That is, before inputting the target inspection image into the area detection model to obtain multiple target inspection areas and the area types corresponding to the multiple target inspection areas respectively, the following steps can also be included:

[0093] Crop the target inspection image to obtain a cropped target inspection image;

[0094] Perform super-resolution processing on the cropped target inspection image to obtain an updated target inspection image.

[0095] Specifically, cropping refers to the process of deleting unnecessary parts from the target inspection image. Super-resolution processing is used to convert a low-resolution image into a high-resolution image, aiming to improve the quality and clarity of the image.

[0096] It should be noted that the methods for cropping and super-resolution processing of the target inspection image are specifically selected according to the actual situation, and the embodiments of this specification do not make any limitations in this regard. The methods for cropping the target inspection image include but are not limited to geometric shape cropping, freehand drawing cropping, and local detail cropping.

[0097] The methods for super-resolution processing of the cropped target inspection image include but are not limited to processing based on the frequency domain method and processing based on the spatial domain method. Among them, the frequency domain method mainly realizes image super-resolution processing by filtering in the frequency domain. Commonly used frequency domain filters include Fourier transform, discrete cosine transform, etc. The spatial domain method mainly includes the interpolation method and the pyramid method. The interpolation method mainly increases the resolution of the image by interpolating the image, while the pyramid method is a process of gradually enlarging a low-resolution image to a high-resolution image.

[0098] Applying the solution of the embodiments of this specification, the target inspection image is cropped to obtain the cropped target inspection image; the cropped target inspection image is subjected to super-resolution processing to obtain the updated target inspection image. Through image cropping and super-resolution processing, the accuracy of region detection, template matching, and information recognition is improved.

[0099] Step 206: Match multiple target inspection regions with multiple target template regions in the target template image to determine the corresponding relationship between the target inspection regions and the target template regions, where each target template region corresponds to an object to be inspected one by one.

[0100] In one or more embodiments of this specification, after obtaining the target inspection image of the target inspection point and parsing out multiple target inspection regions and the inspection information corresponding to the multiple target inspection regions from the target inspection image, further, multiple target inspection regions can be matched with multiple target template regions in the target template image to determine the corresponding relationship between the target inspection regions and the target template regions.

[0101] Specifically, the target template image is the template image corresponding to the target inspection point. In the embodiments of this specification, the same type of inspection points correspond to one template image. The target template image includes multiple target template regions, and each target template region corresponds to an object to be inspected one by one. The object to be inspected refers to an object including inspection information, such as a distribution box including voltage values, a thermometer including temperature values, and so on.

[0102] In practical applications, there are various ways to match multiple target inspection regions with multiple target template regions in the target template image to determine the corresponding relationship between the target inspection regions and the target template regions, which are specifically selected according to the actual situation, and the embodiments of this specification do not make any limitations in this regard.

[0103] In the first possible implementation manner of this specification, for any target inspection area, the inspection area image corresponding to the target inspection area can be matched with the template area images corresponding to multiple target template areas, and the target template area images with relatively high similarity to the inspection area image corresponding to the target inspection area can be screened out from the multiple template area images, and the target template area corresponding to the target template area image can be determined as the target template area corresponding to the target inspection area.

[0104] In the second possible implementation manner of this specification, multiple target inspection areas and multiple target template areas in the target template image can be input into a pre-trained relationship determination model to obtain the corresponding relationship between the target inspection area and the target template area. Among them, the relationship determination model is trained based on a large number of sample area pairs, and the sample area pairs include sample inspection areas and sample template areas with corresponding relationships.

[0105] In the third possible implementation manner of this specification, multiple target inspection areas and multiple target template areas in the target template image can be matched based on a template matching algorithm to determine the corresponding relationship between the target inspection area and the target template area. That is, the above-mentioned matching of multiple target inspection areas and multiple target template areas in the target template image to determine the corresponding relationship between the target inspection area and the target template area may include the following steps:

[0106] Obtain a pre-defined region matching feasible region, where the region matching feasible region is defined based on multiple target inspection areas and multiple target template areas in the target template image;

[0107] Determine the variables affecting the matching relationship and construct a matching relationship objective function;

[0108] Based on the region matching feasible region and the matching relationship objective function, solve the corresponding relationship between the target inspection area and the target template area.

[0109] Specifically, the region matching feasible region can refer to the following formula (1). There are various ways to obtain the pre-defined region matching feasible region, which are specifically selected according to the actual situation, and this specification embodiment does not make any limitations on this. In one possible implementation manner of this specification, the region matching feasible region sent by the user through the client can be received. In another possible implementation manner of this specification, the region matching feasible region can be read from other data acquisition devices or databases.

[0110] C = {a - b|a ∈ A, b ∈ B} (1)

[0111] Among them, C represents the region matching feasible region; A = {a1, a2, …, a n}, a idenote the centroid coordinates of the $i$-th target template region; $B = \{b_1, b_2, \ldots, b$ m $\}$, $b$ i denote the centroid coordinates of the $i$-th target inspection region; the relative displacement $x \in C$ between the target template region and the target inspection region, where $x$ is a variable affecting the matching relationship; $n$ represents the number of target template regions; $m$ represents the number of target inspection regions.

[0112] In practical applications, first, the feasible region of region matching for the problem can be defined through a series of boundary conditions and assumptions to simplify the solution; second, a matching relationship objective function for the problem can be constructed based on indicators such as IoU and Euclidean distance; then, a set of sample points can be constructed within the feasible region of region matching, and the solution that maximizes the matching relationship objective function can be found among the sample points through an optimization algorithm. If not found, continue sampling until the convergence condition is met. Among them, the methods of constructing sample points and finding the solution that minimizes or maximizes the matching relationship objective function include but are not limited to the Monte Carlo method, the gradient descent method, the genetic algorithm, etc. Sampling and optimization algorithms can reduce the calculation time of obtaining $x$ that minimizes or maximizes the value of the matching relationship objective function. It should be noted that within the optimization algorithm, a combinatorial optimization algorithm such as the KM algorithm can be used to solve the corresponding relationship.

[0113] See Figure 3 , Figure 3 which shows a schematic diagram of solving the corresponding relationship in an inspection method provided by an embodiment of this specification. As Figure 3 shown, the assumption is that the displacement of the target inspection region relative to the target template region is all $x$; the boundary condition is that the value of $x$ is within the set of the centroids of all target template regions to the centroids of the target inspection regions; the problem is to find an $x$ such that the overall coincidence degree of the target inspection region after movement with the target template region is the largest, so as to determine the corresponding relationship from the target inspection region to the target template region. When constructing the objective function of the problem based on indicators such as IoU and Euclidean distance, a sequential algorithm can be used to convert it into a single-objective optimization solution. The specific objective function is shown in the following formula (2),

[0114]

[0115] where $match\_num(x)$ represents the number of coincidences between the detected target inspection region and the target template region after the displacement $x$; IoU represents the intersection over union index; $EUC(x)$ represents the sum of the Euclidean distances from the target inspection region to the corresponding target template region.

[0116] Applying the solution of the embodiments of this specification, a predefined area matching feasible region is obtained; variables affecting the matching relationship are determined, and an objective function of the matching relationship is constructed; based on the area matching feasible region and the objective function of the matching relationship, the corresponding relationship between the target inspection area and the target template area is solved, improving the efficiency of determining the corresponding relationship.

[0117] In an alternative embodiment of this specification, since the number, type, and distribution positions of the objects to be recognized in the images taken at inspection points of the same type are usually the same, in order to configure the inspection point templates more efficiently, a template image can be used as the target template image for inspection points of the same type. That is, before matching the multiple target inspection areas and the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection area and the target template area, the following steps may further be included:

[0118] Perform type recognition on the target inspection point to obtain the target inspection point type;

[0119] According to the target inspection point type, screen out the target template image from the template images corresponding to at least one inspection point type.

[0120] Specifically, the target inspection point type includes but is not limited to text type inspection points, indicator light type inspection points, and knob position type inspection points. For example, if the image taken at the inspection point includes text information such as voltage, the type of the inspection point can be a text type inspection point; if the image taken at the inspection point includes various indicator lights, the type of the inspection point can be an indicator light type inspection point.

[0121] In practical applications, there are various ways to perform type recognition on the target inspection point to obtain the target inspection point type, which is specifically selected according to the actual situation, and the embodiments of this specification do not make any limitations in this regard. In a possible implementation manner of this specification, the type of the target inspection point can be recognized according to the position where the target inspection point is located to obtain the target inspection point type. For example, if the target inspection point is in the distribution room, the target inspection point type is a text type inspection point. In another possible implementation manner of this specification, the target inspection point type of the target inspection point can be queried from a pre-set inspection point type configuration table.

[0122] It should be noted that after determining the target inspection point type, the template images corresponding to each inspection point type can be obtained, the inspection point types that are the same as the target inspection point type can be screened out from each inspection point type, and the template images corresponding to the inspection point types that are the same as the target inspection point type can be used as the target template images.

[0123] Apply the solution of the embodiments of this specification to perform type recognition on the target inspection point to obtain the target inspection point type; according to the target inspection point type, select the target template image from the template images corresponding to at least one inspection point type. By configuring a template image for inspection points of the same type, the deployment time of inspection points is greatly shortened, facilitating large-scale deployment and maintenance of inspection points efficiently.

[0124] In the embodiments of this specification, before selecting the target template image from the template images corresponding to at least one inspection point type according to the target inspection point type, the template images corresponding to each inspection point type can be obtained. When obtaining the template images corresponding to each inspection point type, one inspection point can be randomly selected from the inspection points with the same inspection point type, and the inspection device can be scheduled to move to this inspection point to take the template image. However, to ensure the imaging quality and determine the accurate status information of the inspection device, the inspection device can be scheduled to take inspection images at each inspection point, and the template image corresponding to this inspection point type can be selected from the inspection images of the inspection points with the same inspection point type.

[0125] In an optional embodiment of this specification, before selecting the target template image from the template images corresponding to at least one inspection point type according to the target inspection point type, the following steps can also be included:

[0126] Obtain the inspection images of multiple inspection points respectively;

[0127] Perform type recognition on multiple inspection points to determine at least one inspection point type;

[0128] Select the template image corresponding to the first inspection point type from the inspection images corresponding to the first inspection point type, where the first inspection point type is any one of the at least one inspection point type.

[0129] It should be noted that the implementation method of "obtain the inspection images of multiple inspection points respectively" can refer to the implementation method of "obtain the target inspection image of the target inspection point" above, and the embodiments of this specification will not elaborate further.

[0130] Further, after obtaining the inspection images of multiple inspection points respectively, type recognition can be performed on multiple inspection points to determine at least one inspection point type. The implementation method of "perform type recognition on multiple inspection points to determine at least one inspection point type" can refer to the implementation method of "perform type recognition on the target inspection point to obtain the target inspection point type" above, and the embodiments of this specification will not elaborate further.

[0131] In practical applications, there are various ways to screen out the template image corresponding to the first inspection point type from the inspection images corresponding to the first inspection point type. The specific selection is based on the actual situation, and the embodiments of this specification do not make any limitations in this regard. In one possible implementation of this specification, an inspection image can be randomly selected from the inspection images corresponding to the first inspection point type as the template image corresponding to the first inspection point type. In another possible implementation of this specification, in order to ensure the quality of the template image, the quality of the inspection images corresponding to the first inspection point type can be evaluated, and the template image corresponding to the first inspection point type can be screened out according to the evaluation results.

[0132] Applying the solution of the embodiments of this specification, inspection images of multiple inspection points are obtained; the types of multiple inspection points are identified to determine at least one inspection point type; and the template image corresponding to the first inspection point type is screened out from the inspection images corresponding to the first inspection point type, ensuring the accuracy of the template image.

[0133] In an optional embodiment of this specification, the above-mentioned obtaining of the inspection images of multiple inspection points may include the following steps:

[0134] Dispatch the inspection device to move to multiple inspection points for shooting to obtain the inspection images of multiple inspection points respectively;

[0135] After obtaining the inspection images of multiple inspection points, the following steps may further be included:

[0136] Store the state information of the inspection device when the inspection device shoots at multiple inspection points.

[0137] It should be noted that when dispatching the inspection device to move to multiple inspection points for shooting, the inspection device can be dispatched to move to multiple inspection points to shoot the initial inspection images, and the initial inspection images are compared with the preset shooting requirements. If the initial inspection images do not meet the preset shooting requirements, the inspection device is controlled to adjust the device state until the inspection images meeting the preset shooting requirements are shot. Among them, the preset shooting requirements include, but are not limited to, that the clarity is greater than the clarity threshold.

[0138] Furthermore, after dispatching the inspection device to move to multiple inspection points for shooting to obtain the inspection images of multiple inspection points respectively, the state information of the inspection device when the inspection device shoots the inspection images can be stored, which is convenient for directly shooting the inspection images meeting the preset shooting requirements based on the state information of the inspection device when the inspection device moves to the inspection points to shoot images later.

[0139] By applying the solution of the embodiments of this specification, the inspection equipment is scheduled to move to multiple inspection points for shooting, and inspection images of each of the multiple inspection points are obtained; the inspection equipment status information when the inspection equipment is shooting at multiple inspection points is stored, and the inspection equipment status information corresponding to each inspection point can be stored, thereby improving the efficiency of subsequent image shooting at the inspection points.

[0140] In an optional embodiment of the present specification, the above-mentioned screening out the template image corresponding to the first inspection point type from the inspection images corresponding to the first inspection point type may include the following steps:

[0141] Performing a quality assessment on the inspection image corresponding to the first inspection point type to determine an assessment result of the inspection image, wherein the assessment result is used to reflect the quality of the inspection image;

[0142] According to the evaluation result, the template image corresponding to the first inspection point type is screened out.

[0143] It should be noted that there are many ways to perform quality assessment on the inspection image corresponding to the first inspection point type, which can be to calculate the image quality assessment index. The larger the image quality assessment index, the better the image quality. Such as Peak Signal to Noise Ratio (PSNR) and calculation of Structural Similarity Index (SSIM). The image quality can also be assessed using an instruction evaluation model. The method for quality assessment of the inspection image is selected according to the actual situation, and the embodiments of this specification do not impose any restrictions on this.

[0144] Furthermore, after obtaining the evaluation results of the inspection images corresponding to the first inspection point type, the inspection images can be sorted according to the evaluation results, and the top-ranked target images can be used as template images. Inspection images whose evaluation results exceed a preset evaluation threshold can also be used as template images.

[0145] By applying the solution of the embodiment of this specification, the inspection image corresponding to the first inspection point type is evaluated for quality, and the evaluation result of the inspection image is determined; according to the evaluation result, the template image corresponding to the first inspection point type is screened out, thereby ensuring the accuracy of the template image.

[0146] Step 208: Determine the inspection information of the object to be inspected according to the inspection information and the corresponding relationship.

[0147] In one or more embodiments of this specification, a target inspection image of a target inspection point is obtained; from the target inspection image, a plurality of target inspection areas and inspection information respectively corresponding to the plurality of target inspection areas are parsed; after matching the plurality of target inspection areas with a plurality of target template areas in the target template image to determine the corresponding relationship between the target inspection areas and the target template areas, further, the inspection information of the object to be inspected can be determined according to the inspection information and the corresponding relationship.

[0148] It should be noted that since the corresponding relationship is the corresponding relationship between the target inspection area and the target template area, and the target template area corresponds to the object to be inspected one by one, the corresponding relationship between the target inspection area and the object to be inspected can be determined according to the inspection information and the corresponding relationship. Further, the inspection information of the target inspection area is associated with the object to be inspected to determine the inspection information of the object to be inspected.

[0149] Applying the solution of the embodiments of this specification, by matching the target inspection area and a plurality of target template areas, automatic inspection is realized from the perspective of the area. On the basis of improving the inspection efficiency, it can cope with the situation where the deviation between the target inspection image and the target template image is relatively large, further improving the anti-interference ability and inspection accuracy of the inspection.

[0150] In an optional embodiment of this specification, the determining the inspection information of the object to be inspected according to the inspection information and the corresponding relationship may include the following steps:

[0151] According to the corresponding relationship, determine the first target template area corresponding to the first target inspection area, where the first target inspection area is any one of the plurality of target inspection areas, and the first target template area is the target template area in the plurality of target template areas that matches the first target inspection area;

[0152] Determine the first object to be inspected corresponding to the first target template area, and use the inspection information corresponding to the first target inspection area as the inspection information of the first object to be inspected.

[0153] Exemplarily, assume that there are 3 target template areas and 3 target inspection areas. The target inspection areas are Target Inspection Area 1, Target Inspection Area 2, and Target Inspection Area 3 respectively. The target template areas are Target Template Area 1, Target Template Area 2, and Target Template Area 3 respectively. Among them, Target Template Area 1 corresponds to the object to be inspected 1, Target Template Area 2 corresponds to the object to be inspected 2, and Target Template Area 3 corresponds to the object to be inspected 3. Match the 3 target inspection areas and the 3 target template areas, and determine the corresponding relationship between the target inspection areas and the target template areas as "Target Inspection Area 1 corresponds to Target Template Area 2, Target Inspection Area 2 corresponds to Target Template Area 3, and Target Inspection Area 3 corresponds to Target Template Area 1". Since Target Inspection Area 1 corresponds to Target Template Area 2 and Target Template Area 2 corresponds to the object to be inspected 2, the inspection information corresponding to Target Inspection Area 1 is the inspection information of the object to be inspected 2. Similarly, it can be determined that the inspection information corresponding to Target Inspection Area 2 is the inspection information of the object to be inspected 3, and the inspection information corresponding to Target Inspection Area 3 is the inspection information of the object to be inspected 1.

[0154] Applying the solution of the embodiment of this specification, according to the corresponding relationship, determine the first target template area corresponding to the first target inspection area, determine the first object to be inspected corresponding to the first target template area, and use the inspection information corresponding to the first target inspection area as the inspection information of the first object to be inspected. Based on the object detection and template matching algorithm to match the target inspection area and multiple target template areas, the detection and matching accuracy is high, which can cope with the situation where the deviation between the target inspection image and the target template image is large, and improves the anti-interference ability and accuracy of the inspection.

[0155] In an optional embodiment of this specification, after determining the inspection information of the object to be inspected according to the inspection information and the corresponding relationship, the following steps may further be included:

[0156] Verify the inspection information of the object to be inspected, obtain the verification result, and display the verification result to the user.

[0157] Specifically, the verification result includes but is not limited to whether the object to be inspected is operating normally and whether the operating state of the object to be inspected is efficient.

[0158] It should be noted that there are various ways to verify the inspection information of the inspection object, which are specifically selected according to the actual situation, and the embodiments of this specification do not make any limitations in this regard. In one possible implementation of this specification, the reference information of the object to be inspected can be obtained, the reference information and the inspection information can be compared, and the verification result can be determined. For example, if the reference information is to operate efficiently at 26°C and the inspection information is 15°C, it is determined that the object to be inspected is not operating efficiently. In another possible implementation of this specification, the verification result can be directly determined based on the inspection information. For example, if the indicator light of the object to be inspected is red, it is determined that the object to be inspected is operating abnormally.

[0159] In practical applications, after obtaining the verification result, the verification result can be stored and can also be displayed to the user. When displaying the verification result, not only the verification result can be displayed, but also the inspection information, the verification process, the target inspection image, and the target template image can be displayed.

[0160] Applying the solution of the embodiments of this specification to verify the inspection information of the object to be inspected, obtain the verification result, and display the verification result to the user. Visual inspection is realized, and the user experience is improved.

[0161] It is worth noting that the inspection method proposed in the embodiments of this specification can be applied to a variety of scenarios, such as traffic navigation scenarios and data center scenarios. Taking the data center scenario as an example, the target inspection points include but are not limited to distribution rooms, high-voltage rooms, low-voltage rooms, air handling units (AHUs), and pump rooms. The objects to be inspected can be referred to as data equipment to be inspected, and the data equipment to be inspected includes but is not limited to medium-voltage switch cabinets, low-voltage switch cabinets, distribution boxes, three-phase uninterruptible power supplies (UPSs), row head cabinets, and high-voltage direct current (HVDC) power supplies, etc. The target inspection areas include but are not limited to indicator lights, status knobs, voltmeters, pressure gauges, valves, display screens, and sundries, etc.

[0162] The following combines the attached Figure 4 , taking the application of the inspection method provided in this specification in the data center as an example, to further illustrate the inspection method. Among them, Figure 4 shows a flowchart of a data center inspection method provided by an embodiment of this specification, which specifically includes the following steps:

[0163] Step 402: Obtain the target inspection image of the target inspection point in the data center.

[0164] Step 404: Parse out multiple target inspection areas and the inspection information corresponding to the multiple target inspection areas from the target inspection image.

[0165] Step 406: Match multiple target inspection areas with multiple target template areas in the target template image to determine the corresponding relationship between the target inspection areas and the target template areas, where the target template areas correspond to the data devices to be inspected one by one.

[0166] Step 408: Determine the inspection information of the data devices to be inspected according to the inspection information and the corresponding relationship.

[0167] It should be noted that the implementation manners of steps 402 to 408 are the same as those of steps 202 to 208 above, and will not be elaborated in the embodiments of this specification.

[0168] Applying the solution of the embodiments of this specification, by matching the target inspection areas with multiple target template areas, automatic inspection is realized from the perspective of areas, which can cope with the situation where the deviation between the target inspection image and the target template image is large on the basis of improving the inspection efficiency of the data center, and further improves the anti-interference ability and inspection accuracy of the data center inspection.

[0169] See Figure 5 , Figure 5 shows a processing procedure flowchart of an inspection method provided by an embodiment of this specification, which specifically includes inspection point deployment, template image configuration, inspection image capture, target detection and information recognition, and template matching;

[0170] Inspection point deployment: Before scheduling the inspection device to start inspection, inspection point deployment can be performed: schedule the inspection device to move to multiple inspection points for shooting, obtain the inspection images of each of the multiple inspection points, and store the inspection device status information when the inspection device shoots at the multiple inspection points. Specifically, the inspection device can be scheduled to move to multiple inspection points, the camera can be controlled to rotate to a suitable angle, the camera parameters can be adjusted to make the inspection pictures taken by the inspection device meet the preset shooting requirements, and finally the inspection device status information can be recorded;

[0171] Template image configuration: Identify the types of multiple inspection points to determine at least one inspection point type; perform quality evaluation on the inspection images corresponding to the first inspection point type to determine the evaluation result of the inspection images; according to the evaluation result, screen out the template images corresponding to the first inspection point type (screen out the template images); divide multiple target template areas from the template images, and associate the target template areas with the data devices to be inspected, so as to associate the target inspection images with the target template areas during inspection and realize the association between the target inspection images and the data devices to be inspected;

[0172] Inspection image capture: After receiving the inspection task, the scheduling inspection device moves to the target inspection point according to the inspection device status information recorded during the inspection point deployment, and adjusts the camera angle and shooting parameters to capture the target inspection image;

[0173] Target detection and information recognition: Perform image processing on the target inspection image, such as image cropping and image super-resolution processing, to obtain an updated target inspection image; input the updated target inspection image into the region detection model for target detection to obtain multiple target inspection regions and the region types corresponding to the multiple target inspection regions respectively; select the target recognition strategy from multiple pre-set recognition strategies; use the target recognition strategy to perform information recognition on the multiple target inspection regions to obtain the inspection information corresponding to the multiple target inspection regions respectively;

[0174] Template matching: Define the region matching feasible region, determine the variables affecting the matching relationship, construct the matching relationship objective function, and then use the optimization algorithm to solve the corresponding relationship between the target inspection region and the target template region; determine the inspection information of the object to be inspected according to the inspection information and the corresponding relationship, that is, generate the inspection result.

[0175] Applying the solution of the embodiment of this specification, through the region detection model for target detection, multiple target inspection regions and the region types corresponding to the multiple target inspection regions respectively are obtained, and then from multiple target template regions, the target template region corresponding to the target inspection region is found to obtain the corresponding relationship between the target inspection region and the target template region. The factors determining the template matching error are changed to the accuracy of the target detection algorithm and the accuracy of the template matching algorithm. Through algorithm optimization, high-precision inspection can still be maintained when the deviation between the target inspection image and the target template image is large. Moreover, a template image is configured for inspection points of the same type, which shortens the deployment time of inspection points and facilitates large-scale deployment and maintenance of inspection points efficiently.

[0176] Corresponding to the above-mentioned inspection method embodiment, this specification also provides an inspection device embodiment. Figure 6 The structural schematic diagram of an inspection device provided by an embodiment of this specification is shown. As Figure 6 shown, the device includes:

[0177] The first acquisition module 602 is configured to acquire the target inspection image of the target inspection point;

[0178] The first analysis module 604 is configured to analyze multiple target inspection regions and the inspection information corresponding to the multiple target inspection regions respectively from the target inspection image;

[0179] The first matching module 606 is configured to match multiple target inspection areas with multiple target template areas in the target template images, and determine the corresponding relationship between the target inspection areas and the target template areas, where the target template areas correspond to the objects to be inspected one by one;

[0180] The first determination module 608 is configured to determine the inspection information of the objects to be inspected according to the inspection information and the corresponding relationship.

[0181] Optionally, the first acquisition module 602 is further configured to acquire the inspection device status information corresponding to the target inspection point, where the inspection device status information includes at least one of device position information, device attitude information, and device shooting parameters; according to the inspection device status information, schedule the inspection device to move to the target inspection point for shooting to obtain the target inspection image.

[0182] Optionally, the device further includes: an identification module, configured to identify the type of the target inspection point to obtain the target inspection point type; according to the target inspection point type, screen out the target template image from the template images corresponding to at least one inspection point type.

[0183] Optionally, the device further includes: a screening module, configured to acquire the inspection images of multiple inspection points respectively; identify the types of multiple inspection points to determine at least one inspection point type; screen out the template image corresponding to the first inspection point type from the inspection images corresponding to the first inspection point type, where the first inspection point type is any one of the at least one inspection point type.

[0184] Optionally, the screening module is further configured to evaluate the quality of the inspection images corresponding to the first inspection point type to determine the evaluation result of the inspection images, where the evaluation result is used to reflect the quality of the inspection images; according to the evaluation result, screen out the template image corresponding to the first inspection point type.

[0185] Optionally, the screening module is further configured to schedule the inspection device to move to multiple inspection points for shooting to obtain the inspection images of multiple inspection points respectively; the device further includes: a storage module, configured to store the inspection device status information when the inspection device shoots at multiple inspection points.

[0186] Optionally, the first parsing module 604 is further configured to input the target inspection image into a region detection model to obtain multiple target inspection areas and the region types respectively corresponding to the multiple target inspection areas; screen out the target recognition strategy corresponding to the region type from multiple preset recognition strategies; use the target recognition strategy to identify the information of the multiple target inspection areas to obtain the inspection information respectively corresponding to the multiple target inspection areas.

[0187] Optionally, the device further includes: a cropping module configured to crop the target inspection image to obtain a cropped target inspection image; perform super-resolution processing on the cropped target inspection image to obtain an updated target inspection image.

[0188] Optionally, the first matching module 606 is further configured to obtain a predefined region matching feasible region, where the region matching feasible region is defined based on multiple target inspection regions and multiple target template regions in the target template image; determine variables affecting the matching relationship, and construct a matching relationship objective function; based on the region matching feasible region and the matching relationship objective function, solve the correspondence between the target inspection region and the target template region.

[0189] Optionally, the first determination module 608 is further configured to determine, according to the correspondence, the first target template region corresponding to the first target inspection region, where the first target inspection region is any one of the multiple target inspection regions, and the first target template region is the target template region that matches the first target inspection region among the multiple target template regions; determine the first object to be inspected corresponding to the first target template region, and use the inspection information corresponding to the first target inspection region as the inspection information of the first object to be inspected.

[0190] Optionally, the device further includes: a verification module configured to verify the inspection information of the object to be inspected, obtain a verification result, and display the verification result to the user.

[0191] Applying the solution of the embodiment of this specification, by matching the target inspection region and multiple target template regions, automatic inspection is realized from the perspective of regions, which can cope with the situation where the deviation between the target inspection image and the target template image is relatively large on the basis of improving the inspection efficiency, and further improves the anti-interference ability and inspection accuracy of the inspection.

[0192] The above is a schematic solution of an inspection device in this embodiment. It should be noted that the technical solution of this inspection device and the technical solution of the above inspection method belong to the same concept. For the details not described in the technical solution of the inspection device, reference can be made to the description of the technical solution of the above inspection method.

[0193] Corresponding to the above embodiment of the data center inspection method, this specification also provides an embodiment of a data center inspection device. Figure 7 The structural schematic diagram of a data center inspection device provided by an embodiment of this specification is shown. As Figure 7 shown, the device includes:

[0194] A second acquisition module 702 configured to acquire a target inspection image of a target inspection point in the data center;

[0195] The second parsing module 704 is configured to parse multiple target inspection areas and the inspection information corresponding to the multiple target inspection areas respectively from the target inspection image;

[0196] The second matching module 706 is configured to match the multiple target inspection areas with the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection areas and the target template areas, where the target template areas correspond to the data devices to be inspected one by one;

[0197] The second determination module 708 is configured to determine the inspection information of the data devices to be inspected according to the inspection information and the corresponding relationship.

[0198] Applying the solution of the embodiment of this specification, by matching the target inspection areas and the multiple target template areas, automatic inspection is realized from the perspective of areas. On the basis of improving the inspection efficiency of the data center, it can cope with the situation where the deviation between the target inspection image and the target template image is relatively large, and further improves the anti-interference ability and inspection accuracy of the data center inspection.

[0199] The above is a schematic solution of a data center inspection device in this embodiment. It should be noted that the technical solution of this data center inspection device and the technical solution of the above data center inspection method belong to the same concept. For the details not described in detail in the technical solution of the data center inspection device, reference can be made to the description of the technical solution of the above data center inspection method.

[0200] Figure 8 The structural block diagram of a computing device provided by an embodiment of this specification is shown. The components of the computing device 800 include but are not limited to a memory 810 and a processor 820. The processor 820 is connected to the memory 810 through a bus 830, and a database 850 is used to store data.

[0201] The computing device 800 also includes an access device 840, which enables the computing device 800 to communicate via one or more networks 860. Examples of such networks include the Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of wired or wireless network interfaces (e.g., Network Interface Card (NIC)), such as IEEE 802.11 Wireless Local Area Networks (WLAN) wireless interfaces, World Interoperability for Microwave Access (Wi-MAX) interfaces, Ethernet interfaces, Universal Serial Bus (USB) interfaces, cellular network interfaces, Bluetooth interfaces, Near Field Communication (NFC) interfaces, and so on.

[0202] In one embodiment of the present specification, the above components of the computing device 800 and Figure 8 other components not shown may also be connected to each other, for example, via a bus. It should be understood that Figure 8 the block diagram of the computing device shown is for illustrative purposes only and is not a limitation on the scope of the present specification. Those skilled in the art can add or replace other components as needed.

[0203] The computing device 800 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or Personal Computers (PCs). The computing device 800 can also be a mobile or stationary server.

[0204] Among them, the processor 820 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above inspection method or the data center inspection method.

[0205] The above is a schematic solution of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solutions of the above-mentioned inspection method and the data center inspection method belong to the same concept. For the details not described in detail in the technical solution of the computing device, reference can be made to the descriptions of the above-mentioned inspection method or the technical solution of the data center inspection method.

[0206] An embodiment of this specification also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the steps of the above-mentioned inspection method or the data center inspection method are implemented.

[0207] The above is a schematic solution of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solutions of the above-mentioned inspection method and the data center inspection method belong to the same concept. For the details not described in detail in the technical solution of the storage medium, reference can be made to the descriptions of the above-mentioned inspection method or the technical solution of the data center inspection method.

[0208] An embodiment of this specification also provides a computer program. When the computer program is executed on a computer, the computer is made to execute the steps of the above-mentioned inspection method or the data center inspection method.

[0209] The above is a schematic solution of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the technical solutions of the above-mentioned inspection method and the data center inspection method belong to the same concept. For the details not described in detail in the technical solution of the computer program, reference can be made to the descriptions of the above-mentioned inspection method or the technical solution of the data center inspection method.

[0210] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0211] The computer instructions include computer program code, which may be in the form of source code, object code, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, mobile hard disks, magnetic disks, optical discs, computer memories, read-only memories (ROMs), random access memories (RAMs), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0212] It should be noted that for the foregoing method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of this specification are not limited by the described action sequence, because according to the embodiments of this specification, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this specification.

[0213] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0214] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can understand and utilize this specification well. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A patrol inspection method, comprising: Obtaining a target patrol inspection image of a target patrol inspection point; Parsing, from the target patrol inspection image, a plurality of target patrol inspection areas and inspection information respectively corresponding to the plurality of target patrol inspection areas; Matching the plurality of target patrol inspection areas with a plurality of target template areas in a target template image to determine a corresponding relationship between the target patrol inspection areas and the target template areas, wherein the target template areas correspond one-to-one to objects to be patrolled; Determining the inspection information of the objects to be patrolled according to the inspection information and the corresponding relationship.

2. The patrol inspection method according to claim 1, wherein the obtaining a target patrol inspection image of a target patrol inspection point comprises: Obtaining patrol inspection device status information corresponding to the target patrol inspection point, wherein the patrol inspection device status information includes at least one of device position information, device attitude information, and device shooting parameters; Dispatching a patrol inspection device to move to the target patrol inspection point for shooting according to the patrol inspection device status information to obtain a target patrol inspection image.

3. The patrol inspection method according to claim 1 or 2, before matching the plurality of target patrol inspection areas with a plurality of target template areas in a target template image to determine a corresponding relationship between the target patrol inspection areas and the target template areas, further comprising: Performing type recognition on the target patrol inspection point to obtain a target patrol inspection point type; Selecting a target template image from template images corresponding to at least one patrol inspection point type according to the target patrol inspection point type.

4. The patrol inspection method according to claim 3, before selecting a target template image from template images corresponding to at least one patrol inspection point type according to the target patrol inspection point type, further comprising: Obtaining patrol inspection images of a plurality of patrol inspection points respectively; Performing type recognition on the plurality of patrol inspection points to determine at least one patrol inspection point type; Selecting a template image corresponding to a first patrol inspection point type from the patrol inspection images corresponding to the first patrol inspection point type, wherein the first patrol inspection point type is any one of the at least one patrol inspection point type.

5. The patrol inspection method according to claim 4, wherein the selecting a template image corresponding to the first patrol inspection point type from the patrol inspection images corresponding to the first patrol inspection point type comprises: Performing quality evaluation on the patrol inspection images corresponding to the first patrol inspection point type to determine an evaluation result of the patrol inspection images, wherein the evaluation result is used to reflect the quality of the patrol inspection images; Selecting a template image corresponding to the first patrol inspection point type according to the evaluation result.

6. The patrol inspection method according to claim 4, wherein the obtaining patrol inspection images of a plurality of patrol inspection points respectively comprises: Dispatching a patrol inspection device to move to the plurality of patrol inspection points for shooting to obtain the patrol inspection images of the plurality of patrol inspection points respectively; After obtaining the patrol inspection images of the plurality of patrol inspection points respectively, further comprising: Storing the patrol inspection device status information when the patrol inspection device shoots at the plurality of patrol inspection points.

7. The patrol inspection method according to claim 1, wherein the parsing, from the target patrol inspection image, a plurality of target patrol inspection areas and inspection information respectively corresponding to the plurality of target patrol inspection areas comprises: Input the target inspection image into the region detection model to obtain multiple target inspection regions and the region types respectively corresponding to the multiple target inspection regions; From multiple pre-set recognition strategies, screen out the target recognition strategy corresponding to the region type; Use the target recognition strategy to identify information for the multiple target inspection regions to obtain the inspection information respectively corresponding to the multiple target inspection regions.

8. The inspection method according to claim 7, before inputting the target inspection image into the region detection model to obtain multiple target inspection regions and the region types respectively corresponding to the multiple target inspection regions, further includes: Crop the target inspection image to obtain a cropped target inspection image; Perform super-resolution processing on the cropped target inspection image to obtain an updated target inspection image.

9. The inspection method according to claim 1, the step of matching the multiple target inspection regions with the multiple target template regions in the target template image to determine the corresponding relationship between the target inspection region and the target template region includes: Obtain a pre-defined region matching feasible region, where the region matching feasible region is defined based on the multiple target inspection regions and the multiple target template regions in the target template image; Determine the variables affecting the matching relationship and construct a matching relationship objective function; Based on the region matching feasible region and the matching relationship objective function, solve the corresponding relationship between the target inspection region and the target template region.

10. The inspection method according to claim 1, the step of determining the inspection information of the object to be inspected according to the inspection information and the corresponding relationship includes: According to the corresponding relationship, determine the first target template region corresponding to the first target inspection region, where the first target inspection region is any one of the multiple target inspection regions, and the first target template region is the target template region in the multiple target template regions that matches the first target inspection region; Determine the first object to be inspected corresponding to the first target template region, and use the inspection information corresponding to the first target inspection region as the inspection information of the first object to be inspected.

11. The inspection method according to claim 1, after determining the inspection information of the object to be inspected according to the inspection information and the corresponding relationship, further includes: Verify the inspection information of the object to be inspected to obtain a verification result, and display the verification result to the user.

12. A data center inspection method includes: Obtain the target inspection image of the target inspection point in the data center; Parse out multiple target inspection regions and the inspection information respectively corresponding to the multiple target inspection regions from the target inspection image; Match the multiple target inspection regions with the multiple target template regions in the target template image to determine the corresponding relationship between the target inspection region and the target template region, where the target template region corresponds to the data device to be inspected one by one; Determine the inspection information of the data device to be inspected according to the inspection information and the corresponding relationship.

13. A computing device, comprising: a memory and a processor; The memory is configured to store computer-executable instructions, and the processor is configured to execute the computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the method according to any one of claims 1 to 11 or claim 12 are implemented.

14. A computer-readable storage medium storing computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the method according to any one of claims 1 to 11 or claim 12 are implemented.

Images