Inspection method and data center inspection method

Abstract

The present application provides an inspection method and a data center inspection method. The inspection method comprises: acquiring target inspection images of target inspection points; parsing the target inspection images to obtain a plurality of target inspection regions and corresponding inspection information of each of the plurality of target inspection regions; performing matching for the plurality of target inspection regions against a plurality of target template regions in target template images, and determining correspondences between the target inspection regions and the target template regions, wherein the target template regions are in one-to-one correspondence with objects to be inspected; and on the basis of the inspection information and the correspondences, determining inspection information of the objects.

Description

Patrol Inspection Method and Data Center Patrol Inspection Method Technical Field

[0001] Embodiments of this specification relate to the field of computer technology, and particularly to a patrol inspection method. Background Art

[0002] With the development of computer technology, equipment patrol inspection has gradually become more and more important in work and life. The purpose of equipment patrol inspection is to master the operating conditions of equipment and changes in the surrounding environment, discover defects in facilities and potential hazards endangering safety, take effective measures in a timely manner, reduce the chance of sudden failures, keep the equipment in good operating condition, and ensure the safety of the equipment and the stability of the system.

[0003] Currently, the traditional patrol inspection method is mainly manual patrol inspection, which is achieved by patrol inspection personnel walking back and forth in the patrol inspection area to observe. However, this method consumes labor costs and cannot be achieved through manual patrol inspection in some special scenarios such as high temperature, lack of oxygen, and high risk. Moreover, due to the huge number of patrol inspection points, manual patrol inspection may introduce some uncertain factors, resulting in poor accuracy of patrol inspection results. Therefore, there is an urgent need for a patrol inspection scheme with high accuracy. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a patrol inspection method. One or more embodiments of this specification simultaneously relate to a data center patrol inspection method, a patrol inspection device, a data center patrol inspection device, a computing device, a computer-readable storage medium, and a computer program to solve the technical defects existing in the related art.

[0005] According to the first aspect of the embodiments of this specification, a patrol inspection method is provided, including: obtaining a target patrol inspection image of a target patrol inspection point; parsing out a plurality of target patrol inspection areas and inspection information respectively corresponding to the plurality of target patrol inspection areas from the target patrol inspection image; matching the plurality of target patrol inspection areas with a plurality of target template areas in a target template image to determine the corresponding relationship between the target patrol inspection areas and the target template areas, where the target template areas correspond one-to-one with objects to be patrolled; and determining the inspection information of the objects to be patrolled according to the inspection information and the corresponding relationship.

[0006] According to the second aspect of the embodiments of the present specification, a data center inspection method is provided, including: obtaining a target inspection image of a target inspection point in the data center; parsing, from the target inspection image, a plurality of target inspection areas and inspection information respectively corresponding to the plurality of target inspection areas; matching the plurality of target inspection areas with a plurality of target template areas in a target template image to determine the corresponding relationship between the target inspection areas and the target template areas, where the target template areas correspond one-to-one to data devices to be inspected; and determining the inspection information of the data devices to be inspected according to the inspection information and the corresponding relationship.

[0007] According to the third aspect of the embodiments of the present specification, an inspection device is provided, including: a first acquisition module configured to acquire a target inspection image of a target inspection point; a first parsing module configured to parse, from the target inspection image, a plurality of target inspection areas and inspection information respectively corresponding to the plurality of target inspection areas; a first matching module configured to match the plurality of target inspection areas with a plurality of target template areas in a target template image to determine the corresponding relationship between the target inspection areas and the target template areas, where the target template areas correspond one-to-one to objects to be inspected; and a first determination module configured to determine the inspection information of the objects to be inspected according to the inspection information and the corresponding relationship.

[0008] According to the fourth aspect of the embodiments of the present specification, a data center inspection device is provided, including: a second acquisition module configured to acquire a target inspection image of a target inspection point in the data center; a second parsing module configured to parse, from the target inspection image, a plurality of target inspection areas and inspection information respectively corresponding to the plurality of target inspection areas; a second matching module configured to match the plurality of target inspection areas with a plurality of target template areas in a target template image to determine the corresponding relationship between the target inspection areas and the target template areas, where the target template areas correspond one-to-one to data devices to be inspected; and a second determination module configured to determine the inspection information of the data devices to be inspected according to the inspection information and the corresponding relationship.

[0009] According to the fifth aspect of the embodiments of the present specification, a computing device is provided, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the method provided in the first aspect or the second aspect are implemented.

[0010] According to a sixth aspect of the embodiments of the present specification, a computer-readable storage medium is provided, which stores computer-executable instructions that, when executed by a processor, implement the steps of the method provided in the above first aspect or second aspect.

[0011] According to a seventh aspect of the embodiments of the present specification, a computer program is provided, wherein when the computer program is executed on a computer, the computer is made to execute the steps of the method provided in the above first aspect or second aspect.

[0012] The inspection method provided by an embodiment of the present specification includes: obtaining a target inspection image of a target inspection point; parsing, from the target inspection image, a plurality of target inspection areas and inspection information respectively corresponding to the plurality of target inspection areas; matching the plurality of target inspection areas with a plurality of target template areas in a target template image to determine a corresponding relationship between the target inspection areas and the target template areas, wherein the target template areas correspond to objects to be inspected; and determining the inspection information of the objects to be inspected according to the inspection information and the corresponding relationship. By matching the target inspection areas with the plurality of target template areas, automatic inspection is achieved from the perspective of areas, which can cope with the situation where there is a large deviation between the target inspection image and the target template image on the basis of improving the inspection efficiency, and further improves the anti-interference ability and inspection accuracy of the inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] FIG. 1 is an architecture diagram of an inspection system provided by an embodiment of the present specification;

[0014] FIG. 2 is a flowchart of an inspection method provided by an embodiment of the present specification;

[0015] FIG. 3 is a schematic diagram for solving the corresponding relationship in an inspection method provided by an embodiment of the present specification;

[0016] FIG. 4 is a flowchart of a data center inspection method provided by an embodiment of the present specification;

[0017] FIG. 5 is a flowchart of a processing procedure of an inspection method provided by an embodiment of the present specification;

[0018] FIG. 6 is a schematic structural diagram of an inspection device provided by an embodiment of the present specification;

[0019] FIG. 7 is a schematic structural diagram of a data center inspection device provided by an embodiment of the present specification;

[0020] FIG. 8 is a block diagram of the structure of a computing device. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] Numerous specific details are set forth in the following description in order to provide a thorough understanding of this specification. However, this specification can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the connotation of this specification. Therefore, this specification is not limited by the specific implementations disclosed below.

[0022] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a", "the", and "said" used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0023] It should be understood that although the terms first, second, etc. may be used in one or more embodiments of this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "when...", "while...", or "in response to determining".

[0024] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of the relevant data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entrances are provided for the user to select to authorize or reject.

[0025] First, the noun terms involved in one or more embodiments of this specification are explained.

[0026] Object detection: Object detection refers to the task of classifying and finding a variable number of objects in an image.

[0027] YOLO: YOLO (You Only Look Once) is a real-time object detection algorithm that simplifies the object detection problem into a regression problem and can simultaneously predict multiple objects, their bounding boxes, and class labels in an image.

[0028] Semantic segmentation: Semantic segmentation refers to the task of classifying each pixel of an image into each instance (each instance corresponds to a class).

[0029] Optical character recognition: Optical character recognition (OCR) refers to the task of converting text images into a machine-readable text format. Optical character recognition usually recognizes characters by detecting the edges and textures of characters in the image and then converts them into readable text data.

[0030] Intersection over Union (IoU): Intersection over Union (IoU) is an indicator for measuring the detection accuracy in object detection. It represents the degree of overlap between two regions, and its value ranges from 0 to 1, where 1 indicates complete overlap and 0 indicates no overlap.

[0031] Monte Carlo method: The Monte Carlo method (MCM), also known as the statistical simulation method and random sampling technique, is a random simulation method based on probability and statistical theory.

[0032] Euclidean distance: Euclidean distance (EUC) refers to the distance between any two points in an M-dimensional space. It is one of the most commonly used distance metrics in Euclidean geometry and has wide applications in fields such as computer science and physics.

[0033] KM algorithm: The KM algorithm (Kuhn-Munkres Algorithm) is a combinatorial optimization algorithm for solving the weighted bipartite graph matching problem based on the Hungarian algorithm.

[0034] Gradient descent method: The gradient descent method (Gradient Descent Algorithm) is commonly used for parameter estimation and function optimization in machine learning. Its basic principle is to iterate along the gradient direction of the loss function to minimize the loss function.

[0035] Common inspection robots on the market delimit areas based on template images to match inspection targets and template targets. Specifically, when deploying inspection points, an image is taken as the template image, and the position information of a certain target to be recognized is marked with a rectangular box in the template image. During inspection, the inspection robot moves to the same position as the inspection point during deployment, takes an inspection image, and directly uses the position information of the target to be recognized recorded in the template image as the position information of the corresponding target to be recognized in the inspection image.

[0036] However, the above method assumes that the values of the position information of the target to be recognized in the inspection image and the template image are very close. If the deviation between the two is large, there may be no target to be recognized or only part of the target to be recognized in the corresponding rectangular box in the inspection image, resulting in an incorrect matching result. Therefore, the greater the deviation between the inspection image and the template image, the lower the accuracy of the matched target to be recognized. Moreover, due to the complex inspection scenario in the data center, the channels are narrow, which is likely to interfere with the positioning of the inspection robot, resulting in a large deviation between the inspection image and the template image, and the matching accuracy using the above method is relatively low. In addition, in the above method, a template image must be configured for each inspection point. For the data center, the number of inspection points for the inspection robot to patrol is huge. For example, there are hundreds of inspection points in a power distribution room. Configuring a template image for each inspection point during deployment results in a long total time for deploying inspection points.

[0037] To solve the above problems, the embodiments of this specification propose a solution for matching the inspection area and the template area based on the object detection algorithm and the template matching algorithm. The factors determining the matching error are changed from the positioning error to the errors of the object detection algorithm accuracy and the matching algorithm accuracy. Through algorithm optimization, high accuracy of the object detection algorithm and the matching algorithm can still be maintained in the case of large positioning error, which can cope with the situation of large deviation between the inspection image and the template image, and improve the anti-interference ability and accuracy of the inspection robot during inspection.

[0038] Specifically, the embodiments of this specification propose an inspection method, which obtains the target inspection image of the target inspection point; parses out multiple target inspection areas and the inspection information corresponding to the multiple target inspection areas from the target inspection image; matches the multiple target inspection areas with the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection area and the target template area, where the target template area corresponds to the object to be inspected one by one; and determines the inspection information of the object to be inspected according to the inspection information and the corresponding relationship.

[0039] In this specification, an inspection method is provided. This specification also relates to a data center inspection method, an inspection device, a data center inspection device, a computing device, and a computer-readable storage medium, which will be described in detail one by one in the following embodiments.

[0040] Referring to FIG. 1, FIG. 1 shows an architecture diagram of an inspection system provided by an embodiment of this specification. The inspection system may include a client 100 and a server 200.

[0041] The client 100 is configured to send a target inspection image of a target inspection point to the server 200.

[0042] The server 200 is configured to parse out multiple target inspection areas and the inspection information respectively corresponding to the multiple target inspection areas from the target inspection image; match the multiple target inspection areas with multiple target template areas in the target template image to determine the correspondence between the target inspection area and the target template area, where the target template area corresponds to the object to be inspected one by one; determine the inspection information of the object to be inspected according to the inspection information and the correspondence; and send the inspection information of the object to be inspected to the client 100. The server 200 is configured to parse out multiple target inspection areas and the inspection information respectively corresponding to the multiple target inspection areas from the target inspection image; match the multiple target inspection areas with multiple target template areas in the target template image to determine the correspondence between the target inspection area and the target template area, where the target template area corresponds to the object to be inspected one by one; determine the inspection information of the object to be inspected according to the inspection information and the correspondence; and send the inspection information of the object to be inspected to the client 100. The server 200 is configured to parse out multiple target inspection areas and the inspection information respectively corresponding to the multiple target inspection areas from the target inspection image; match the multiple target inspection areas with multiple target template areas in the target template image to determine the correspondence between the target inspection area and the target template area, where the target template area corresponds to the object to be inspected one by one; determine the inspection information of the object to be inspected according to the inspection information and the correspondence; and send the inspection information of the object to be inspected to the client 100.

[0043] The client 100 is further configured to receive the inspection information of the object to be inspected sent by the server 200.

[0044] Applying the solution of the embodiment of this specification, by matching the target inspection area and multiple target template areas, automatic inspection is realized from the perspective of the area, which can cope with the situation where the deviation between the target inspection image and the target template image is relatively large on the basis of improving the inspection efficiency, and further improves the anti-interference ability and inspection accuracy of the inspection.

[0045] In practical applications, the inspection system may include multiple clients 100 and a server 200. Among them, the client 100 may include terminal devices, and the server 200 may include cloud devices. Multiple clients 100 can establish a communication connection through the server 200. In the inspection scenario, the server 200 is used to provide inspection services among multiple clients 100. Multiple clients 100 can be used as the sending end or the receiving end respectively, and communicate through the server 200.

[0046] The user can interact with the server 200 through the client 100 to receive data sent by other clients 100, or send data to other clients 100, etc. In the inspection scenario, it can be that the user publishes a data stream to the server 200 through the client 100, and the server 200 generates an inspection result based on the data stream and pushes the inspection result to other clients that have established communication.

[0047] Among them, a connection is established between the client 100 and the server 200 through a network. The network provides a medium for the communication link between the client 100 and the server 200. The network can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc. The data transmitted by the client 100 may need to be processed such as encoding, transcoding, compression, etc. before being published to the server 200.

[0048] The client 100 can be a browser, an APP (Application), or a web application such as an H5 (HyperText Markup Language 5) application, or a light application (also known as a mini-program, a lightweight application program), or a cloud application, etc. The client 100 can be developed based on the software development kit (SDK) of the corresponding service provided by the server 200, such as developed based on the real-time communication (RTC) SDK. The client 100 can be deployed in an electronic device and needs to rely on the device or certain APPs in the device to run, etc. The electronic device can, for example, have a display screen and support information browsing, etc., such as it can be a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. Various other types of applications can usually be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0049] The server 200 may include servers that provide various services. For example, a server that provides communication services for multiple clients, or a server for background training that supports the models used on the clients, or a server that processes the data sent by the clients, etc. It should be noted that the server 200 may be implemented as a distributed server cluster composed of multiple servers, or may be implemented as a single server. The server may also be a server of a distributed system, or a server combined with a blockchain. The server may also be a cloud server such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content delivery network (CDN), and basic cloud computing services such as big data and artificial intelligence platforms, or an intelligent clothing computing server or an intelligent K host with artificial intelligence technology.

[0050] It should be noted that the inspection method provided in the embodiments of this specification is generally executed by the server. However, in other embodiments of this specification, the client may also have a similar function to the server, so as to execute the inspection method provided in the embodiments of this specification. In other embodiments, the inspection method provided in the embodiments of this specification may also be jointly executed by the client and the server.

[0051] Referring to FIG. 2, FIG. 2 shows a flowchart of an inspection method provided in an embodiment of this specification, which specifically includes the following steps 202 to 208.

[0052] Step 202: Obtain a target inspection image of a target inspection point.

[0053] In one or more embodiments of this specification, at the beginning of the inspection, a target inspection image of a target inspection point may be obtained to perform the inspection based on the target inspection image and obtain an inspection result.

[0054] Specifically, the target inspection point refers to the target position where the inspection device moves. The inspection scenario may include one inspection point or multiple inspection points. The number of inspection points is specifically set according to the actual situation, and this specification does not make any limitation on this. The target inspection image refers to the image taken by the inspection device at the target inspection point.

[0055] In practical applications, there are various ways to obtain the target inspection image of the target inspection point, which are specifically selected according to the actual situation, and the embodiments of this specification do not make any limitations in this regard. In one possible implementation of this specification, the target inspection image of the target inspection point sent by the user through the client can be received. In another possible implementation of this specification, the target inspection image of the target inspection point can be read from other data acquisition devices or databases. In still another possible implementation of this specification, the inspection device can be scheduled to move to the target inspection point to capture the target inspection image in real time.

[0056] In an optional embodiment of this specification, obtaining the target inspection image of the target inspection point may include the following steps: obtaining the inspection device status information corresponding to the target inspection point, where the inspection device status information includes at least one of device position information, device attitude information, and device shooting parameters; according to the inspection device status information, scheduling the inspection device to move to the target inspection point for shooting to obtain the target inspection image.

[0057] Specifically, the inspection device status information is used to describe the position, attitude, and shooting parameters of the inspection device at the target inspection point. The device position information includes but is not limited to the device position coordinates and the device position area. The device attitude information includes but is not limited to the device shooting angle and the device shooting direction. The device shooting parameters include but are not limited to the camera fill light intensity and the camera focal length. The inspection device includes but is not limited to inspection cameras, inspection scanning robots, and the like.

[0058] It should be noted that there are various ways to obtain the inspection device status information corresponding to the target inspection point, which are specifically selected according to the actual situation, and the embodiments of this specification do not make any limitations in this regard. In one possible implementation of this specification, the inspection device status information corresponding to the target inspection point sent by the user through the client can be received. In another possible implementation of this specification, the inspection device status information corresponding to the target inspection point can be read from other data acquisition devices or databases.

[0059] In practical applications, there are various ways to schedule the inspection device to move to the target inspection point for shooting according to the inspection device status information to obtain the target inspection image, which are specifically selected according to the actual situation, and the embodiments of this specification do not make any limitations in this regard. In one possible implementation of this specification, an inspection instruction can be sent to the inspection device, where the inspection instruction carries the inspection device status information, and the inspection device moves to the target inspection point for shooting based on the inspection device status information in response to the inspection instruction to obtain the target inspection image. In another possible implementation of this specification, it can be based Generate a patrol path with a short distance based on the status information of the patrol device, and send a patrol instruction to the patrol device. The patrol instruction carries the patrol path and the status information of the patrol device. The patrol device moves to the target patrol point based on the patrol path in response to the patrol instruction, and adjusts the camera for shooting based on the status information of the patrol device to obtain the target patrol image.

[0060] Applying the solution of the embodiments of this specification, obtain the status information of the patrol device corresponding to the target patrol point. According to the status information of the patrol device, dispatch the patrol device to move to the target patrol point for shooting to obtain the target patrol image. By dispatching the patrol device to take real-time shots to obtain the target patrol image, the timeliness of the target patrol image is ensured.

[0061] Step 204: Parse out multiple target patrol areas and the patrol information respectively corresponding to the multiple target patrol areas from the target patrol image.

[0062] In one or more embodiments of this specification, after obtaining the target patrol image of the target patrol point, further, multiple target patrol areas and the patrol information respectively corresponding to the multiple target patrol areas can be parsed out from the target patrol image.

[0063] Specifically, the target patrol area refers to the area in the target patrol image that includes patrol information. The patrol information includes but is not limited to status information and digital information. Taking the target patrol image as the patrol image of the distribution room as an example, the multiple target patrol areas can include the areas where multiple distribution boxes are located. The patrol information respectively corresponding to the multiple target patrol areas can be voltage, indicator light color, etc.

[0064] In practical applications, there are various ways to parse out multiple target patrol areas and the patrol information respectively corresponding to the multiple target patrol areas from the target patrol image, which are specifically selected according to the actual situation, and the embodiments of this specification do not make any limitations in this regard. In one possible implementation manner of this specification, a preset area division rule can be obtained. For example, the target patrol image is evenly divided into N patrol areas, and information recognition is performed on these N patrol areas respectively, and the patrol areas that do not include patrol information are discarded to obtain the target patrol areas and the patrol information respectively corresponding to the multiple target patrol areas. Here, N is the number of patrol areas and is a positive integer, and the preset area division rule is specifically set according to the actual situation. In another possible implementation manner of this specification, a region detection model can be used to parse out multiple target patrol areas from the target patrol image, and an identification strategy can be used to identify the patrol information respectively corresponding to the multiple target patrol areas.

[0065] In an optional embodiment of this specification, since the inspection information in multiple target inspection areas may vary. For example, the inspection information in a target inspection area may be the status information of whether an indicator light is on, or the voltage value on a distribution box, or the gear position indicated by a status knob. Therefore, the corresponding target recognition strategy can be selected according to the area type of each target inspection area, and the inspection information corresponding to the target inspection area can be obtained by using the target recognition strategy. That is, parsing out multiple target inspection areas and the inspection information corresponding to the multiple target inspection areas from the target inspection image can include the following steps: input the target inspection image into the area detection model to obtain multiple target inspection areas and the area types corresponding to the multiple target inspection areas; select the target recognition strategy corresponding to the area type from a plurality of pre-set recognition strategies; use the target recognition strategy to identify the information of the multiple target inspection areas to obtain the inspection information corresponding to the multiple target inspection areas respectively.

[0066] Specifically, the area detection model can be understood as a target detection algorithm model. The area detection model can be built based on the YOLO framework and trained with a large number of sample inspection images and the sample labels corresponding to each sample inspection image, so as to achieve a high accuracy. Among them, the sample labels include sample inspection area labels and sample area type labels. The area detection model can detect the input target inspection image and obtain the area coordinates and area types of the target inspection areas in the target inspection image. Among them, the area coordinates can only include the coordinates of the upper left corner and the lower right corner of the area. The multiple recognition strategies include but are not limited to various recognition algorithms such as image classification, semantic segmentation, key point detection, OCR, and image processing algorithms. The area coordinates and area types of the target inspection areas in the target inspection image. Among them, the area coordinates can only include the coordinates of the upper left corner and the lower right corner of the area. The multiple recognition strategies include but are not limited to various recognition algorithms such as image classification, semantic segmentation, key point detection, OCR, and image processing algorithms.

[0067] Further, after obtaining the area types corresponding to the multiple target inspection areas respectively, the recognition strategy whose corresponding type is the same as the area type can be selected from the multiple recognition strategies, and this recognition strategy can be determined as the target recognition strategy. For example, if the area type is a text area, then the OCR strategy, which is the recognition strategy corresponding to the text area, can be selected from the multiple recognition strategies.

[0068] Applying the solution of the embodiment of this specification, for the detected multiple target inspection areas, according to the area types corresponding to the multiple target inspection areas respectively, the inspection information is determined through the target recognition strategy corresponding to the area type, and the recognition result of the detected target is obtained. Since the accuracy of the area detection model and the recognition strategy is relatively high, the accuracy of the multiple target inspection areas and the inspection information is further improved.

[0069] In a possible implementation of this specification, since the target inspection image may include blank areas, be too large in size and poor in clarity, therefore, the target inspection image can be only cropped or super-resolution processed. Further, the target inspection image can be first cropped, and then the cropped target inspection image can be super-resolution processed to obtain an updated target inspection image. That is, before inputting the target inspection image into the region detection model to obtain multiple target inspection regions and the region types respectively corresponding to the multiple target inspection regions, the following steps can also be included: Cropping the target inspection image to obtain a cropped target inspection image; Performing super-resolution processing on the cropped target inspection image to obtain an updated target inspection image.

[0070] Specifically, cropping refers to the process of deleting unnecessary parts from the target inspection image. Super-resolution processing is used to convert a low-resolution image into a high-resolution image, with the aim of improving the quality and clarity of the image.

[0071] It should be noted that the methods of cropping and super-resolution processing the target inspection image are specifically selected according to the actual situation, and the embodiments of this specification do not make any limitations on this. The methods of cropping the target inspection image include but are not limited to geometric shape cropping, freehand drawing cropping, and local detail cropping.

[0072] The methods of performing super-resolution processing on the cropped target inspection image include but are not limited to processing based on the frequency domain method and processing based on the spatial domain method. Among them, the frequency domain method mainly realizes image super-resolution processing by filtering in the frequency domain. Commonly used frequency domain filters include Fourier transform, discrete cosine transform, etc. The spatial domain method mainly includes interpolation method and pyramid method. The interpolation method mainly increases the resolution of the image by interpolating the image, while the pyramid method is a process of gradually magnifying a low-resolution image to a high-resolution image.

[0073] Applying the solution of the embodiments of this specification, cropping the target inspection image to obtain a cropped target inspection image; Performing super-resolution processing on the cropped target inspection image to obtain an updated target inspection image. Through image cropping and super-resolution processing, the accuracy of region detection, template matching, and information recognition is improved.

[0074] Step 206: Matching the multiple target inspection regions with the multiple target template regions in the target template image to determine the corresponding relationship between the target inspection regions and the target template regions, where the target template regions correspond one-to-one to the objects to be inspected.

[0075] In one or more embodiments of this specification, after obtaining the target inspection image of the target inspection point and parsing out multiple target inspection areas and the inspection information corresponding to the multiple target inspection areas from the target inspection image, further, the multiple target inspection areas can be matched with the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection areas and the target template areas.

[0076] Specifically, the target template image is the template image corresponding to the target inspection point. In the embodiments of this specification, the same type of inspection points corresponds to one template image. The target template image includes multiple target template areas, and the target template areas correspond one-to-one to the objects to be inspected. The objects to be inspected refer to the objects including inspection information, such as a distribution box including voltage values, a thermometer including temperature values, and so on.

[0077] In practical applications, there are various ways to match the multiple target inspection areas with the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection areas and the target template areas. The specific selection is made according to the actual situation, and the embodiments of this specification do not make any limitations on this.

[0078] In the first possible implementation manner of this specification, for any target inspection area, the inspection area image corresponding to the target inspection area can be subjected to similarity matching with the template area images corresponding to the multiple target template areas, and the target template area images with relatively high similarity to the inspection area image corresponding to the target inspection area are screened out from the multiple template area images, and the target template area corresponding to the target template area image is determined as the target template area corresponding to the target inspection area.

[0079] In the second possible implementation manner of this specification, the multiple target inspection areas and the multiple target template areas in the target template image can be input into a pre-trained relationship determination model to obtain the corresponding relationship between the target inspection areas and the target template areas. Among them, the relationship determination model is trained based on a large number of sample area pairs, and the sample area pairs include sample inspection areas and sample template areas with corresponding relationships.

[0080] In the third possible implementation manner of this specification, multiple target inspection areas can be matched with multiple target template areas in the target template image based on a template matching algorithm to determine the corresponding relationship between the target inspection area and the target template area. That is, the above-mentioned matching of multiple target inspection areas with multiple target template areas in the target template image to determine the corresponding relationship between the target inspection area and the target template area may include the following steps: Obtain a predefined region matching feasible region, where the region matching feasible region is defined based on multiple target inspection areas and multiple target template areas in the target template image; determine variables affecting the matching relationship and construct a matching relationship objective function; based on the region matching feasible region and the matching relationship objective function, solve the corresponding relationship between the target inspection area and the target template area.

[0081] Specifically, the region matching feasible region can be seen in the following formula (1). There are multiple ways to obtain the predefined region matching feasible region, which are specifically selected according to the actual situation, and this specification embodiment does not make any limitations on this. In one possible implementation manner of this specification, the region matching feasible region sent by the user through the client can be received. In another possible implementation manner of this specification, the region matching feasible region can be read from other data acquisition devices or databases. C = {a - b | a ∈ A, b ∈ B} (1)

[0082] Where, C represents the region matching feasible region; A = {a1, a2, ···, an}, n} T ai represents the centroid coordinates of the i-th target template area; B = {b1, b2, ···, bm}, bi represents the centroid coordinates of the i-th target inspection area; the relative displacement x of the target template area and the target inspection area belongs to C, and x is a variable affecting the matching relationship; n represents the number of target template areas; m represents the number of target inspection areas.

[0083] In practical applications, first, the feasible region of problem area matching can be defined through a series of boundary conditions and assumption conditions to simplify the solution; second, an objective function of the matching relationship of the problem can be constructed based on indicators such as IoU and Euclidean distance; then, a set of sample points are constructed within the feasible region of area matching, and the solution that maximizes the objective function of the matching relationship is found among the sample points through an optimization algorithm. If not found, sampling continues until the convergence condition is met. Among them, the methods of constructing sample points and finding the solution that minimizes or maximizes the objective function of the matching relationship include but are not limited to Monte Carlo method, gradient descent method, genetic algorithm, etc. The calculation time of obtaining x that minimizes or maximizes the value of the objective function of the matching relationship can be reduced through sampling and optimization algorithms. It should be noted that within the optimization algorithm, a combinatorial optimization algorithm such as the KM algorithm can be used to solve the corresponding relationship.

[0084] Referring to FIG. 3, FIG. 3 shows a schematic diagram of solving the corresponding relationship in a patrol inspection method provided by an embodiment of this specification. As shown in FIG. 3, the assumption condition is that the displacement of the target patrol inspection area relative to the target template area is all x; the boundary condition is that the value of x is within the set of the centroids of all target template areas to the centroid of the target patrol inspection area; the problem is to find an x such that the overall coincidence degree between the target patrol inspection area after movement and the target template area is the largest, so as to determine the corresponding relationship between the target patrol inspection area and the target template area. When constructing the objective function of the problem based on indicators such as IoU and Euclidean distance, a sequential algorithm can be used to convert it into a single-objective optimization solution. The specific objective function is shown in the following formula (2), represents the number of coincidences between the detected target patrol inspection area and the target template area after the displacement x moves; EUC(x) represents the Euclidean distance from the target patrol inspection area to the corresponding target template area

[0086] Applying the solution of the embodiment of this specification, a predefined feasible region of area matching is obtained; variables affecting the matching relationship are determined, and an objective function of the matching relationship is constructed; based on the feasible region of area matching and the objective function of the matching relationship, the corresponding relationship between the target patrol inspection area and the target template area is solved, improving the efficiency of determining the corresponding relationship.

[0087] In an optional embodiment of this specification, since the number, type, and distribution position of the objects to be recognized are usually the same in the images captured at inspection points of the same type, in order to configure the inspection point templates more efficiently, a template image can be used as the target template image for inspection points of the same type. That is, before matching the multiple target inspection areas with the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection area and the target template area, the following steps can also be included: performing type recognition on the target inspection point to obtain the target inspection point type; and screening out the target template image from the template images corresponding to at least one inspection point type according to the target inspection point type.

[0088] Specifically, the target inspection point types include but are not limited to text type inspection points, indicator light type inspection points, and knob position type inspection points. For example, if the image captured at the inspection point includes text information such as voltage, the type of the inspection point can be a text type inspection point; or if the image captured at the inspection point includes various indicator lights, the type of the inspection point can be an indicator light type inspection point.

[0089] In practical applications, there are various ways to perform type recognition on the target inspection point to obtain the target inspection point type, and specific selection is made according to the actual situation. The embodiments of this specification do not make any limitations in this regard. In a possible implementation manner of this specification, the type of the target inspection point can be recognized according to the position where the target inspection point is located to obtain the target inspection point type. For example, if the target inspection point is in the distribution room, the target inspection point type is a text type inspection point. In another possible implementation manner of this specification, the target inspection point type of the target inspection point can be queried from a pre-set inspection point type configuration table. It should be noted that after determining the target inspection point type, the template images corresponding to each inspection point type can be obtained, the inspection point types that are the same as the target inspection point type are screened out from each inspection point type, and the template image corresponding to the inspection point type that is the same as the target inspection point type is used as the target template image.

[0090] Applying the solution of the embodiments of this specification, performing type recognition on the target inspection point to obtain the target inspection point type; and screening out the target template image from the template images corresponding to at least one inspection point type according to the target inspection point type. By configuring a template image for inspection points of the same type, the deployment time of inspection points is greatly shortened, which is convenient for large-scale deployment and maintenance of inspection points efficiently.

[0091]

[0092] ​In the embodiments of this specification, before screening out the target template image from the template images corresponding to at least one inspection point type according to the target inspection point type, the template images corresponding to each inspection point type can be obtained. When obtaining the template images corresponding to each inspection point type, one inspection point can be randomly selected from the inspection points with the same inspection point type, and the inspection device can be dispatched to move to this inspection point to capture the template image. However, in order to ensure the imaging quality and determine the accurate status information of the inspection device, the inspection device can be dispatched to capture inspection images at each inspection point, and the template image corresponding to this inspection point type can be screened out from the inspection images of the inspection points with the same inspection point type.

[0093] In an alternative embodiment of this specification, before screening out the target template image from the template images corresponding to at least one inspection point type according to the target inspection point type, the following steps can also be included: obtaining the inspection images of multiple inspection points respectively; performing type recognition on the multiple inspection points to determine at least one inspection point type; screening out the template image corresponding to the first inspection point type from the inspection images corresponding to the first inspection point type, where the first inspection point type is any one of the at least one inspection point type.

[0094] It should be noted that the implementation method of "obtaining the inspection images of multiple inspection points respectively" can refer to the implementation method of "obtaining the target inspection image of the target inspection point" above, and the embodiments of this specification will not elaborate on it.

[0095] Furthermore, after obtaining the inspection images of multiple inspection points respectively, type recognition can be performed on the multiple inspection points to determine at least one inspection point type. The implementation method of "performing type recognition on the multiple inspection points to determine at least one inspection point type" can refer to the implementation method of "performing type recognition on the target inspection point to obtain the target inspection point type" above, and the embodiments of this specification will not elaborate on it.

[0096] In practical applications, there are various ways to screen out the template image corresponding to the first inspection point type from the inspection images corresponding to the first inspection point type, which can be specifically selected according to the actual situation, and the embodiments of this specification do not make any limitations in this regard. In a possible implementation manner of this specification, one inspection image can be randomly selected from the inspection images corresponding to the first inspection point type as the template image corresponding to the first inspection point type. In another possible implementation manner of this specification, in order to ensure the quality of the template image, the quality of the inspection images corresponding to the first inspection point type can be evaluated, and the template image corresponding to the first inspection point type can be screened out according to the evaluation results.

[0097] Apply the solution of the embodiments of this specification to obtain inspection images of multiple inspection points; perform type recognition on the multiple inspection points to determine at least one inspection point type; and screen out the template image corresponding to the first inspection point type from the inspection images corresponding to the first inspection point type, ensuring the accuracy of the template image. The template image corresponding to the first inspection point type is ensured to be accurate.

[0098] In an optional embodiment of this specification, the above-mentioned obtaining of the inspection images of multiple inspection points respectively may include the following steps: dispatch the inspection device to move to multiple inspection points for shooting to obtain the inspection images of multiple inspection points respectively; after obtaining the inspection images of multiple inspection points respectively, the following steps may further be included: store the inspection device status information when the inspection device shoots at multiple inspection points.

[0099] It should be noted that when dispatching the inspection device to move to multiple inspection points for shooting, the inspection device can be dispatched to move to multiple inspection points to shoot initial inspection images, and compare the initial inspection images with the preset shooting requirements. If the initial inspection images do not meet the preset shooting requirements, the inspection device is controlled to adjust the device status until the inspection images that meet the preset shooting requirements are shot. Among them, the preset shooting requirements include, but are not limited to, the clarity being greater than the clarity threshold.

[0100] Furthermore, after dispatching the inspection device to move to multiple inspection points for shooting to obtain the inspection images of multiple inspection points respectively, the inspection device status information when the inspection device shoots the inspection images can be stored, which is convenient for directly shooting inspection images that meet the preset shooting requirements based on the inspection device status information when the inspection device moves to the inspection points to shoot images later.

[0101] Apply the solution of the embodiments of this specification to dispatch the inspection device to move to multiple inspection points for shooting to obtain the inspection images of multiple inspection points respectively; store the inspection device status information when the inspection device shoots at multiple inspection points, and the inspection device status information corresponding to each inspection point can be stored, improving the efficiency of shooting images at the inspection points later.

[0102] In an optional embodiment of this specification, the above-mentioned screening out of the template image corresponding to the first inspection point type from the inspection images corresponding to the first inspection point type may include the following steps: perform quality evaluation on the inspection images corresponding to the first inspection point type to determine the evaluation result of the inspection images, where the evaluation result is used to reflect the quality level of the inspection images; and screen out the template image corresponding to the first inspection point type according to the evaluation result.

[0103] It should be noted that there are various ways to evaluate the quality of the inspection images corresponding to the first type of inspection point. It can be to calculate the image quality evaluation index. The larger the image quality evaluation index, the better the quality of the image. For example, the peak signal-to-noise ratio (PSNR, Peak Signal to Noise Ratio), the calculated structural similarity index (SSIM, Structural Similarity Index). o It is also possible to use an instruction evaluation model to evaluate the quality of the image. The specific method for evaluating the quality of the inspection image is selected according to the actual situation, and the embodiments of this specification do not make any limitations in this regard.

[0104] Further, after obtaining the evaluation results of the inspection images corresponding to the first type of inspection point respectively, the inspection images can be sorted according to the evaluation results, and the target images with higher rankings can be used as template images. It is also possible to use the inspection images whose evaluation results exceed the preset evaluation threshold as template images.

[0105] Applying the solution of the embodiments of this specification to evaluate the quality of the inspection images corresponding to the first type of inspection point, determine the evaluation results of the inspection images; according to the evaluation results, screen out the template images corresponding to the first type of inspection point. This ensures the accuracy of the template images.

[0106] Step 208: Determine the inspection information of the object to be inspected according to the inspection information and the corresponding relationship.

[0107] In one or more embodiments of this specification, obtain the target inspection images of the target inspection points; parse out multiple target inspection areas and the inspection information corresponding to the multiple target inspection areas respectively from the target inspection images; match the multiple target inspection areas with the multiple target template areas in the target template image, and after determining the corresponding relationship between the target inspection area and the target template area, further, the inspection information of the object to be inspected can be determined according to the inspection information and the corresponding relationship. It should be noted that since the corresponding relationship is the corresponding relationship between the target inspection area and the target template area, and the target template area corresponds to the object to be inspected one by one, the corresponding relationship between the target inspection area and the object to be inspected can be determined according to the inspection information and the corresponding relationship, and further, the inspection information of the target inspection area is associated with the object to be inspected to determine the inspection information of the object to be inspected.

[0108]

[0109] ​Applying the solution of the embodiments of this specification, by matching the target inspection area and multiple target template areas, automatic inspection is achieved from the perspective of areas. On the basis of improving the inspection efficiency, it can cope with the situation where the deviation between the target inspection image and the target template image is relatively large, further improving the anti-interference ability and inspection accuracy of the inspection.

[0110] In an optional embodiment of this specification, the above determining the inspection information of the object to be inspected according to the inspection information and the corresponding relationship may include the following steps: determining, according to the corresponding relationship, the first target template area corresponding to the first target inspection area, where the first target inspection area is any one of the multiple target inspection areas, and the first target template area is the target template area that matches the first target inspection area among the multiple target template areas; determining the first object to be inspected corresponding to the first target template area, and using the inspection information corresponding to the first target inspection area as the inspection information of the first object to be inspected.

[0111] Exemplarily, assume that there are 3 target template areas and 3 target inspection areas. The target inspection areas are respectively target inspection area 1, target inspection area 2, and target inspection area 3. The target template areas are respectively target template area 1, target template area 2, and target template area 3. Among them, target template area 1 corresponds to object to be inspected 1, target template area 2 corresponds to object to be inspected 2, and target template area 3 corresponds to object to be inspected 3. Matching the 3 target inspection areas and the 3 target template areas, the corresponding relationship between the target inspection area and the target template area is determined as "target inspection area 1 corresponds to target template area 2, target inspection area 2 corresponds to target template area 3, and target inspection area 3 corresponds to target template area 1". Since target inspection area 1 corresponds to target template area 2 and target template area 2 corresponds to object to be inspected 2, therefore, the inspection information corresponding to target inspection area 1 is the inspection information of object to be inspected 2. Similarly, it can be determined that the inspection information corresponding to target inspection area 2 is the inspection information of object to be inspected 3, and the inspection information corresponding to target inspection area 3 is the inspection information of object to be inspected 1.

[0112] Applying the solution of the embodiment of this specification, according to the corresponding relationship, determine the first target template area corresponding to the first target inspection area, determine the first object to be inspected corresponding to the first target template area, and use the inspection information corresponding to the first target inspection area as the inspection information of the first object to be inspected. Based on the object detection and template matching algorithms to match the target inspection area and multiple target template areas, the detection and matching accuracy is high, which can cope with the situation where the deviation between the target inspection image and the target template image is large, and improve the anti-interference ability and accuracy of the inspection.

[0113] In an optional embodiment of this specification, after determining the inspection information of the object to be inspected according to the inspection information and the corresponding relationship, the following steps may further be included: verifying the inspection information of the object to be inspected to obtain a verification result, and presenting the verification result to the user.

[0114] Specifically, the verification result includes, but is not limited to, whether the object to be inspected is operating normally and whether the operating state of the object to be inspected is efficient.

[0115] It should be noted that there are various ways to verify the inspection information of the object to be inspected, which are specifically selected according to the actual situation, and this specification does not make any limitations in this regard. In a possible implementation manner of this specification, the reference information of the object to be inspected can be obtained, the reference information and the inspection information can be compared, and the verification result can be determined. For example, if the reference information is operating efficiently at 26°C and the inspection information is 15°C, then it is determined that the verification result is that the object to be inspected is not operating efficiently. In another possible implementation manner of this specification, the verification result can be directly determined based on the inspection information. For example, if the indicator light of the object to be inspected is red, then it is determined that the verification result is that the object to be inspected is operating abnormally.

[0116] In practical applications, after obtaining the verification result, the verification result can be stored and can also be presented to the user. When presenting the verification result, not only the verification result can be presented, but also the inspection information, the verification process, the target inspection image, and the target template image can be presented.

[0117] Applying the solution of the embodiment of this specification to verify the inspection information of the object to be inspected, obtain the verification result, and present the verification result to the user. Visual inspection is realized, and the user experience is improved.

[0118] It should be noted that the inspection method proposed in the embodiments of this specification can be applied to various scenarios, such as traffic navigation scenarios and data center scenarios. Taking the data center scenario as an example, the target inspection points include but are not limited to distribution rooms, strong electrical rooms, weak electrical rooms, air handling units (AHUs), and pump rooms. The objects to be inspected can be referred to as data devices to be inspected, and the data devices to be inspected include but are not limited to medium-voltage switchgear, low-voltage switchgear, distribution boxes, three-phase uninterruptible power supplies (UPSs), row head cabinets, and high-voltage direct current (HVDC) power supplies, etc. The target inspection areas include but are not limited to indicator lights, status knobs, voltmeters, pressure gauges, valves, display screens, and sundries, etc.

[0119] Combined with FIG. 4 below, taking the application of the inspection method provided in this specification in a data center as an example, the inspection method will be further described. Among them, FIG. 4 shows a flowchart of a data center inspection method provided by an embodiment of this specification, which specifically includes the following steps 402 to step 408.

[0120] Step 402: Obtain the target inspection image of the target inspection point in the data center.

[0121] Step 404: Parse out multiple target inspection areas and the inspection information corresponding to the multiple target inspection areas respectively from the target inspection image.

[0122] Step 406: Match the multiple target inspection areas with the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection areas and the target template areas, where the target template areas correspond one-to-one with the data devices to be inspected.

[0123] Step 408: Determine the inspection information of the data devices to be inspected according to the inspection information and the corresponding relationship.

[0124] It should be noted that the implementation manners of steps 402 to 408 are the same as those of the above steps 202 to 208, and the embodiments of this specification will not elaborate further.

[0125] Applying the solution of the embodiments of this specification, by matching the target inspection areas with multiple target template areas, automatic inspection is realized from the perspective of areas, which can cope with the situation where the deviation between the target inspection image and the target template image is large on the basis of improving the inspection efficiency of the data center, and further improves the anti-interference ability and inspection accuracy of the data center inspection.

[0126] Refer to FIG. 5. FIG. 5 shows a flowchart of the processing procedure of an inspection method provided by an embodiment of this specification, which specifically includes inspection point deployment, template image configuration, inspection image capture, target detection and information recognition, and template matching;

[0127] Inspection point deployment: Before scheduling the inspection device to start inspection, inspection point deployment can be performed: Schedule the inspection device to move to multiple inspection points for shooting, obtain the inspection images of each of the multiple inspection points, and store the inspection device status information when the inspection device shoots at the multiple inspection points. Specifically, the inspection device can be scheduled to move to multiple inspection points, and the camera can be controlled to rotate to a suitable angle, and the camera parameters can be adjusted to make the inspection pictures taken by the inspection device meet the preset shooting requirements. Finally, the inspection device status information is recorded;

[0128] Template image configuration: Identify the types of multiple inspection points, and determine at least one inspection point type; Evaluate the quality of the inspection images corresponding to the first inspection point type to determine the evaluation result of the inspection images; According to the evaluation result, filter out the template images corresponding to the first inspection point type (filter out the template images); Divide multiple target template areas from the template images, and associate the target template areas with the devices to be inspected, so as to associate the target inspection images and the target template areas during inspection, and realize the association between the target inspection images and the devices to be inspected;

[0129] Inspection image capture: After receiving the inspection task, schedule the inspection device to move to the target inspection point according to the inspection device status information recorded during inspection point deployment, and adjust the camera angle and shooting parameters to capture the target inspection image;

[0130] Target detection and information recognition: Perform image processing on the target inspection image, such as image cropping and image super-resolution processing, to obtain the updated target inspection image; Input the updated target inspection image into the region detection model for target detection to obtain multiple target inspection regions and the region types respectively corresponding to the multiple target inspection regions; From multiple pre-set recognition strategies, filter out the target recognition strategy; Use the target recognition strategy to perform information recognition on the multiple target inspection regions to obtain the inspection information respectively corresponding to the multiple target inspection regions;

[0131] Template matching: Define the region matching feasible region, determine the variables affecting the matching relationship, construct the matching relationship objective function, and then use the optimization algorithm to solve the corresponding relationship between the target inspection region and the target template region; According to the inspection information and the corresponding relationship, determine the inspection information of the object to be inspected, that is, generate the inspection result.

[0132] Applying the solution of the embodiments of this specification, target detection is performed through a region detection model to obtain multiple target inspection regions and the region types corresponding to the multiple target inspection regions respectively. Then, from the multiple target template regions, the target template region corresponding to the target inspection region is searched to obtain the correspondence between the target inspection region and the target template region, turning the factors determining the template matching error into the accuracy of the target detection algorithm and the accuracy of the template matching algorithm. Through algorithm optimization, high-precision inspection can still be maintained even when there is a large deviation between the target inspection image and the target template image. Moreover, a template image is configured for inspection points of the same type, shortening the deployment time of inspection points and facilitating large-scale deployment and maintenance of inspection points efficiently.

[0133] Corresponding to the above embodiments of the inspection method, this specification also provides an embodiment of an inspection device. FIG. 6 shows a schematic structural diagram of an inspection device provided by an embodiment of this specification. As shown in FIG. 6, the device includes a first acquisition module 602, a first parsing module 604, a first matching module 606, and a first determination module 608.

[0134] The first acquisition module 602 is configured to acquire a target inspection image of a target inspection point.

[0135] The first parsing module 604 is configured to parse out multiple target inspection regions and the inspection information corresponding to the multiple target inspection regions respectively from the target inspection image.

[0136] The first matching module 606 is configured to match the multiple target inspection regions with the multiple target template regions in the target template image to determine the correspondence between the target inspection region and the target template region, where the target template region corresponds to the object to be inspected one by one.

[0137] The first determination module 608 is configured to determine the inspection information of the object to be inspected according to the inspection information and the correspondence.

[0138] Optionally, the first acquisition module 602 is further configured to acquire the inspection device status information corresponding to the target inspection point, where the inspection device status information includes at least one of device position information, device attitude information, and device shooting parameters; according to the inspection device status information, the inspection device is scheduled to move to the target inspection point for shooting to obtain the target inspection image.

[0139] Optionally, the device further includes: an identification module configured to identify the type of a target inspection point to obtain the target inspection point type; and screen out a target template image from at least one template image corresponding to the inspection point types according to the target inspection point type.

[0140] Optionally, the device further includes: a screening module configured to obtain the inspection images of multiple inspection points respectively; identify the types of the multiple inspection points to determine at least one inspection point type; and screen out the template image corresponding to the first inspection point type from the inspection images corresponding to the first inspection point type, where the first inspection point type is any one of the at least one inspection point type.

[0141] Optionally, the screening module is further configured to perform quality assessment on the inspection images corresponding to the first inspection point type to determine the evaluation result of the inspection images, where the evaluation result is used to reflect the quality level of the inspection images; and screen out the template image corresponding to the first inspection point type according to the evaluation result.

[0142] Optionally, the screening module is further configured to dispatch an inspection device to move to multiple inspection points for photographing to obtain the inspection images of the multiple inspection points respectively; and the device further includes: a storage module configured to store the status information of the inspection device when photographing at the multiple inspection points.

[0143] Optionally, the first parsing module 604 is further configured to input the target inspection image into a region detection model to obtain multiple target inspection regions and the region types respectively corresponding to the multiple target inspection regions; screen out the target recognition strategy corresponding to the region type from multiple pre-set recognition strategies; and perform information recognition on the multiple target inspection regions by using the target recognition strategy to obtain the inspection information respectively corresponding to the multiple target inspection regions.

[0144] Optionally, the device further includes: a cropping module configured to crop the target inspection image to obtain the cropped target inspection image; and perform super-resolution processing on the cropped target inspection image to obtain the updated target inspection image.

[0145] Optionally, the first matching module 606 is further configured to obtain a pre-defined region matching feasible region, where the region matching feasible region is defined based on the multiple target inspection regions and the multiple target template regions in the target template image; determine the variables affecting the matching relationship and construct a matching relationship objective function; and solve the corresponding relationship between the target inspection region and the target template region based on the region matching feasible region and the matching relationship objective function.

[0146] Optionally, the first determination module 608 is further configured to determine a first target template area corresponding to the first target inspection area according to the correspondence relationship, where the first target inspection area is any one of a plurality of target inspection areas, and the first target template area is a target template area in the plurality of target template areas that matches the first target inspection area; determine a first object to be inspected corresponding to the first target template area, and use the inspection information corresponding to the first target inspection area as the inspection information of the first object to be inspected.

[0147] Optionally, the apparatus further includes: a verification module configured to verify the inspection information of the object to be inspected, obtain a verification result, and display the verification result to the user.

[0148] Applying the solution of the embodiments of this specification, by matching the target inspection area and a plurality of target template areas, automatic inspection is realized from the perspective of the area. On the basis of improving the inspection efficiency, it can cope with the situation where the deviation between the target inspection image and the target template image is relatively large, further improving the anti-interference ability and inspection accuracy of the inspection.

[0149] The above is a schematic solution of an inspection apparatus according to this embodiment. It should be noted that the technical solution of this inspection apparatus and the technical solution of the above inspection method belong to the same concept. For the details not described in the technical solution of the inspection apparatus, reference can be made to the description of the technical solution of the above inspection method.

[0150] Corresponding to the above data center inspection method embodiments, this specification also provides data center inspection apparatus embodiments. FIG. 7 shows a schematic structural diagram of a data center inspection apparatus provided by an embodiment of this specification. As shown in FIG. 7, the apparatus includes a second acquisition module 702, a second parsing module 704, a second matching module 706, and a second determination module 708.

[0151] The second acquisition module 702 is configured to acquire a target inspection image of a target inspection point in the data center.

[0152] The second parsing module 704 is configured to parse out a plurality of target inspection areas and inspection information respectively corresponding to the plurality of target inspection areas from the target inspection image.

[0153] The second matching module 706 is configured to match the plurality of target inspection areas with the plurality of target template areas in the target template image to determine the correspondence relationship between the target inspection area and the target template area, where the target template area corresponds to the data device to be inspected one by one.

[0154] The second determination module 708 is configured to determine the inspection information of the data devices to be inspected according to the inspection information and the corresponding relationship.

[0155] By applying the solution of the embodiments of this specification, through matching the target inspection area and multiple target template areas, automatic inspection is realized from the perspective of the area. On the basis of improving the inspection efficiency of the data center, it can cope with the situation where there is a large deviation between the target inspection image and the target template image, further improving the anti-interference ability and inspection accuracy of the data center inspection.

[0156] The above is a schematic solution of a data center inspection device in this embodiment. It should be noted that the technical solution of this data center inspection device and the technical solution of the above data center inspection method belong to the same concept. For the details not described in detail in the technical solution of the data center inspection device, reference can be made to the description of the technical solution of the above data center inspection method.

[0157] FIG. 8 shows a structural block diagram of a computing device provided by an embodiment of this specification. The components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 through a bus 830, and a database 850 is used to store data.

[0158] The computing device 800 further includes an access device 840, and the access device 840 enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of wired or wireless network interfaces (e.g., a network interface card (NIC)), such as an IEEE802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (WiMAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0159] In one embodiment of this specification, the above components of the computing device 800 and other components not shown in FIG. 8 may also be connected to each other, for example, via a bus. It should be understood that the block diagram of the computing device shown in FIG. 8 is for illustrative purposes only and is not a limitation on the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0160] The computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.) or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 800 may also be a mobile or stationary server.

[0161] Wherein, the processor 820 is configured to execute the following computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the above inspection method or the data center inspection method are implemented.

[0162] The above is a schematic solution of a computing device in this embodiment. It should be noted that the technical solution of this computing device and the technical solutions of the above inspection method and the data center inspection method belong to the same concept. For the details not described in detail in the technical solution of the computing device, reference may be made to the descriptions of the technical solutions of the above inspection method or the data center inspection method.

[0163] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above inspection method or data center inspection method.

[0164] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned inspection method and data center inspection method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned inspection method or data center inspection method.

[0165] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is instructed to execute the steps of the above inspection method or data center inspection method.

[0166] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the above-mentioned inspection method and data center inspection method belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be referred to the description of the technical scheme of the above-mentioned inspection method or data center inspection method.

[0167] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order shown. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0168] The computer instructions include computer program code, which may be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, external hard drives, magnetic disks, optical discs, computer memories, read-only memories (ROM), random access memories (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0169] It should be noted that for the foregoing method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the embodiments of this specification are not limited by the described order of actions, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this specification.

[0170] In the above embodiments, the descriptions of the various embodiments have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0171] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can well understand and utilize this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

Claims 1. A patrol inspection method, comprising: Obtain the target inspection image of the target inspection point; Parse out multiple target inspection areas and the inspection information corresponding to the multiple target inspection areas from the target inspection image; Match the multiple target inspection areas with the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection area and the target template area, where the target template area corresponds one-to-one with the object to be inspected; determine the inspection information of the object to be inspected according to the inspection information and the corresponding relationship.

2. The inspection method according to claim 1, wherein the obtaining of the target inspection image of the target inspection point comprises: Obtain the inspection device status information corresponding to the target inspection point, where the inspection device status information includes at least one of device position information, device attitude information, and device shooting parameters; according to the inspection device status information, schedule the inspection device to move to the target inspection point for shooting to obtain the target inspection image.

3. The inspection method according to claim 1 or 2, before matching the multiple target inspection areas and the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection area and the target template area, further includes: Perform type recognition on the target inspection point to obtain the target inspection point type; According to the target inspection point type, select the target template image from the template images corresponding to at least one inspection point type.

4. The inspection method according to claim 3, before screening out the target template image from the template images corresponding to at least one inspection point type according to the target inspection point type, further includes: Obtain the inspection images of multiple inspection points respectively; Perform type recognition on the multiple inspection points to determine at least one inspection point type; Select the template image corresponding to the first inspection point type from the inspection images corresponding to the first inspection point type, where the first inspection point type is any one of the at least one inspection point type.

5. The inspection method according to claim 4, wherein screening out the template image corresponding to the first inspection point type from the inspection images corresponding to the first inspection point type includes: Perform quality evaluation on the inspection images corresponding to the first inspection point type to determine the evaluation result of the inspection images, where the evaluation result is used to reflect the quality of the inspection images; select the template image corresponding to the first inspection point type according to the evaluation result.

6. The inspection method according to claim 4, wherein the obtaining of the inspection images of the respective inspection points comprises: Schedule the inspection device to move to the multiple inspection points for shooting to obtain the inspection images of the multiple inspection points respectively; After obtaining the inspection images of the multiple inspection points respectively, it further includes: storing the inspection device status information when the inspection device shoots at the multiple inspection points.

7. According to the inspection method described in claim 1, the step of parsing out multiple target inspection areas and the inspection information corresponding to the multiple target inspection areas from the target inspection image includes: Input the target inspection image into the region detection model to obtain multiple target inspection areas and the region types corresponding to the multiple target inspection areas respectively; Select the target recognition strategy corresponding to the region type from a plurality of pre-set recognition strategies; Use the target recognition strategy to perform information recognition on the multiple target inspection areas to obtain the inspection information corresponding to the multiple target inspection areas respectively.

8. Before inputting the target inspection image into the region detection model to obtain multiple target inspection regions and the region types respectively corresponding to the multiple target inspection regions according to the inspection method described in claim 7, the method further includes: Crop the target inspection image to obtain the cropped target inspection image; Perform super-resolution processing on the cropped target inspection image to obtain the updated target inspection image.

9. The inspection method according to claim 1, wherein the matching of the multiple target inspection areas and the multiple target template areas in the target template image to determine the corresponding relationship between the target inspection area and the target template area includes: Obtain a predefined region matching feasible region, where the region matching feasible region is defined based on the multiple target inspection regions and multiple target template regions in the target template image; determine variables affecting the matching relationship, and construct a matching relationship objective function; based on the region matching feasible region and the matching relationship objective function, solve the corresponding relationship between the target inspection region and the target template region.

10. The inspection method according to claim 1, wherein determining the inspection information of the object to be inspected according to the inspection information and the corresponding relationship comprises: According to the corresponding relationship, determine the first target template region corresponding to the first target inspection region, where the first target inspection region is any one of the multiple target inspection regions, and the first target template region is the target template region in the multiple target template regions that matches the first target inspection region; determine the first object to be inspected corresponding to the first target template region, and use the inspection information corresponding to the first target inspection region as the inspection information of the first object to be inspected.

11. The inspection method according to claim 1, after determining the inspection information of the object to be inspected according to the inspection information and the corresponding relationship, further comprising: Verify the inspection information of the object to be inspected, obtain a verification result, and display the verification result to the user.

12. A data center inspection method, comprising: Obtain the target inspection image of the target inspection point in the data center. Parse out multiple target inspection regions and the inspection information corresponding to the multiple target inspection regions respectively from the target inspection image. Match the multiple target inspection regions with the multiple target template regions in the target template image, and determine the corresponding relationship between the target inspection region and the target template region, where the target template region corresponds to the data device to be inspected one by one; according to the inspection information and the corresponding relationship, determine the inspection information of the data device to be inspected.

13. A computing device, comprising: A memory and a processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method according to any one of claims 1 to 11 or claim 12 are implemented.

14. A computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the method according to any one of claims 1 to 11 or claim 12 are implemented. ​

Images