Equipment inspection method and device, computer equipment, storage medium and program product

Abstract

The invention relates to an equipment inspection method and device, computer equipment, a storage medium and a program product. The method comprises the steps of obtaining a device image of a to-be-detected device through an augmented reality AR device, determining a device area image in the device image, determining a device type of the to-be-detected device according to the device area image and a device identification model, and then performing routing inspection on the to-be-detected device according to the device type to obtain a routing inspection result. According to the embodiment of the invention, the equipment type of the to-be-detected equipment is identified based on the equipment identification model, so that the inspection result is automatically determined based on the equipment type, the inspection efficiency is improved, and the inspection accuracy and reliability can be improved by automatically determining the inspection result based on artificial intelligence.

Description

Technical Field

[0001] The present application relates to the technical field of substation inspection, and in particular to an equipment inspection method, device, computer equipment, storage medium and program product. Background Art

[0002] With the rapid development of power grids, the number of expanded and newly built substations is increasing, which puts higher requirements on the work quality and business skills of substation operation and maintenance personnel. Equipment inspection, as one of the important contents of substation operation and maintenance management, is of great significance for timely detection of equipment abnormalities and defects, prevention of accidents, and ensuring the safe operation of equipment and power grids.

[0003] Traditional substation equipment inspection mainly relies on regular manual inspections by inspection personnel, which results in low inspection efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide an equipment inspection method, device, computer equipment, storage medium and program product that can improve inspection efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a device inspection method. The method comprises:

[0006] Acquire a device image of the device to be inspected through an augmented reality (AR) device;

[0007] determining a device region image in the device image;

[0008] Determine the device type of the device to be detected according to the device area image and the device recognition model;

[0009] The device to be detected is inspected according to the device type to obtain an inspection result.

[0010] In one embodiment, determining the device type of the device to be detected according to the device area image and the device recognition model includes:

[0011] Dividing the device area image into a plurality of data blocks;

[0012] Determining the significance of each of the data blocks based on a convolutional neural network in the device recognition model;

[0013] Determine a target data block from each of the data blocks according to each of the saliencies;

[0014] Focusing on the target data block, the device type of the device to be detected is determined based on the random forest module in the device identification model.

[0015] In one embodiment, the device to be detected is inspected according to the device type to obtain an inspection result, including:

[0016] Obtaining the input operation index of the device to be detected;

[0017] Determine a preset interval corresponding to the device type;

[0018] If the operating index is not within the preset range, it is determined that the inspection result is that the device to be inspected has a fault.

[0019] In one of the embodiments, when the inspection result indicates that the device to be inspected has a fault, an operation instruction video corresponding to the device type is determined;

[0020] The operation guidance video is displayed through the AR device; the operation guidance video is used to instruct the operation and maintenance personnel to perform the fault handling operation according to the operation guidance video.

[0021] In one of the embodiments, the device image of the device to be detected is sent to a control device corresponding to the device type;

[0022] A device inspection instruction sent by the control device is received; the device inspection instruction is used to instruct to perform a fault handling operation according to the device inspection instruction.

[0023] In one embodiment, determining the device region image in the device image includes:

[0024] The device area image is determined based on the bilateral filtering method.

[0025] In a second aspect, the present application also provides an equipment inspection device. The device comprises:

[0026] An acquisition module is used to acquire a device image of a device to be detected through an augmented reality (AR) device;

[0027] A first determining module, used for determining a device area image in the device image;

[0028] A second determination module is used to determine the device type of the device to be detected according to the device area image and the device recognition model;

[0029] The inspection module is used to inspect the device to be inspected according to the device type and obtain the inspection result.

[0030] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0031] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.

[0032] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of any of the above methods when executed by a processor.

[0033] The above-mentioned equipment inspection method, device, computer equipment, storage medium and program product obtain the equipment image of the equipment to be inspected through the augmented reality AR device, determine the equipment area image in the equipment image, determine the equipment type of the equipment to be inspected according to the equipment area image and the equipment recognition model, and then inspect the equipment to be inspected according to the equipment type to obtain the inspection result. In the embodiment of the present application, the equipment type of the equipment to be inspected is identified based on the equipment recognition model, and the inspection result is automatically determined based on the equipment type, thereby improving the inspection efficiency, and the automatic determination of the inspection result based on artificial intelligence can also improve the accuracy and reliability of the inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1 is an internal structure diagram of a computer device provided in an embodiment of the present application;

[0035] FIG2 is a schematic diagram of a flow chart of an equipment inspection method provided in an embodiment of the present application;

[0036] FIG3 is a schematic diagram of a flow chart of a method for determining a device type provided in an embodiment of the present application;

[0037] FIG4 is a flow chart of a method for determining an inspection result provided in an embodiment of the present application;

[0038] FIG5 is a schematic diagram of a flow chart of a method for displaying an operation guidance video according to an embodiment of the present application;

[0039] FIG6 is a flow chart of a method for receiving a device inspection instruction provided in an embodiment of the present application;

[0040] FIG7 is a flow chart of an equipment inspection method in a power grid production field provided by an embodiment of the present application;

[0041] FIG8 is a structural block diagram of an equipment inspection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] With the rapid development of power grids, the number of expanded and newly built substations is increasing, which puts higher requirements on the work quality and business skills of substation operation and maintenance personnel. Equipment inspection, as one of the important contents of substation operation and maintenance management, is of great significance for timely detection of equipment abnormalities and defects, prevention of accidents, and ensuring the safe operation of equipment and power grids.

[0044] Traditional substation equipment inspection mainly relies on regular manual inspections by inspection personnel, which results in low inspection efficiency.

[0045] The device inspection method provided in the embodiment of the present application can be applied in the application environment as shown in Figure 1. Figure 1 is an internal structure diagram of a computer device provided in the embodiment of the present application. The computer device can be a server, and its internal structure diagram can be as shown in Figure 1. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a device inspection method is implemented.

[0046] Those skilled in the art will understand that the structure shown in FIG. 1 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0047] In one embodiment, as shown in FIG. 2 , FIG. 2 is a flow chart of a device inspection method provided in an embodiment of the present application, which can be applied to the computer device in FIG. 1 , and the method includes the following steps:

[0048] S201, obtaining a device image of a device to be detected through an augmented reality (AR) device.

[0049] Exemplarily, the AR device may be, for example, a wearable portable AR terminal device, such as AR glasses, an AR helmet, and the like.

[0050] In one embodiment, the operation and maintenance personnel can wear the AR device to enter the area to be inspected. After entering the area to be inspected, the AR device can collect the device image of the device to be inspected in the area to be inspected, and then send the device image to the computer device as shown in Figure 1.

[0051] Optionally, the geographical location of the AR device can be obtained, and when the AR device is detected to enter the area to be detected, the check-in operation is automatically performed. At the same time, after the AR device is detected to enter the area to be detected, an inspection instruction can also be generated, and the inspection instruction can be displayed through the AR device to guide the operation and maintenance personnel to the location of the device to be detected through the inspection instruction. When the operation and maintenance personnel are detected to arrive at the location of the device to be detected, the device image of the device to be detected can be obtained through the AR device, and the operation and maintenance personnel can be instructed to collect the operating indicators of the device to be detected.

[0052] The operating indicators of the device to be detected may include, for example, pressure, temperature, voltage, power and other indicators when the device to be detected is running.

[0053] S202: Determine a device area image in the device image.

[0054] Optionally, the device image may be preprocessed by denoising, contrast enhancement, grayscale conversion, etc., and then the device area and the background area may be separated by an image segmentation algorithm to obtain a device area image.

[0055] Exemplarily, the device image may be preprocessed by bilateral filtering to achieve denoising of the device image and retain information about the device region boundary, which can improve the accuracy and precision of the segmented device region image.

[0056] S203: Determine the device type of the device to be detected according to the device area image and the device recognition model.

[0057] In one embodiment, the device area image can be divided into multiple data blocks, and the first feature data of the device area image and the second feature data of the device area boundary can be extracted based on the pre-trained convolutional neural network in the device recognition model. Then, the significance of each data block is determined based on the first feature data and the second feature data, and the target data block is determined based on the significance of each data block. Finally, with the target data block as the focus, the first feature data and the second feature data are classified based on the random forest module in the device recognition model to determine the device type of the device to be detected.

[0058] S204, inspect the device to be inspected according to the device type and obtain the inspection result.

[0059] Optionally, the operating indicators of the equipment to be tested input by the operation and maintenance personnel can be obtained, and then the preset interval corresponding to the equipment type can be obtained. The operating indicators are tested according to the preset interval. If the operating indicators are not within the preset interval, the inspection result is determined to be that the equipment to be tested has a fault; if the operating indicators are within the preset interval, the inspection result is determined to be that the equipment to be tested is operating normally.

[0060] Alternatively, the device image of the device to be inspected can also be input into a pre-trained image recognition model to identify the operating indicators of the device to be inspected through the image recognition model, and then the preset interval corresponding to the device type is obtained, and the operating indicators are inspected according to the preset interval. If the operating indicators are not within the preset interval, the inspection result is determined to be that the device to be inspected has a fault; if the operating indicators are within the preset interval, the inspection result is determined to be that the device to be inspected is operating normally.

[0061] In the embodiment of the present application, an augmented reality AR device is used to obtain a device image of the device to be detected, and a device area image in the device image is determined. The device type of the device to be detected is determined based on the device area image and the device recognition model, and then the device to be detected is inspected according to the device type to obtain the inspection result. In the embodiment of the present application, the device type of the device to be detected is identified based on the device recognition model, and the inspection result is automatically determined based on the device type, thereby improving the inspection efficiency, and the automatic determination of the inspection result based on artificial intelligence can also improve the accuracy and reliability of the inspection.

[0062] Referring to FIG. 3 , FIG. 3 is a flow chart of a method for determining a device type provided in an embodiment of the present application. This embodiment relates to a possible implementation method of determining the device type of a device to be detected based on a device area image and a device recognition model. Based on the above embodiment, the above S203 includes the following steps:

[0063] S301, dividing the device area image into multiple data blocks.

[0064] In one embodiment, the device area image may be divided into a plurality of data blocks based on the following formula (1):

[0065] (1)

[0066] In formula (1), is the centroid of the data block, is the pixel segmentation function, is the windowing function, is the size of the data block, For data blocks.

[0067] S302: Determine the significance of each data block based on the convolutional neural network in the device recognition model.

[0068] Optionally, the first feature data of the device area image and the second feature data of the device area boundary can be extracted based on the pre-trained convolutional neural network in the device recognition model. Then, the saliency calculation is performed based on the first feature data and the second feature data to determine the saliency of each data block, that is,

[0069] In the embodiment of the present application, the accuracy of the convolutional neural network can be tested by a normalized distribution function, and the normalized function can be expressed as the following formula (2):

[0070] (2)

[0071] In formula (2), is the weight vector, is the energy function, For data blocks, is a distinctive mark. Can be 0 or 1. When , the performance is not significant; when When, it is manifested significantly.

[0072] Alternatively, the data block can be determined by the following formula (3): The significance value of :

[0073] (3)

[0074] For example, assume that the data block of the local salient state is The set of significance values ​​is , then the energy function can be expressed as the following formula (4):

[0075] (4)

[0076] In formula (4), For data blocks The distinctive mark of For data blocks The distinctive mark of and are adjacent data blocks, is the data energy term, is the data smoothing term.

[0077] S303: Determine a target data block from each data block according to each saliency.

[0078] Exemplarily, a preset threshold may be set, and then a data block corresponding to a significance greater than the preset threshold is determined as a target data block.

[0079] S304, focusing on the target data block, and determining the device type of the device to be detected based on the random forest module in the device identification model.

[0080] In the embodiment of the present application, the target data block can be used as the focus to determine the device type of the device to be detected based on the random forest algorithm.

[0081] In one embodiment, the function expression of the random forest algorithm can be, for example, , is the first Any tree in the random forest can be trained based on the data block samples.

[0082] For example, a data block sample can be represented as .in, is the device tag, For device samples. When there is When there are three types of equipment, the Gini coefficient can be expressed as: Where Represents the ratio of the sample size, that is, the proportion of the sample size of this device type to the total sample size.

[0083] Generally, when the device types in the data block samples are the same, the Gini coefficient is at its minimum; when the device types in the data block samples are different and the data block samples of each device type are evenly distributed, the Gini coefficient is larger. When selecting the features of the device image, each image dimension is processed separately. When the intra-class spread is small, the larger the inter-class spread is, the more accurate the sample classification results will be. If the distribution coefficient is at a larger value, the corresponding dimensional features are more in line with the standards and requirements of image classification.

[0084] In an embodiment of the present application, the device area image is divided into multiple data blocks, the significance of each data block is determined based on the convolutional neural network in the device recognition model, and the target data block is determined from each data block according to the significance. With the target data block as the focus, the device type of the device to be detected is determined based on the random forest module in the device recognition model. In this way, the significance of each data block can be determined while considering the information differentiation characteristics between the device area and the device area boundary, thereby improving the accuracy of device type recognition based on the significance of the data block and the random forest algorithm.

[0085] Referring to Figure 4, Figure 4 is a flow chart of a method for determining inspection results provided by an embodiment of the present application. This embodiment involves a possible implementation method of how to inspect the device to be inspected according to the device type and obtain the inspection result. Based on the above embodiment, the above S204 includes the following steps:

[0086] S401, obtaining the input operation index of the device to be detected.

[0087] Optionally, when it is detected that the operation and maintenance personnel arrive at the location of the device to be detected and instruct the operation and maintenance personnel to collect the operation index of the device to be detected, the operation index of the device to be detected input by the operation and maintenance personnel can be obtained.

[0088] Exemplarily, the operating indicators of the equipment to be tested may include, for example, the reading of a pressure gauge, temperature, humidity, power, etc. of the equipment to be tested.

[0089] S402: Determine a preset interval corresponding to the device type.

[0090] In the embodiment of the present application, different preset intervals can be set for different device types in advance, and the device types and preset intervals can be stored in a database in correspondence. After the device type of the device to be detected is determined, the corresponding preset interval can be searched from the database according to the device type of the device to be detected.

[0091] S403: If the operation index is not within the preset range, it is determined that the inspection result is that the device to be inspected has a fault.

[0092] Exemplarily, the operating indicator can be detected according to a preset interval. If the operating indicator is not in the preset interval, the inspection result is determined to be that the equipment to be detected has a fault; if the operating indicator is in the preset interval, the inspection result is determined to be that the equipment to be detected is operating normally.

[0093] When it is determined that the inspection result shows that the equipment to be inspected has a fault, a prompt message can be generated and displayed through the AR device to remind the operation and maintenance personnel that the equipment to be inspected has a fault so that the operation and maintenance personnel can take corresponding processing measures in time.

[0094] In an embodiment of the present application, the operating indicators of the input equipment to be detected are obtained, and a preset interval corresponding to the equipment type is determined. If the operating indicators are not within the preset interval, the inspection result is determined to be that there is a fault in the equipment to be detected, so that fault detection can be automatically performed according to the preset interval corresponding to the equipment type, thereby improving the efficiency and accuracy of equipment inspection.

[0095] Referring to Figure 5, Figure 5 is a flow chart of a method for displaying an operation guidance video provided in an embodiment of the present application. Based on the above embodiment, the method further includes the following steps:

[0096] S501, when the inspection result shows that the device to be inspected has a fault, determine an operation instruction video corresponding to the device type.

[0097] In one embodiment, different operation instruction videos can be configured for different equipment types in advance, and the equipment types and operation instruction videos can be stored in a database in correspondence. When the inspection result is determined to be a fault in the equipment to be detected, the corresponding operation instruction video can be first searched from the database according to the equipment type of the equipment to be detected.

[0098] Exemplarily, the operation instruction video may include, for example, a demonstration video of operation steps such as startup, disassembly, maintenance, and debugging when inspecting the equipment to be inspected.

[0099] S502, displaying an operation guidance video through an AR device.

[0100] The operation guidance video is used to instruct the operation and maintenance personnel to perform troubleshooting operations according to the operation guidance video.

[0101] For example, an operation guidance video can be displayed through an AR device, and the operation and maintenance personnel can perform troubleshooting operations such as starting up, disassembling, repairing, and debugging the equipment to be tested according to the content of the operation guidance video.

[0102] In an embodiment of the present application, when the inspection result shows that the equipment to be inspected has a fault, an operation guidance video corresponding to the equipment type is determined, and the operation guidance video is displayed through an AR device, thereby instructing the operation and maintenance personnel to perform fault handling operations according to the operation guidance video. This can reduce the training cost and operation difficulty of the operation and maintenance personnel, and can also improve the efficiency and accuracy of equipment inspections.

[0103] Referring to Figure 6, Figure 6 is a flow chart of a method for receiving a device inspection instruction provided in an embodiment of the present application. Based on the above embodiment, the method further includes the following steps:

[0104] S601: When the inspection result shows that the device to be inspected has a fault, the device image of the device to be inspected is sent to a control device corresponding to the device type.

[0105] Optionally, the device image of the device to be detected may include pictures, videos, etc. Since the AR device is a wearable portable AR terminal device, the device image of the device to be detected collected by the AR device may be an image from the first-person perspective of the operation and maintenance personnel.

[0106] In a possible implementation, when it is determined that the inspection result shows that the device to be inspected has a fault and the fault cannot be eliminated according to the operation guidance video, the device image of the device to be inspected can be sent to a control device corresponding to the device type via a wireless communication network.

[0107] S602, receiving a device inspection instruction sent by a control device.

[0108] The device inspection instruction is used to instruct to perform a fault handling operation according to the device inspection instruction.

[0109] Exemplarily, when the control device corresponding to the device type receives the device image of the device to be detected, it can display the device image, and the remote expert or engineer can view the device image of the device to be detected and input the device inspection instruction to the control device. After obtaining the device inspection instruction, the control device can send the device inspection instruction to the computer device in Figure 1 through the wireless communication network, and display the device inspection instruction through the AR device, so as to instruct the operation and maintenance personnel to perform the fault handling operation according to the operation guidance video through the device inspection instruction.

[0110] In an embodiment of the present application, when the inspection result shows that the device to be inspected has a fault, the device image of the device to be inspected is sent to a control device corresponding to the device type, and an equipment inspection instruction sent by the control device is received, thereby instructing the operation and maintenance personnel to perform fault handling operations according to the operation guidance video. When the operation and maintenance personnel are unable to independently eliminate the equipment fault, they can seek help from remote experts in a timely manner, thereby improving the efficiency and accuracy of equipment inspections.

[0111] Based on the above embodiment, the above S202 can be implemented by the following method:

[0112] The device area image is determined based on the bilateral filtering method.

[0113] Optionally, the device image may be preprocessed by a bilateral filtering method to achieve denoising of the device image and retain information on the device region boundary, which can improve the accuracy and precision of the segmented device region image.

[0114] After the device area image is determined by the bilateral filtering method, the device area image can be divided into multiple data blocks by the following formula (5):

[0115] (5)

[0116] In formula (2), is the centroid of the data block, is the segmentation function of the bilateral filtered image, is the windowing function, is the size of the data block, For data blocks.

[0117] In the embodiment of the present application, the device area image is determined based on the bilateral filtering method, so that the device image can be denoised and the information of the device area boundary is retained, thereby improving the accuracy and precision of the segmented device area image.

[0118] Referring to FIG. 7 , FIG. 7 is a flow chart of a method for inspecting equipment in a power grid production field provided by an embodiment of the present application. The method comprises the following steps:

[0119] S701, obtaining a device image of a device to be detected through an AR device.

[0120] S702: Determine the device type of the device to be detected according to the device area image and the device recognition model.

[0121] S703: Obtain the input operation index of the device to be detected.

[0122] S704: If the operation index is not within the preset interval corresponding to the device type, it is determined that the inspection result is that the device to be inspected has a fault.

[0123] S705: When the inspection result shows that the device to be inspected has a fault, determine an operation instruction video corresponding to the device type.

[0124] S706, displaying an operation guidance video through the AR device.

[0125] S707: When the inspection result shows that the device to be inspected has a fault, the device image of the device to be inspected is sent to a control device corresponding to the device type.

[0126] S708, receiving a device inspection instruction sent by the control device.

[0127] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0128] Based on the same inventive concept, the embodiment of the present application also provides an equipment inspection device for implementing the equipment inspection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more equipment inspection device embodiments provided below can refer to the limitations on the equipment inspection method above, and will not be repeated here.

[0129] In one embodiment, as shown in FIG8 , FIG8 is a structural block diagram of an equipment inspection device provided in an embodiment of the present application, and the device 800 includes:

[0130] The acquisition module 801 is used to acquire a device image of the device to be detected through an augmented reality (AR) device.

[0131] The first determining module 802 is used to determine a device area image in the device image.

[0132] The second determination module 803 is used to determine the device type of the device to be detected according to the device area image and the device recognition model.

[0133] The inspection module 804 is used to inspect the devices to be inspected according to the device type and obtain the inspection results.

[0134] In one embodiment, the second determining module 803 includes:

[0135] The division unit is used to divide the device area image into multiple data blocks.

[0136] The first determination unit is used to determine the significance of each data block based on a convolutional neural network in the device recognition model.

[0137] The second determining unit is used to determine a target data block from each data block according to each saliency.

[0138] The third determination unit is used to determine the device type of the device to be detected based on the random forest module in the device identification model with the target data block as the focus.

[0139] In one embodiment, the inspection module 804 includes:

[0140] The acquisition unit is used to acquire the input operation index of the device to be detected.

[0141] The fourth determining unit is configured to determine a preset interval corresponding to the device type.

[0142] The fifth determining unit is used to determine that the inspection result is that the equipment to be inspected has a fault if the operation index is not within a preset range.

[0143] In one embodiment, the apparatus 800 further includes:

[0144] The third determination module is used to determine the operation guidance video corresponding to the device type when the inspection result shows that the device to be inspected has a fault.

[0145] The display module is used to display an operation guidance video through an AR device; the operation guidance video is used to instruct the operation and maintenance personnel to perform fault handling operations according to the operation guidance video.

[0146] In one embodiment, the apparatus 800 further includes:

[0147] The sending module is used to send the device image of the device to be detected to the control device corresponding to the device type.

[0148] The receiving module is used to receive the device inspection instruction sent by the control device; the device inspection instruction is used to instruct to perform a fault handling operation according to the device inspection instruction.

[0149] In one embodiment, the first determining module 802 includes:

[0150] The sixth determining unit is used to determine the device area image based on the bilateral filtering method.

[0151] Each module in the above-mentioned equipment inspection device can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0152] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0153] Acquire a device image of the device to be inspected through an augmented reality (AR) device;

[0154] determining a device region image in the device image;

[0155] Determine the device type of the device to be detected based on the device area image and the device recognition model;

[0156] The equipment to be tested is inspected according to the equipment type to obtain the inspection results.

[0157] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0158] Divide the device area image into a plurality of data blocks;

[0159] Determine the significance of each data block based on the convolutional neural network in the device recognition model;

[0160] Determine a target data block from each data block according to each saliency;

[0161] Focusing on the target data block, the device type of the device to be detected is determined based on the random forest module in the device identification model.

[0162] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0163] Obtaining the input operation index of the device to be tested;

[0164] Determine a preset interval corresponding to the device type;

[0165] If the operating index is not within the preset range, the inspection result is determined to be that the equipment to be inspected has a fault.

[0166] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0167] If the inspection result shows that the device to be inspected has a fault, determine the operation guidance video corresponding to the device type;

[0168] The operation guidance video is displayed through the AR device; the operation guidance video is used to instruct the operation and maintenance personnel to perform fault handling operations according to the operation guidance video.

[0169] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0170] Sending a device image of the device to be detected to a control device corresponding to the device type;

[0171] A device inspection instruction sent by a receiving control device is used to instruct to perform a fault handling operation according to the device inspection instruction.

[0172] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0173] The device area image is determined based on the bilateral filtering method.

[0174] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0175] Acquire a device image of the device to be inspected through an augmented reality (AR) device;

[0176] determining a device region image in the device image;

[0177] Determine the device type of the device to be detected based on the device area image and the device recognition model;

[0178] The equipment to be tested is inspected according to the equipment type to obtain the inspection results.

[0179] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0180] Divide the device area image into a plurality of data blocks;

[0181] Determine the significance of each data block based on the convolutional neural network in the device recognition model;

[0182] Determine a target data block from each data block according to each saliency;

[0183] Focusing on the target data block, the device type of the device to be detected is determined based on the random forest module in the device identification model.

[0184] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0185] Obtaining the input operation index of the device to be tested;

[0186] Determine a preset interval corresponding to the device type;

[0187] If the operating index is not within the preset range, the inspection result is determined to be that the equipment to be inspected has a fault.

[0188] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0189] If the inspection result shows that the device to be inspected has a fault, determine the operation guidance video corresponding to the device type;

[0190] The operation guidance video is displayed through the AR device; the operation guidance video is used to instruct the operation and maintenance personnel to perform fault handling operations according to the operation guidance video.

[0191] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0192] Sending a device image of the device to be detected to a control device corresponding to the device type;

[0193] A device inspection instruction sent by a receiving control device is used to instruct to perform a fault handling operation according to the device inspection instruction.

[0194] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0195] The device area image is determined based on the bilateral filtering method.

[0196] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0197] Acquire a device image of the device to be inspected through an augmented reality (AR) device;

[0198] determining a device region image in the device image;

[0199] Determine the device type of the device to be detected based on the device area image and the device recognition model;

[0200] The equipment to be tested is inspected according to the equipment type to obtain the inspection results.

[0201] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0202] Divide the device area image into a plurality of data blocks;

[0203] Determine the significance of each data block based on the convolutional neural network in the device recognition model;

[0204] Determine a target data block from each data block according to each saliency;

[0205] Focusing on the target data block, the device type of the device to be detected is determined based on the random forest module in the device identification model.

[0206] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0207] Obtaining the input operation index of the device to be tested;

[0208] Determine a preset interval corresponding to the device type;

[0209] If the operating index is not within the preset range, the inspection result is determined to be that the equipment to be inspected has a fault.

[0210] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0211] If the inspection result shows that the device to be inspected has a fault, determine the operation guidance video corresponding to the device type;

[0212] The operation guidance video is displayed through the AR device; the operation guidance video is used to instruct the operation and maintenance personnel to perform fault handling operations according to the operation guidance video.

[0213] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0214] Sending a device image of the device to be detected to a control device corresponding to the device type;

[0215] A device inspection instruction sent by a receiving control device is used to instruct to perform a fault handling operation according to the device inspection instruction.

[0216] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0217] The device area image is determined based on the bilateral filtering method.

[0218] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0219] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0220] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A device inspection method, characterized in that: The method comprises: Acquire a device image of the device to be inspected through an augmented reality (AR) device; Determine a device area image in the device image; Determine the device type of the device to be detected according to the device area image and the device recognition model; The device to be detected is inspected according to the device type to obtain an inspection result.

2. The method according to claim 1, characterized in that The determining the device type of the device to be detected according to the device area image and the device recognition model includes: Dividing the device area image into a plurality of data blocks; Determining the significance of each of the data blocks based on a convolutional neural network in the device recognition model; Determine a target data block from each of the data blocks according to each of the saliencies; Focusing on the target data block, the device type of the device to be detected is determined based on the random forest module in the device identification model.

3. The method according to claim 1 or 2, characterized in that: The inspecting the device to be inspected according to the device type to obtain the inspection result includes: Obtaining input operation indicators of the device to be detected; Determining a preset interval corresponding to the device type; If the operating indicator is not within the preset range, it is determined that the inspection result is that the device to be inspected has a fault.

4. The method according to claim 3, characterized in that The method further comprises: When the inspection result indicates that the device to be inspected has a fault, determining an operation guidance video corresponding to the device type; The operation guidance video is displayed through the AR device; the operation guidance video is used to instruct the operation and maintenance personnel to perform fault handling operations according to the operation guidance video.

5. The method according to claim 3, characterized in that: The method further comprises: When the inspection result indicates that the device to be inspected has a fault, sending the device image of the device to be inspected to a control device corresponding to the device type; A device inspection instruction sent by the control device is received; the device inspection instruction is used to instruct to perform a fault handling operation according to the device inspection instruction.

6. The method according to claim 1, characterized in that The determining of the device area image in the device image includes: The device area image is determined based on a bilateral filtering method.

7. An equipment inspection device, characterized in that: The device comprises: An acquisition module is used to acquire a device image of a device to be detected through an augmented reality (AR) device; A first determining module, used to determine a device area image in the device image; A second determination module is used to determine the device type of the device to be detected according to the device area image and the device recognition model; The inspection module is used to inspect the device to be detected according to the device type and obtain the inspection result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images