A self-propelled robot inspection method and system

Abstract

The present invention provides a patrol inspection method and system for a self-propelled robot; wherein, the method comprises: acquiring first image data of the patrol inspection site, determining scene attribute data of the patrol inspection site based on the first image data; optimizing a risk judgment model based on the scene attribute data; acquiring second image data of the patrol inspection site, identifying ground risk points of the patrol inspection site based on the second image data and the optimized risk judgment model; and adjusting the patrol inspection plan of the self-propelled robot based on the identified ground risk points. The present invention can identify that the self-propelled robot can analyze the ground risk points of the patrol inspection site, thereby improving patrol inspection safety; and can also optimize the risk judgment model based on the scene attributes of the patrol inspection site, thereby improving the efficiency of identifying ground risk points.

Description

Technical Field

[0001] The present invention relates to the technical field of electric power inspection, and in particular to an inspection method, system, electronic equipment and computer storage medium of a self-moving robot. Background Art

[0002] Self-moving robots are very common intelligent power inspection equipment platforms, mainly including crawler-type and wheeled types. The application of self-moving robots in the field of power inspection has greatly improved the efficiency of power inspection and effectively reduced the safety risks of manual inspection.

[0003] The existing technology of self-propelled robots mainly focuses on the improvement of image recognition technology, that is, the continuous improvement of image recognition algorithms to obtain accurate recognition of abnormal conditions of power equipment. However, when performing inspection tasks, self-propelled robots walking on the ground are easily affected by ground conditions. For example, ground conditions such as puddles and mud can easily cause the self-propelled robots to get stuck, resulting in failure of the inspection task and damage to the self-propelled robots. Summary of the invention

[0004] In order to at least solve the technical problems existing in the above-mentioned background technology, the present invention provides an inspection method, system, electronic device and computer storage medium of a self-moving robot.

[0005] A first aspect of the present invention provides an inspection method for a self-propelled robot, comprising the following steps:

[0006] Acquire first image data of an inspection scene, and determine scene attribute data of the inspection scene according to the first image data;

[0007] Optimizing the risk judgment model according to the scenario attribute data;

[0008] Acquire second image data of the inspection site, and identify ground risk points of the inspection site according to the second image data and the optimized risk judgment model;

[0009] The inspection plan of the self-propelled robot is adjusted according to the identified ground risk points.

[0010] Further, determining the scene attribute data of the inspection site according to the first image data includes:

[0011] Determine a plurality of target objects from the first image data based on the first element, and extract first features thereof;

[0012] removing image regions corresponding to the target objects in the first image data to obtain second image data;

[0013] The second image data is subjected to recognition processing according to the second factor, and the scene attribute data is determined according to the recognition processing result.

[0014] Further, determining the scene attribute data according to the recognition processing result includes:

[0015] Determine the number of the subsidiary objects according to the recognition processing result, and if the number is less than a first threshold, determine that the scene attribute data is a first scene; if the number is greater than or equal to the first threshold, determine that the scene attribute data is a second scene;

[0016] The first scene corresponds to a closed scene, and the second scene corresponds to a non-closed scene.

[0017] Further, if the number is smaller than a second threshold, and the number of target objects is smaller than a third threshold, it is determined that the scene attribute data is a second scene.

[0018] Further, removing the image area corresponding to the target object in the first image data to obtain the second image data includes:

[0019] Determine the bottom image area corresponding to each of the target objects;

[0020] removing an image area corresponding to the target object from the first image data to obtain third image data;

[0021] The third image data is cropped according to each of the bottom image regions to obtain second image data.

[0022] Further, the cropping of the third image data according to each of the bottom image regions to obtain the second image data includes: determining a position of the bottom image region corresponding to each of the target objects in the first image data;

[0023] A plurality of bottom image regions are determined according to the positions, and the third image data is cropped based on the bottom image regions to obtain the second image data.

[0024] Furthermore, the optimizing process of the risk judgment model according to the scene attribute data includes:

[0025] Determine a number of risk identification sub-models according to the scenario attribute data and the preset association relationship;

[0026] Activate the risk identification sub-models corresponding to the risk judgment model.

[0027] A second aspect of the present invention provides an inspection system for a self-propelled robot, comprising an acquisition module, a processing module, and a storage module; the processing module is connected to the acquisition module and the storage module;

[0028] The storage module is used to store executable computer program code;

[0029] The acquisition module is used to acquire image data of the inspection site and transmit it to the processing module;

[0030] The processing module is used to execute the method as described in any of the preceding items by calling the executable computer program code in the storage module.

[0031] The third aspect of the present invention provides an electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the method as described in any of the preceding items.

[0032] A fourth aspect of the present invention provides a computer storage medium having a computer program stored thereon, and the computer program, when executed by a processor, executes the method as described in any of the preceding items.

[0033] The solution of the present invention is as follows: the present invention first determines the scene attribute data of the inspection site where the self-mobile robot is currently located based on the first image data of the inspection site, and optimizes the risk judgment model used to analyze ground risk points based on this, thereby improving the pertinence and recognition efficiency of the risk judgment model; then, during the execution of the inspection task, the risk judgment model is used to analyze the second image data to identify various ground risk points, and the inspection plan can be adjusted in time based on this to ensure that the self-mobile robot is not disturbed or damaged by harsh ground conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 It is a flow chart of a patrol inspection method of a self-moving robot disclosed in an embodiment of the present invention;

[0036] Figure 2 It is a structural schematic diagram of an inspection system of a self-moving robot disclosed in an embodiment of the present invention;

[0037] Figure 3It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0040] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0041] It should be understood that although the terms first, second, third, etc. may be used to describe ... in the embodiments of the present application, these ... should not be limited to these terms. These terms are only used to distinguish .... For example, without departing from the scope of the embodiments of the present application, the first ... may also be referred to as the second ..., and similarly, the second ... may also be referred to as the first ....

[0042] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0043] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a product or system. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the product or system including the elements.

[0044] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0045] Embodiment 1

[0046] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a patrol inspection method of a self-propelled robot disclosed in an embodiment of the present invention. Figure 1 As shown, a patrol inspection method of a self-moving robot according to an embodiment of the present invention comprises the following steps:

[0047] Acquire first image data of an inspection scene, and determine scene attribute data of the inspection scene according to the first image data;

[0048] Optimizing the risk judgment model according to the scenario attribute data;

[0049] Acquire second image data of the inspection site, and identify ground risk points of the inspection site according to the second image data and the optimized risk judgment model;

[0050] The inspection plan of the self-propelled robot is adjusted according to the identified ground risk points.

[0051] In the embodiment of the present invention, as described in the background technology, the prior art focuses more on the improvement of image recognition technology, that is, continuously improving the image recognition algorithm to obtain accurate identification of abnormal conditions of power equipment, while the research on the ground risks of the inspection site where the self-moving robot is located is obviously insufficient, resulting in the self-moving robot being easily trapped by ground risks (such as puddles, mud, etc.). In response to this situation, the present invention first determines the scene attribute data of the inspection site where the self-moving robot is currently located based on the first image data of the inspection site, and optimizes the risk judgment model used to analyze the ground risk points based on this, thereby improving the pertinence and recognition efficiency of the risk judgment model; then, during the execution of the inspection task, the risk judgment model is used to analyze the second image data to identify various ground risk points, and the inspection plan can be adjusted in time based on this to ensure that the self-moving robot is not disturbed or damaged by harsh ground conditions.

[0052] The implementation subject of all embodiments of the present invention can be a field end or a server end, wherein the field end can be a processing device equipped on a self-moving robot or a remote controller controlled by it, and the processing device can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and can even be a terminal device with data processing functions such as a smart phone, a tablet computer, and a wearable device. The server can be an independent physical server, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), big data, and artificial intelligence platforms.

[0053] Further, determining the scene attribute data of the inspection site according to the first image data includes:

[0054] Determine a plurality of target objects from the first image data based on the first element, and extract first features thereof;

[0055] removing image regions corresponding to the target objects in the first image data to obtain second image data;

[0056] The second image data is subjected to recognition processing according to the second factor, and the scene attribute data is determined according to the recognition processing result.

[0057] Further, determining the scene attribute data according to the recognition processing result includes:

[0058] Determine the number of the subsidiary objects according to the recognition processing result, and if the number is less than a first threshold, determine that the scene attribute data is a first scene; if the number is greater than or equal to the first threshold, determine that the scene attribute data is a second scene;

[0059] The first scene corresponds to a closed scene, and the second scene corresponds to a non-closed scene.

[0060] In the embodiment of the present invention, the inspection scene generally includes two scene attributes: closed scene (such as a substation) and closed scene (such as an outdoor high-voltage transmission line tower), and different scene attributes have different characteristics. In view of this situation, the present invention first determines the target object, such as a transformer, a mutual inductor, an insulator, etc., from the first image data according to the first element. Then, the remaining image of the first image data, i.e., the second image data, is used to identify the attached objects according to the second element, and then the scene attribute data can be determined according to the number of attached objects in the second image data.

[0061] An example is given below: the subsidiary object may be vegetation. Under normal circumstances, the number of vegetation coexisting with power equipment in a closed scene should be significantly less than that in an unclosed scene. Therefore, the present invention can determine scene attribute data based on the number of subsidiary objects.

[0062] It should be noted that the first element can be a specified shape and color (such as gray and orange commonly used for power equipment), while the second element can include the degree of aggregation of a single object in addition to shape and color (mainly green). The first element is used to describe the external characteristics of normal power equipment, and the second element is used to describe the external characteristics of vegetation that exists in the same spatial range as the power equipment.

[0063] Further, if the number is smaller than a second threshold, and the number of target objects is smaller than a third threshold, it is determined that the scene attribute data is a second scene.

[0064] In an embodiment of the present invention, under normal maintenance conditions, the number of target objects in a closed scene will not be lower than a third threshold (for example, 2), and the number of subsidiary objects in a non-closed scene will not be lower than the second threshold. If the actual detection result is that the number of subsidiary objects is less than the second threshold and the number of target objects is less than the third threshold, it is very likely that it is a single power equipment that is outdoors but in a closed state. For example, a fence is set at the base of an outdoor high-voltage transmission tower, thereby forming a quasi-closed scene. However, in this scene, the self-moving robot will not actually enter the fence for inspection. Therefore, this case is still determined to be a non-closed scene, that is, the second scene.

[0065] The second threshold is smaller than the first threshold.

[0066] Further, removing the image area corresponding to the target object in the first image data to obtain the second image data includes:

[0067] Determine the bottom image area corresponding to each of the target objects;

[0068] removing an image area corresponding to the target object from the first image data to obtain third image data;

[0069] The third image data is cropped according to each of the bottom image regions to obtain second image data.

[0070] In the embodiment of the present invention, the first image data may actually include vegetation data located far away, and these vegetation data may interfere with the determination of the current scene attribute data. Therefore, after removing the image area corresponding to the target object in the first image data, the present invention further performs cropping processing on the first image data according to the bottom image area corresponding to each target object extracted above, so as to accurately extract the ground area image of the current scene.

[0071] Further, the cropping of the third image data according to each of the bottom image regions to obtain the second image data includes: determining a position of the bottom image region corresponding to each of the target objects in the first image data;

[0072] A plurality of bottom image regions are determined according to the positions, and the third image data is cropped based on the bottom image regions to obtain the second image data.

[0073] In the embodiment of the present invention, several bottom image areas located at the outermost sides of the first image data can be screened out according to the positions of the bottom image areas in the first image data, and then a cropping frame can be determined, and the second image data can be cropped accordingly.

[0074] If there is only one target object, there is also only one bottom image area corresponding to the target object. In this case, the bottom image area can be directly used as a reference for downward cropping, thereby roughly obtaining the ground area image of the current scene.

[0075] Furthermore, the optimizing process of the risk judgment model according to the scene attribute data includes:

[0076] Determine a number of risk identification sub-models according to the scenario attribute data and the preset association relationship;

[0077] Activate the risk identification sub-models corresponding to the risk judgment model.

[0078] In an embodiment of the present invention, the present invention sets a risk judgment model, which can be a collective model, including multiple risk identification sub-models, which are used to specifically identify different risk types, so that its identification adaptability is wider. At the same time, the maintenance requirements of different scenes are different. For example, for closed scenes such as substations, the ground risk points are mainly pits and protrusions (bricks, etc.), while for non-closed scenes such as outdoor high-voltage transmission line towers, the ground risk points are more diverse. In addition to potholes and protrusions, they can also be stagnant water, silt, shrubs, etc. Therefore, the present invention only activates the specified risk identification sub-model in the risk judgment model according to the scene attribute data, thereby significantly improving the recognition efficiency of the risk judgment model.

[0079] Embodiment 2

[0080] See also Figure 2 , Figure 2 Schematic diagram of the structure of a self-propelled robot inspection system disclosed in an embodiment of the present invention. Figure 2 As shown, an inspection system for a self-moving robot according to an embodiment of the present invention comprises an acquisition module (101), a processing module (102), and a storage module (103); the processing module (102) is connected to the acquisition module (101) and the storage module (103);

[0081] The storage module (103) is used to store executable computer program code;

[0082] The acquisition module (101) is used to acquire image data of the inspection site and transmit it to the processing module (102);

[0083] The processing module (102) is used to execute the method as described in any of the preceding items by calling the executable computer program code in the storage module (103).

[0084] The specific functions of the inspection system of a self-moving robot in this embodiment refer to the above-mentioned embodiment 1. Since the system in this embodiment adopts all the technical solutions of the above-mentioned embodiments, it has at least all the beneficial effects brought by the technical solutions of the above-mentioned embodiments, which will not be described one by one here.

[0085] Embodiment 3

[0086] See also Figure 3 , Figure 3 An electronic device disclosed in an embodiment of the present invention comprises: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the method described in the first embodiment.

[0087] Embodiment 4

[0088] The embodiment of the present invention further discloses a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the first embodiment is executed.

[0089] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, system or device or used in combination with it.

[0090] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, system, or device.

[0091] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0092] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0093] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present invention, all of which belong to the scope of protection of the present invention.

Claims

1. A patrol inspection method for a self-propelled robot, characterized in that: The steps include: Acquire first image data of an inspection scene, and determine scene attribute data of the inspection scene according to the first image data; Optimizing the risk judgment model according to the scenario attribute data; Acquire second image data of the inspection site, and identify ground risk points of the inspection site according to the second image data and the optimized risk judgment model; Adjusting the inspection plan of the self-propelled robot according to the identified ground risk points; The determining of the scene attribute data of the inspection site according to the first image data includes: Determine a plurality of target objects from the first image data based on the first element, and extract first features thereof; removing image regions corresponding to the target objects in the first image data to obtain second image data; Performing recognition processing on the second image data according to the second factor, and determining the scene attribute data according to the recognition processing result; The determining the scene attribute data according to the recognition processing result includes: The number of subsidiary objects is determined according to the recognition processing result. If the number is less than a first threshold, the scene attribute data is determined to be a first scene; if the number is greater than or equal to the first threshold, the scene attribute data is determined to be a second scene; wherein the first scene corresponds to a closed scene, and the second scene corresponds to a non-closed scene; wherein the subsidiary objects are vegetation.

2. The inspection method of a self-propelled robot according to claim 1, characterized in that: If the number is smaller than the second threshold, and the number of target objects is smaller than the third threshold, the scene attribute data is determined to be the second scene.

3. The inspection method of a self-propelled robot according to claim 2, characterized in that: The removing the image area corresponding to the target object from the first image data to obtain the second image data includes: Determine the bottom image area corresponding to each of the target objects; removing an image area corresponding to the target object from the first image data to obtain third image data; The third image data is cropped according to each of the bottom image regions to obtain second image data.

4. The inspection method of a self-propelled robot according to claim 3, characterized in that: The step of cropping the third image data according to each of the bottom image regions to obtain the second image data includes: determining a position of the bottom image region corresponding to each of the target objects in the first image data; A plurality of bottom image regions are determined according to the positions, and the third image data is cropped based on the bottom image regions to obtain the second image data.

5. A patrol inspection method for a self-propelled robot according to any one of claims 1 to 4, characterized in that: The optimizing process of the risk judgment model according to the scene attribute data includes: Determine a number of risk identification sub-models according to the scenario attribute data and the preset association relationship; Activate the risk identification sub-models corresponding to the risk judgment model.

6. An inspection plan formulation system based on artificial intelligence, comprising an acquisition module, a processing module, and a storage module; the processing module is connected to the acquisition module and the storage module; the storage module is used to store executable computer program codes; the acquisition module is used to acquire image data of the inspection site and transmit it to the processing module; it is characterized in that: The processing module is used to execute the method according to any one of claims 1 to 5 by calling the executable computer program code in the storage module.

7. An electronic device comprising: A memory storing executable program code; A processor coupled to the memory; characterized in that: the processor calls the executable program code stored in the memory to execute the method according to any one of claims 1-5.

8. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is executed.

Images