Event-driven complex environment autonomous inspection method and application

Abstract

The invention discloses an event-driven complex environment autonomous inspection method and application, and relates to an unmanned inspection technology, and the method comprises the steps: collecting equipment state data in an inspection process in real time; abnormal equipment is positioned based on the equipment state data, and an abnormal type and an abnormal degree are identified; detecting whether secondary abnormity exists in a preset range by taking the abnormal equipment as a center, if the secondary abnormity exists, interrupting the basic inspection route, and generating a sub-task route taking an abnormal point as a starting point; and carrying out spiral expansion troubleshooting on all the influenced equipment along the sub-task route until no newly added abnormity exists, integrating the abnormity type, the abnormal equipment coordinate, the abnormity degree and the range information, and returning to the interruption point to continue to complete the remaining inspection tasks. The problems that an existing unmanned inspection mode is poor in environment perception and abnormity judgment capacity and depends on manual intervention and assistance, and consequently rapid and accurate information support cannot be provided for emergency intervention can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned inspection, and in particular to an event-driven autonomous inspection method for complex environment, an event-driven autonomous inspection device for complex environment, an electronic device and a computer readable storage medium. BACKGROUND

[0002] Unmanned inspection devices have been popularized in the fields of industry, energy and the like, and are mainly used to replace long-time and high-frequency manual inspection. The existing unmanned inspection devices mainly adopt the mode of setting a fixed route, detecting specific targets and returning monitoring information for inspection, and have poor ability of environment perception and abnormality determination, resulting in weak autonomy and the need for certain manual intervention and assistance.

[0003] In order to cope with abnormal conditions and unexpected conditions in the scene, the unmanned inspection equipment needs to have the ability of autonomous response and evaluation and judgment. In particular, before manual intervention, it is crucial to determine the abnormal type, abnormal range and influence degree of the abnormality in advance to provide fast and accurate information support for subsequent manual decision and emergency intervention, and therefore it is necessary to improve the autonomy and intelligent degree of the unmanned device in the inspection task. SUMMARY

[0004] In order to overcome the defects of the prior art, the embodiments of the present application provide an event-driven autonomous inspection method for complex environment and application, which can solve the problem that the existing unmanned inspection mode has poor ability of environment perception and abnormality determination, relies on manual intervention and assistance, and cannot provide fast and accurate information support for emergency intervention.

[0005] In one aspect, the embodiments of the present application provide an event-driven autonomous inspection method for complex environment, which comprises: setting a basic inspection route, a device information library along the route and a device abnormality detection target item, and collecting device state data in real time during the inspection process; positioning an abnormal device based on the device state data and identifying an abnormal type and an abnormal degree; detecting whether there is a secondary abnormality in a preset range centered on the abnormal device, and if the secondary abnormality exists, interrupting the basic inspection route, generating a sub-task route with the abnormal point as the starting point; expanding and checking all affected devices along the sub-task route in a spiral manner until there is no new abnormality, integrating the abnormal type, abnormal device coordinates, abnormal degree and range information, and returning to the interruption point to continue completing the remaining inspection task.

[0006] In one embodiment of the present application, the step of generating the sub-task route specifically comprises: taking the abnormal device on the basic inspection route as the starting point, expanding and checking the range according to the real-time detection result step by step; when a new abnormality is found in the travel direction, expanding the checking radius and reducing the device moving speed.

[0007] In an embodiment of the present application, the extended range of investigation adopts a spiral search strategy, including: taking the initial abnormal device as the center, expanding outward along the concentric path; if there is no new abnormality after covering each layer, the expansion is terminated, otherwise the expansion continues to the outer layer.

[0008] In an embodiment of the present application, the evaluation step of the abnormality degree includes: establishing a multi-dimensional scoring model based on the temperature gradient change rate of the device state data, the sound source intensity threshold and the visual feature damage degree; outputting the abnormality level and the recommended disposal priority according to the multi-dimensional scoring model.

[0009] In an embodiment of the present application, after completing the remaining inspection tasks, it further includes: automatically associating the historical abnormality database for trend comparison; marking whether the abnormality is first occurrence or periodic recurrence.

[0010] In an embodiment of the present application, after collecting the device state data in the inspection process in real time, the device state data is processed by a sensor fusion algorithm, specifically including: synchronously processing the visible light data and the thermal imaging data in the device state data by a convolutional neural network; filtering environmental noise by using voiceprint recognition technology to extract device feature spectrum.

[0011] In an embodiment of the present application, the return of the interrupt point to continue to complete the remaining inspection tasks adopts a path optimization algorithm, including: calculating the shortest obstacle avoidance path from the interrupt point to the current inspection position; based on the shortest obstacle avoidance path, automatically supplementing the missed preset device nodes of the interrupted tasks along the way.

[0012] On the other hand, the embodiment of the present application also proposes an event-driven autonomous inspection device in complex environment, including: a data acquisition module for setting a basic inspection route, a device information database along the route and a device abnormality detection target item, and collecting device state data in real time during inspection; an abnormality recognition module for locating abnormal devices and identifying abnormal types and abnormal degrees based on the device state data; a route planning module for detecting whether there is a secondary abnormality in a preset range with the abnormal device as the center, generating a sub-task route with the abnormal point as the starting point if the secondary abnormality exists; an inspection completion module for expanding the investigation along the sub-task route until there is no new abnormality, integrating the abnormal type, abnormal device coordinates, abnormal degree and range information, and returning to the interrupt point to continue to complete the remaining inspection tasks.

[0013] In still another aspect, an electronic device is provided, which includes a memory and one or more processors connected to the memory, the memory storing a computer program, and the processor is configured to execute the computer program to implement the event-driven autonomous inspection method for complex environment as described in any one of the above embodiments.

[0014] In still another aspect, a computer readable storage medium is provided, which stores computer executable instructions for implementing the event-driven autonomous inspection method for complex environment as described in any one of the above embodiments.

[0015] From the above, the above embodiments of the present application can have at least one or more of the following beneficial effects compared with the prior art: The event-driven autonomous inspection method for complex environment proposed by the present application, based on the traditional autonomous inspection scheme, designs a corresponding inspection task updating method for sudden abnormal events, re-establishes the inspection sub-tasks and sub-routes, to autonomously respond to environmental changes; after discovering and identifying the abnormality, the spread range of the abnormality is judged, and the spread range is measured on site around the abnormal area, expanding the inspection coverage of the fixed route, which can effectively demarcate the abnormal influence area, and form a more comprehensive and accurate inspection result. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the specification and illustrate the illustrative embodiments of the present application and the explanation thereof, and do not constitute an improper limitation to the present application. In the drawings: Figure 1 A flow chart of the event-driven autonomous inspection method for complex environment provided by the embodiments of the present application; Figure 2 A specific execution logic schematic diagram of the event-driven autonomous inspection method for complex environment provided by the embodiments of the present application; Figure 3 A dynamic planning inspection route schematic diagram provided by the embodiments of the present application; Figure 4 A structure schematic diagram of the event-driven autonomous inspection device provided by the embodiments of the present application; Figure 5 A structure schematic diagram of the electronic device provided by the embodiments of the present application; Figure 6 A structure schematic diagram of the computer readable storage medium provided by the embodiments of the present application. DETAILED DESCRIPTION

[0017] It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other in the case of no conflict. The present application will be described below with reference to the accompanying drawings and in combination with the embodiments.

[0018] In order for those skilled in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments, and should all belong to the protection scope of the present application.

[0019] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are applicable to distinguish similar objects, and do not have to be used to describe a particular order or sequence. It should be understood that the terms thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0020] It should also be noted that the division of the plurality of embodiments in the present application is only for the convenience of description, and should not constitute a special limitation. The features in various embodiments can be combined with each other in the case of no conflict, and can be mutually quoted.

[0021] As shown in Figure 1 The first embodiment of the present application proposes an event-driven autonomous inspection method in a complex environment, for example, including: step S1, setting a basic inspection route, a device information library along the route, and a device abnormality detection target item, and collecting device state data in real time during the inspection process; step S2, positioning abnormal devices based on the device state data and identifying abnormal types and abnormal degrees; step S3, detecting whether there is a secondary abnormality in a preset range centered on the abnormal device, if the secondary abnormality exists, interrupting the basic inspection route, and generating a sub-task route starting from the abnormal point; step S4, expanding and checking all affected devices along the sub-task route in a spiral manner until there is no new abnormality, integrating the abnormal type, abnormal device coordinates, abnormal degree and range information, and returning to the interruption point to continue completing the remaining inspection task.

[0022] Specifically, in combination with Figure 2As shown, in step S1, first, the initial conditions of the inspection are determined, including: determining the inspection task scenario and the pre-set basic inspection route; determining the type, number and distribution of devices along the route; and determining the abnormal detection target items for different devices. Then, the device state data in the multi-source information detection inspection process is performed, specifically including: detecting the environment and devices through the visible light camera, thermal imaging camera and sound sensor; obtaining device state, temperature, sound and other information based on visual recognition, temperature detection and sound source detection.

[0023] In step S2, the device generating the anomaly is located according to the detected device state data, the abnormal type is identified, and the development degree of the anomaly is evaluated for the detected abnormal type.

[0024] In step S3, the abnormal influence trend and the affected range are evaluated. Figure 3 As shown, the abnormal influence trend and the affected range are evaluated. Specifically, whether there is a secondary anomaly in a preset range (detection diameter R≤5m range) centered on the abnormal device is detected. If there is no other secondary anomaly in the preset range, the anomaly is limited to the device itself, and there is no abnormal affected range; if there is a secondary anomaly in the preset range, the anomaly has affected other devices, and there is an abnormal affected range expansion, and the surrounding other devices need to be investigated.

[0025] Further, the current area that may be affected is detected preventively, the fixed inspection route is interrupted, and the inspection route is re-planned. Specifically, for example, the abnormal device discovery place on the fixed route is taken as the starting point, and the investigation task sub-route is re-planned. The sub-route planning needs to be determined according to the real-time detection result, that is, after the affected device is found, its type is identified and abnormal detection is performed, so as to expand the investigation range step by step.

[0026] In step S4, if an anomaly is found in the direction of travel, the investigation range needs to be further expanded until there is no anomaly in the range. Then, the direction is turned, and the investigation is continued to achieve full coverage of the abnormal affected range centered on the initial abnormal device.

[0027] In one embodiment, in the abnormal affected area, the moving speed is reduced, and the environmental anomaly detection precision is improved. The high-energy anomaly (such as working medium leakage, power cable, high-temperature anomaly) is investigated, the distance between the inspection device and the abnormal area is increased, and the safety of the inspection device itself is ensured.

[0028] In one implementation, for example, the affected area is detected and assessed. If the affected area is unaffected, an abnormality assessment conclusion is formed and the information is returned. If the affected area is affected, the affected device and the abnormality type are identified, the degree of the abnormality is assessed, and the secondary affected area is further detected until all affected areas are covered. An overall abnormality assessment conclusion is formed, including key information such as type, coordinates, degree, and range. Finally, the fixed route is returned to the interruption point to continue the remaining fixed route inspection tasks.

[0029] In one embodiment, the return to the interruption point to continue completing the remaining inspection tasks adopts a path optimization algorithm, by calculating the shortest obstacle avoidance path from the interruption point to the current inspection position, and automatically supplementing the preset device nodes that were missed by the task interruption based on the shortest obstacle avoidance path.

[0030] In one embodiment, after completing the remaining inspection tasks, for example, the historical anomaly database is automatically associated to perform trend comparison, marking whether the anomaly occurs for the first time or recurs periodically, so as to more accurately evaluate the abnormal state of the current detection.

[0031] To sum up, the first embodiment of the present application proposes an event-driven autonomous inspection method for complex environments. On the basis of the traditional autonomous inspection scheme, a corresponding inspection task update method is designed for sudden abnormal events, and inspection sub-tasks and sub-routes are re-established to autonomously respond to environmental changes. After discovering and identifying the anomaly, the scope of the anomaly is judged, and the scope of the anomaly is measured on the spot around the abnormal area, expanding the inspection coverage of the fixed route, which can effectively delineate the abnormal impact area and form a more comprehensive and accurate inspection result.

[0032] In addition, if Figure 4 As shown, the second embodiment of the present application further proposes an event-driven complex environment autonomous inspection device 20, for example including: a data acquisition module 201, an anomaly recognition module 202, a route planning module 203 and an inspection completion module 204.

[0033] The data acquisition module 201 is configured to set a basic inspection route, a device information library along the route, and a device abnormality detection target item, and acquire device state data in real time during the inspection; the abnormality identification module 202 is configured to locate an abnormal device based on the device state data and identify an abnormality type and an abnormality degree; the route planning module 203 is configured to detect whether there is a secondary abnormality in a preset range with the abnormal device as the center, and if the secondary abnormality exists, the basic inspection route is interrupted, and a sub-task route with the abnormal point as the starting point is generated; and the inspection completion module 204 is configured to perform spiral expansion and investigation on all affected devices along the sub-task route until no new abnormality is found, integrate the abnormality type, abnormal device coordinates, abnormality degree, and range information, and return to the interruption point to continue to complete the remaining inspection task.

[0034] It should be noted that the event-driven autonomous inspection method of the complex environment implemented by the event-driven autonomous inspection device of the complex environment disclosed in the second embodiment of the present application is as described in the first embodiment, and therefore will not be described in detail here. Alternatively, each module in the second embodiment and the above-mentioned other operations or functions are respectively used to implement the method described in the first embodiment, and the beneficial effects of the event-driven autonomous inspection device provided in the present embodiment are the same as those of the event-driven autonomous inspection method provided in the first embodiment. In order to be brief, they will not be described here.

[0035] As shown in Figure 5 The third embodiment of the present application further provides an electronic device 30, for example, comprising at least one processing unit 31 and at least one storage unit 32, wherein the storage unit 32 stores a computer program, and when the computer program is executed by the processing unit 31, the processing unit 31 executes the method as described in the first embodiment, and the beneficial effects of the electronic device 30 provided in the present embodiment are the same as those of the event-driven autonomous inspection method provided in the first embodiment.

[0036] As shown in Figure 6 The fourth embodiment of the present application further provides a computer readable storage medium 40, which stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method are implemented, and the beneficial effects of the computer readable storage medium 40 provided in the present embodiment are the same as those of the event-driven autonomous inspection method provided in the first embodiment.

[0037] The computer readable storage medium can include any type of disk including floppy disks, optical disks, DVDs, CD-ROMs, micro-drives, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic or optical cards, nano-systems (including molecular memory ICs), or any type of media or device suitable for storing instructions and / or data.

[0038] It should be noted that, for the foregoing method embodiments, the purposes of brief description, the foregoing method embodiments are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, some steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0039] In the above embodiments, the description of each embodiment is focused on, and the part not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0040] In several embodiments provided by the present application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely schematic. The division of the units is only a logical function division. There can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some services interfaces, devices or units, and can be electrical or other forms.

[0041] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0042] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0043] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present application. The aforementioned memory includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0044] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be instructed by a program to be completed by relevant hardware, and the program can be stored in a computer readable memory, which can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0045] The above is only exemplary embodiments of the present disclosure, and cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Those skilled in the art will easily think of embodiments of the present disclosure after considering the specification and practicing the disclosure herein. The present application is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or conventional technical means in the art that are not described in the present disclosure. The specification and examples are only considered as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

[0046] The technical features of the above embodiments can be combined in any way. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not contradict, they should be considered as within the scope of the present disclosure.

[0047] Those skilled in the art readily understand that the above only describes preferred embodiments of the present application and does not limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An event-driven autonomous inspection method for complex environments, characterized in that: include: Set basic inspection routes, equipment information along the routes, and equipment anomaly detection targets, and collect equipment status data in real time during the inspection process; Locating abnormal devices and identifying the type and degree of abnormality based on the device status data; Detect whether there is a secondary anomaly within a preset range with the abnormal device as the center. If the secondary anomaly exists, interrupt the basic inspection route and generate a subtask route with the abnormal point as the starting point; Perform spiral expansion inspections on all affected devices along the subtask route until no new anomalies are found, integrate the anomaly type, abnormal device coordinates, anomaly degree and range information, and return to the interruption point to continue completing the remaining inspection tasks.

2. The event-driven complex environment autonomous inspection method according to claim 1 is characterized in that: The step of generating the subtask route specifically includes: Starting from the abnormal equipment on the basic inspection route, the inspection scope is gradually expanded according to the real-time detection results; When new anomalies are discovered in the direction of travel, expand the inspection radius and reduce the equipment's movement speed.

3. The event-driven complex environment autonomous inspection method according to claim 2 is characterized in that: The expanded investigation scope adopts a spiral search strategy, including: With the initial abnormal device as the center, expand outward along concentric circles; If there is no new abnormality after each layer is covered, the expansion will be terminated, otherwise it will continue to extend to the outer layer.

4. The event-driven complex environment autonomous inspection method according to claim 1 is characterized in that: The step of evaluating the abnormality level includes: Establishing a multi-dimensional scoring model based on the temperature gradient change rate, sound source intensity threshold, and visual feature damage degree of the equipment status data; Output the abnormality level and recommended handling priority according to the multi-dimensional scoring model.

5. The event-driven complex environment autonomous inspection method according to claim 1 is characterized in that: After completing the remaining inspection tasks, the following steps are also included: Automatically correlate historical anomaly database for trend comparison; Note whether the abnormality is occurring for the first time or is recurring periodically.

6. The event-driven complex environment autonomous inspection method according to claim 1 is characterized in that: After collecting the device status data in the inspection process in real time, the sensor fusion algorithm is used to process the device status data, specifically including: synchronously processing the visible light data and the thermal imaging data in the device status data through a convolutional neural network; Voiceprint recognition technology is used to filter environmental noise and extract the device's characteristic sound spectrum.

7. The event-driven complex environment autonomous inspection method according to claim 1 is characterized in that: The path optimization algorithm is used to return to the interruption point and continue to complete the remaining inspection tasks, including: Calculate the shortest obstacle avoidance path from the interruption point to the current inspection position; Automatically check for missing preset equipment nodes along the shortest obstacle avoidance path.

8. An event-driven autonomous inspection device for complex environments, characterized in that: include: The data collection module is used to set the basic inspection route, the equipment information database along the route, and the equipment anomaly detection target items, and collect equipment status data in real time during the inspection process; an abnormality identification module, configured to locate abnormal devices and identify the type and degree of abnormality based on the device status data; A route planning module is used to detect whether there is a secondary anomaly within a preset range centered on the abnormal device. If the secondary anomaly exists, the basic inspection route is interrupted and a subtask route is generated starting from the abnormal point. The inspection completion module is used to perform a spiral expansion inspection of all affected devices along the subtask route until no new anomalies are found, integrate the anomaly type, abnormal device coordinates, anomaly degree and range information, and return to the interruption point to continue completing the remaining inspection tasks.

9. An electronic device, characterized in that: include: A memory and one or more processors connected to the memory, the memory storing a computer program, the processor being configured to execute the computer program to implement the event-driven complex environment autonomous inspection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable commands, and the computer-executable commands are used to execute the event-driven complex environment autonomous inspection method according to any one of claims 1 to 7.

Images