Control method, apparatus, electronic device, and computer-readable storage medium for maintenance support using drones

Abstract

A control method, apparatus, electronic device, storage medium, and program for maintenance support using unmanned aerial vehicles are provided. The method includes: determining a target workpiece and its storage location according to a collaboration message obtained by a first terminal at a construction location, the collaboration message being for indicating a target workpiece that is missing for a maintenance operation; planning a flight path for a transport task based on the construction location and the storage location; and controlling a drone to transport the target workpiece to the construction location based on the flight path. According to the solution of the present disclosure, when a workpiece required for performing a maintenance operation is missing, it can be transported by the drone quickly, thereby assisting the maintenance operation and shortening the time required for the maintenance operation.

Description

[Technical field]

[0001] The present disclosure relates to the field of drone technology, and in particular to the fields of drone control, path planning, and device maintenance. [Background technology]

[0002] In traditional maintenance work, when maintenance personnel suddenly find that they are missing necessary tools or parts during maintenance work, they usually have to interrupt the current work process and return to the warehouse or other storage area to obtain the necessary items, which results in a significant decrease in maintenance efficiency, especially in small spaces, complex environments, and difficult-to-access sites, which will have a significant impact on maintenance progress and related production operations. Summary of the Invention [Problem to be solved by the invention]

[0003] The present disclosure provides a control method, apparatus, electronic device, storage medium, and program for drone-based maintenance assistance to solve or alleviate one or more technical problems in the prior art. [Means for solving the problem]

[0004] In a first aspect, the present disclosure provides a control method for maintenance assistance by an unmanned aerial vehicle, the method comprising: Determining a target workpiece and its storage location according to a collaboration message obtained by a first terminal at a construction location, the collaboration message being for indicating a target workpiece that is missing due to a maintenance operation; Planning a flight path for a transportation task based on the construction location and the storage location; and controlling the drone to deliver the target workpiece to the construction location based on the flight path.

[0005] In a second aspect, the present disclosure provides an apparatus for unmanned aerial vehicle-based maintenance assistance, the apparatus comprising: A workpiece inquiry module for determining a target workpiece and its storage location according to a collaboration message obtained by a first terminal at a construction location, the collaboration message being for indicating a target workpiece that is missing due to a maintenance operation; and a path planning module for planning a flight path of a transportation task based on the construction location and the storage location; and a control module for controlling the drone to deliver the target workpiece to the construction location based on the flight path.

[0006] In a third aspect, the present disclosure provides an electronic device, the device comprising: At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, cause the implementation of any one of the methods in the embodiments of the present disclosure.

[0007] In a fourth aspect, there is provided a non-transitory computer-readable storage medium having stored thereon computer instructions for causing a computer to perform any one of the methods in the embodiments of the present disclosure.

[0008] In a fifth aspect, a program is provided, which, when executed by a processor, implements any one of the methods in the embodiments of the present disclosure.

[0009] The beneficial effects of the solution provided by the present disclosure include at least the following:

[0010] If a workpiece required to perform a maintenance operation is missing, it can be quickly transported by the drone, thereby assisting the maintenance operation and shortening the time required for the maintenance operation.

[0011] It should be understood that the contents described herein are not intended to describe key or important features of the embodiments of the present disclosure, nor are they used to limit the scope of the present disclosure, other features of the present disclosure are facilitated by the following specification. [Brief description of the drawings]

[0012] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the several accompanying drawings indicate the same or similar components or elements. The accompanying drawings are not necessarily drawn to scale. It should be understood that the drawings depict only some examples provided by the present disclosure and should not be considered as limiting the scope of the present disclosure. [Figure 1] 1 is a flowchart of a method for controlling maintenance assistance by an unmanned aerial vehicle according to one embodiment of the present disclosure. [Diagram 2] 1 is a schematic diagram showing the configuration of an apparatus for maintenance support using an unmanned aerial vehicle according to one embodiment of the present disclosure. FIG. [Diagram 3] An embodiment of the present disclosure is a block diagram of an electronic device for implementing a control method for maintenance support using an unmanned aerial vehicle. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The present disclosure will now be described in more detail with reference to the accompanying drawings, in which like reference numbers represent the same or similar elements, and in which various aspects of the embodiments are illustrated, and which, unless otherwise noted, are not necessarily drawn to scale.

[0014] Furthermore, in order to better explain the present disclosure, many specific details are described in the following specific embodiments. Those skilled in the art should understand that the present disclosure can be similarly implemented without some details. In some embodiments, methods, means, components, circuits, etc. that are well known to those skilled in the art are not described in detail so that the gist of the present disclosure is clear.

[0015] 1 is a flowchart of a method for controlling maintenance support by an unmanned aerial vehicle according to an embodiment of the present disclosure. As shown in FIG. 1, the method includes at least the following steps:

[0016] In S101, a target workpiece and its storage location are determined according to a collaboration message obtained by a first terminal at a construction location, the collaboration message being for indicating a target workpiece that is missing due to a maintenance work.

[0017] The control method for drone-assisted maintenance according to the embodiment of the present disclosure can be applied to a data processing device, thereby obtaining an execution system for executing the method. For example, the device can realize the control method for drone-assisted maintenance when deployed and executed on a first server or another processing device.

[0018] The first server may be an independent server, or may be a server cluster or distributed system, or a cloud server that provides underlying cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network piece services, cloud communications, middleware services, and big data and artificial intelligence platforms.

[0019] In an embodiment of the present disclosure, the first terminal is an electronic device carried to a maintenance site by a maintenance technician, and the first terminal can communicate with the first server via a wireless network. Optionally, the first terminal may be a terminal device such as a smartphone, a tablet, a laptop, a smart watch, a personal digital assistant (PDA), or a device such as smart glasses, a following drone, or a video collection device for assisting and recording the maintenance process, but is not limited thereto. An application capable of supporting maintenance by a drone is installed and executed in the first terminal.

[0020] The maintenance personnel can generate and send a cooperation message to the execution system via the first terminal. The cooperation message indicates a missing item for the ongoing maintenance work and indicates that cooperation is required to obtain the item. The cooperation message can include a voice message dictated by the maintenance personnel, and the system can determine the missing workpiece information through voice recognition and semantic analysis. The cooperation message can also be the missing workpiece information determined by the system through semantic analysis based on a text message manually entered by the maintenance personnel. Furthermore, the cooperation message can be the missing workpiece information directly selected by the maintenance personnel on the dialogue interface of the preset program executed on the first terminal, such as by clicking and selecting.

[0021] In addition, in some complex construction environments, for example, between intersecting pipes, inside devices such as reactors and agitators, the space is very limited and it is inconvenient for maintenance personnel to operate, so that the cooperation message can be generated more conveniently in the form of voice.

[0022] After determining the missing workpiece, the system can inquire about the storage location of the workpiece through inventory information stored in the system, and can also send information about the target workpiece to a second server running an inventory management system and inquire about the storage location of the target workpiece through the second server.

[0023] In S102, a flight path for the transportation task is planned based on the construction location and the storage location.

[0024] When the first terminal transmits a cooperation message to the system, the first terminal can simultaneously transmit its own position, i.e., the construction position, to the system as part of the cooperation message. The system plans a flight route using a pre-set map based on the construction position and the storage position.

[0025] Route planning must take into account factors such as the terrain, obstacles, and flight restrictions between the storage location of the target workpiece and the construction location, and can be achieved by utilizing elevation information and building distribution from map data.

[0026] In S103, the drone is controlled to deliver the target workpiece to the construction location based on the flight path.

[0027] The system generates a planned flight path and sends it as a navigation command to the drone performing the transportation task, thereby controlling the drone to fly along the planned flight path, which must be designed to efficiently reach the storage location of the target workpiece and ensure flight safety to avoid collisions and other potential hazards.

[0028] After the drone loads the target workpiece, the flight control system can start to execute the transportation task based on the command and control the drone to fly along the planned path. During the flight, the drone can sense the surrounding environment and its own state in real time. This sensing can be realized by mounting sensors such as cameras, laser radars, etc. If new obstacles or other risks are discovered during the flight, the flight control system can adjust the flight path based on existing related obstacle avoidance technologies, such as changing the path or taking obstacle avoidance measures.

[0029] After the drone arrives at the work site, it needs to perform at least one of the following operations, such as hovering, landing, throwing, and lowering the hanging bottle, based on the conditions such as the loading method of the target workpiece, the environment of the work site, and the dimensions of the target workpiece, so as to ensure that the maintenance personnel can safely pick up the target workpiece. Then, according to specific needs, the drone may take off again in the air, return to the drone's stopping position according to the previously planned path, or perform other tasks.

[0030] According to the solution of the embodiment of the present disclosure, when the maintenance work cannot continue due to lack of required workpieces, they can be immediately transported by the drone, thereby assisting the maintenance work and shortening the time required for the maintenance work. Since stopping many important equipment for maintenance can lead to the work stop of the entire production process and cause extremely large economic losses, timely and prompt completion of maintenance can reduce the losses as much as possible.

[0031] In a possible embodiment, in S101, determining the target workpiece and its storage location according to the collaboration message obtained by the first terminal at the construction site includes the following steps:

[0032] In S1011, in response to a collaboration message obtained by a first terminal at a construction location, determine workpiece information of a tool or part that is missing for a maintenance work based on semantic information included in the collaboration message.

[0033] In S1012, a target workpiece and its storage location are determined from the inventory based on the workpiece information.

[0034] In an embodiment of the present disclosure, after receiving the collaboration message sent by the first terminal at the construction site, the system performs semantic analysis processing on the voice message or the manually entered text message in the collaboration message to identify the workpiece information, which is the missing tool or part for the maintenance work, for example, converting the voice information into text using speech recognition technology, and extracting the important information using natural language processing (NLP) technology.

[0035] After obtaining the workpiece information, the system determines the storage location of the target workpiece and its inventory by querying a local workpiece information database or by communicating with a second server running an inventory management system. Optionally, the workpiece information includes information such as the workpiece number, name, and specifications, so that the target workpiece can be accurately located.

[0036] According to the solution of the embodiment of the present disclosure, the whole process is through semantic analysis and information query, which allows the system to understand the needs of the maintenance personnel and identify the exact location of the required tools or parts in the inventory, which helps to streamline the maintenance work and ensures that the drone can quickly retrieve and transport the missing workpieces.

[0037] In one possible embodiment, planning a flight path of the transport task based on the construction location and the storage location in S102 includes the following steps.

[0038] In S1021, environmental information for drone flight is determined based on the construction location.

[0039] In the embodiment of the present disclosure, the environmental information can be understood as information indicating the complexity of the environment, and usually, the environmental complexity of an open area is lower than that of an indoor area, and in an indoor area, the environmental complexity of an aisle area is lower than that of a factory area having production equipment. The more complex the environment, the more difficult it is to fly a drone. The environmental information for flying a drone can be understood as the difficulty information of flying a drone. Based on the construction location, the geographic information of the factory area where the construction location is located, the distribution location of each building and equipment, etc. are determined from a pre-set map. The spatial layout of the factory area where the construction location is located is analyzed, including the height of the building, the location and height of the equipment, etc. For each building and equipment, for example, environmental rules indicating which areas are open and flyable, and which areas are flight restricted, can be set in advance. In addition, flight difficulty levels can be set for areas where various types of equipment are located, for example, the difficulty of flying in open areas and areas more than 3 meters above equipment is Level 1, the difficulty of flying in areas where Class A equipment is located is Level 2, the difficulty of flying in areas where Class B equipment is located is Level 3, the difficulty of flying between complex pipeline networks is Level 4, and the difficulty of flying inside pipelines and equipment is Level 5.

[0040] In S1022, a task type is determined based on the environment information.

[0041] Based on the environmental information, the flying difficulty of the current task can be determined and the task type can be determined. For example, if the construction site is in an outdoor open area or an indoor area with a large space, the task type may be a simple type since the drone can fly in a straight line, whereas if the construction site is in an area where facilities are dense and various pipeline networks intersect, the task type may be a complex type and may involve complex operations such as obstacle avoidance. The task type can be classified according to whether the level of flying difficulty is greater than a preset threshold (e.g., level 4).

[0042] For a drone with multiple different loading methods, the drone type or loading method can also be limited when determining the task type based on the environmental information. For example, if the transportation task is complex and does not have the descent condition of the drone, a drone with a small volume and high adaptability can be preferentially selected, and the loading method can be preferentially selected as a method such as throwing or hovering.

[0043] In S1023, a flight path for the transport task is planned based on the task type, the construction location, and the storage location.

[0044] For different task types, it is necessary to select an appropriate flight strategy, and plan a flight path in a pre-defined map based on the construction and storage locations. For example, a high-speed direct flight strategy can be adopted for areas with low flight difficulty, and a low-speed or high-altitude flight strategy can be adopted for areas with high flight difficulty.

[0045] According to the solution of the embodiments of the present disclosure, in different environments, optimal flight paths can be dynamically planned based on map information and environmental rules, ensuring that the drone can safely and efficiently transport target workpieces in various scenarios.

[0046] In one possible implementation, planning the flight path of the transport task based on the task type, the construction location, and the storage location in S1023 further includes: If the task type is a non-complex type, planning a plurality of first routes connecting the work position and the storage position by using an open area and / or a passage area in a preset map; Evaluating the duration and safety of a plurality of first routes; Selecting a path from the plurality of first paths based on the evaluation result and the timeliness requirement to obtain a flight path for the transport task.

[0047] In the embodiment of the present disclosure, first, a plurality of routes connecting the construction position and the storage position are determined using preset map information, which may be open areas or passage areas. The open areas include the open areas above various equipment inside the factory, and the passage areas include the pedestrian passages and equipment passages inside the factory.

[0048] Next, the time required for the route is evaluated, and the system estimates the flight time for each linked route. This process can take into account flight distance, flight speed, expected risk areas, etc. For each linked route, the safety must be evaluated, including whether the flight area is restricted and obstacles such as mobile equipment and workers that may appear. Taking into consideration the time required and safety, etc., a total score can be generated for each linked route, and an evaluation result can be obtained.

[0049] The timeliness requirement is the maximum allowable flight time of the transportation task. The timeliness requirement may be indicated by the maintenance personnel in the collaboration message, for example, the terms "as soon as possible" and "as soon as possible" in the voice message. Based on the evaluation result and the timeliness requirement, a route selection is made, and a route with a reasonable flight time and a high safety score is selected as the final flight route.

[0050] According to the solution of the embodiment of the present disclosure, when the task is non-complex, the system can select the optimal flight path based on map information and pre-set rules, comprehensively considering factors such as time required and safety, and ensure that the drone completes the transportation task of the target workpiece in a safe manner within the limited time required.

[0051] In one possible implementation, planning the flight path of the transport task based on the task type, the construction location, and the storage location in S1023 further includes: If the task type is a complex type, acquiring a history path of the first terminal; planning and generating a second route based on the construction location and the storage location; Adjusting and optimizing the second path using at least a portion of the historical path to obtain a flight path for the transport task.

[0052] In the embodiment of the present disclosure, when the task is of a complex type, i.e., the environment is a complex area, the pre-set map information usually does not include specific data about these areas, so the planned flight path is only a straight line path from the entrance of the area to the construction position, and it may be difficult to freely move between the gaps between the facilities. In this area, the drone performing the transportation task may need to perform obstacle avoidance multiple times to reach the construction position. If the environment is too complex, the drone performing the transportation task may take too long or may have difficulty reaching the construction site. Thus, the system can obtain the historical path when the maintenance personnel holds the first terminal and enters the construction position, and generate the flight path of the transportation task under the assistance of the historical path. The historical path can be obtained by the record of the first terminal device or other position tracking technology.

[0053] Based on the construction location and storage location, the system plans and generates an initial second route, which may be to plan a basic route based on information such as a pre-set map, an open passage, etc.

[0054] Using at least a portion of the historical path, the system can adjust and optimize the initial second path, specifically including trajectory matching, dynamic adjustments, and obstacle avoidance processing.

[0055] In trajectory matching, a historical path is matched with a second path to find adjacent or similar portions.

[0056] In dynamic adjustment, depending on the actual progress in the historical path, the second path is dynamically adjusted to adapt to the actual situation.

[0057] During obstacle avoidance processing, if an obstacle avoidance trajectory exists in the historical route, this information is used to adjust the second route to avoid the obstacle.

[0058] According to the solution of the embodiment of the present disclosure, historical route information is utilized, combined with the planned second route, and a final flight route is generated through matching and adjustment, which can better adapt to complex construction environments and improve the accuracy and efficiency of drone navigation.

[0059] In one possible embodiment, the first terminal includes a following drone and the historical path of the first terminal includes a historical flight path of the following drone.

[0060] In the embodiment of the present disclosure, the historical flight path formed during the process of the following drone following the maintenance personnel into the complex construction environment and reaching the construction position can be used as a reference for the flight path of the drone that transports the target workpiece. By referring to this historical flight path and planning the flight path of the transport task, the drone that performs the transport task can be guided to learn the driving trajectory of the following drone in this space, so as to better avoid the transport drone from entering a branching path or colliding in this complex space, and thus ensure the rapid delivery of the target workpiece.

[0061] In addition, the following drone can follow the maintenance personnel using existing related following technology, which helps the maintenance personnel to perform operations such as process records, remote calls, and equipment history information inquiries. When following, the following drone usually only needs to reach the walking speed of a human, and the flight speed is relatively low, so that even if it encounters an obstacle during flight, it has enough time to avoid the obstacle, and the historical flight path obtained thereby can help the drone performing the transportation task to reduce the obstacle avoidance operations, and thus arrive at the work position at a faster speed, thereby improving the efficiency of the maintenance work.

[0062] The following drone needs to record its flight trajectory during the maintenance process to form a historical flight path. The flight trajectory of the following drone can be sent to the system in real time, and can also be sent to the system together with sending a cooperation message.

[0063] In one possible embodiment, adjusting and optimizing the second path using at least a portion of the historical path to obtain a flight path of the transport task specifically includes: determining a complex area based on a construction location; Determining a first trajectory point on the second path that enters the complex area; determining a second trajectory point on the historical path that is closest to the first trajectory point; Merging a part of the second path located after the first trajectory point and a part of the history path located after the second trajectory point to obtain a merged path; The method includes piecing together a portion of the second path located before the first trajectory point with the merging path to obtain a flight path for the transportation task.

[0064] In an embodiment of the present disclosure, determining a complex area based on the construction location includes determining a complex area that needs to be flown over based on the construction location and the environmental information, the complex area including narrow passages, tall obstacles, etc.

[0065] Then, determine a first trajectory point on the second route that enters the complex area, i.e., an entry point for flying over the complex area. Then, find a second trajectory point on the historical route that is closest to the first trajectory point. That is, find a trajectory point on the historical route that can be docked with the second route.

[0066] The part of the second path located after the first trajectory point is merged with the part of the history path located after the closest second trajectory point to form a fused path. The purpose of this step is to ensure the rationality and safety of the fused path by utilizing the experience information in the history path.

[0067] The part of the second path that is located before the first trajectory point is stitched together with the fused path to obtain the final flight path. This step ensures that the guided path can navigate efficiently in complex areas while maintaining the continuity of the original path.

[0068] In flight route planning, any of the following fusion methods can be adopted to fuse the historical route and the secondary route.

[0069] First, weighted fusion. That is, two routes can be fused by assigning weights to them. Weights can be assigned based on factors such as the reliability and accuracy of the historical route. For example, if the historical route is the historical flight route of a following drone, it can be weighted higher since it has higher reference value. If it is the historical route of a mobile phone, PDA, or other device, it can be weighted lower.

[0070] The second method is path interpolation, that is, the key points of the historical path and the key points of the second path are interpolated to generate a new path. This method is based on the fact that the historical path and the new path have some similarities, and these similarities can be smoothly merged. The specific steps are as follows:

[0071] 1. Analyze the historical route. First, analyze the historical route and identify the curves and turns in it. This process can be completed by detecting the curvature and direction changes of the route. Parts with curvature larger than a threshold generally correspond to curves, and parts with large changes in direction may represent turns.

[0072] 2. Set key points. Set key points at the locations of curves and turns, key points can be set at the highest curvature points of the path, turning points of the path, or other scenario-specific locations. Key points are used for subsequent interpolation.

[0073] 3. Select an interpolation algorithm. Select an appropriate interpolation algorithm to generate a smooth curve between the set key points. Common interpolation methods include linear interpolation, spline interpolation, Bezier curve, etc. Select an interpolation algorithm according to the characteristics of the path and the requirements for smoothness.

[0074] 4. Interpolate: Using the selected interpolation algorithm, an interpolated path is generated for each pair of keypoints to create a new path that tries to match the curves and turns of the historical path as closely as possible while maintaining smoothness between the keypoints.

[0075] 5. Integrate the new path: The new path generated by the interpolation is integrated with other parts of the original path to form a complete planned path.

[0076] Third, route guidance, i.e. the historical route can be regarded as the guided route, and the new route can be adjusted based on the guided route when generating it. For example, a route planning algorithm such as the A* algorithm can be used to search based on the guided route to obtain the new route.

[0077] In one possible embodiment, the method comprises: Estimating a completion status of the transportation task by the plurality of loaded unmanned aerial vehicles based on a current state of the plurality of loaded unmanned aerial vehicles; The method further includes determining a payload drone that can complete the transport task and requires the shortest time based on the completion status.

[0078] In the embodiment of the present disclosure, a loaded drone is a drone with a loading capability, which can carry and transport goods or loads. Such drones are usually equipped with a special loading platform that can be connected with various loads to accomplish different types of tasks. For example, the drone may be equipped with a jigs such as suction cups, grippers, etc., and can carry goods by suction or gripping. The drone may also be provided with a special cargo compartment, for example, a cargo compartment can be installed at the bottom of the drone, or one battery storehouse can be used to install a cargo compartment to transport cargo in the case of a dual battery model. A hook can also be designed at the bottom or other position of the drone, and various equipment or goods can be hung, such as a cabin that can be raised and lowered by a cable.

[0079] Obtain the current status of multiple loaded drones, including information such as power, load status, and current location. Based on the estimation results of each drone, select the drone that can complete the task and takes the shortest time. Through the selection, select the loaded drone that can complete the task the fastest and ensure the task is successful. After that, it is necessary to assign the transportation task to the selected drone and notify it to perform the task.

[0080] According to the solution of the embodiment of the present disclosure, it is possible to ensure that the optimal one for completing a task is selected from among multiple loaded drones, taking into account the required time, amount of power, etc., so as to maximize the overall transportation efficiency.

[0081] In one possible embodiment, planning a flight path of the transportation task based on the construction location and the storage location in S102 includes: generating a workpiece retrieval task based on the current location of the loaded drone and the storage location; If the workpiece retrieval task is completed, planning a flight path for the transport task based on the storage location and the construction location.

[0082] In an embodiment of the present disclosure, before controlling the drone to perform the transport task, a workpiece pick-up task needs to be generated based on the current position of the on-board drone and the storage location of the target workpiece, since the drone and the storage location of the target workpiece are usually not in the same location.

[0083] After the drone reaches the target location, it performs a specific workpiece removal operation, which may include removing the part using a robotic arm and a tool such as a chuck, with the specific workpiece removal method depending on the nature of the target workpiece.

[0084] Once the workpiece removal task is successful, the drone will update its status and notify the system that the task is complete.

[0085] FIG. 2 is a schematic diagram showing the configuration of a maintenance support device using an unmanned vehicle provided according to an embodiment of the present disclosure. As shown in FIG. 2, the device includes: A workpiece inquiry module 201 for determining a target workpiece and its storage location according to a collaboration message obtained by a first terminal at a construction site, the collaboration message being for indicating a target workpiece that is missing due to a maintenance work; a path planning module 202 for planning a flight path of the transportation task based on the construction location and the storage location; and a control module 203 for controlling the drone to deliver the target workpiece to the construction location based on the flight path.

[0086] In one possible embodiment, the workpiece query module 201 is In response to the cooperation message obtained by the first terminal at the construction location, determining workpiece information of the tools or parts that are missing for the maintenance work based on the semantic information included in the cooperation message; and determining a target workpiece and its storage location from the inventory based on the workpiece information.

[0087] In one possible embodiment, the path planning module 202: an information determination submodule for determining environmental information for drone flight based on the construction location; a type determination submodule for determining a task type based on the environmental information; A planning sub-module for planning a flight path for the transport task based on a task type, a construction location, and a storage location.

[0088] In one possible embodiment, the planning submodule comprises: If the task type is a non-complex type, planning a plurality of first routes connecting the work position and the storage position by using an open area and / or a passage area in a preset map; Evaluating the duration and safety of a plurality of first routes; Selecting a path from the plurality of first paths based on the evaluation result and the timeliness requirement to obtain a flight path for the transport task.

[0089] In one possible embodiment, the planning submodule comprises: If the task type is a complex type, acquiring a history path of the first terminal; planning and generating a second route based on the construction location and the storage location; and adjusting and optimizing the second path using at least a portion of the historical path to obtain a flight path for the transport task.

[0090] In one possible embodiment, the first terminal includes a following drone and the historical path of the first terminal includes a historical flight path of the following drone.

[0091] In one possible embodiment, the planning submodule comprises: determining a complex area based on a construction location; Determining a first trajectory point on the second path that enters the complex area; determining a second trajectory point on the historical path that is closest to the first trajectory point; Merging a part of the second path located after the first trajectory point and a part of the history path located after the second trajectory point to obtain a merged path; The second path is further used to connect a portion of the second path located before the first trajectory point with the fusion path to obtain a flight path of the transportation task.

[0092] In one possible embodiment, the device comprises: Estimating a completion status of the transportation task by the plurality of loaded unmanned aerial vehicles based on a current state of the plurality of loaded unmanned aerial vehicles; and determining, based on the completion status, a payload drone that can complete the transport task and that requires the shortest time.

[0093] In one possible embodiment, the path planning module 202: generating a workpiece retrieval task based on the current location of the loaded drone and the storage location; When the workpiece retrieval task is completed, it is further used to plan the flight path of the transport task based on the storage location and the construction location.

[0094] The specific functions and exemplary descriptions of each module and sub-module of the apparatus according to the embodiments of the present disclosure may refer to the relevant descriptions of the corresponding steps in the above-mentioned method embodiments, and will not be repeated here.

[0095] FIG. 3 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 3, the electronic device includes a memory 310 and a processor 320, and the memory 310 stores a computer program executable by the processor 320. The number of the memory 310 and the processor 320 may be one or more. The memory 310 may store one or more computer programs, which, when executed by the electronic device, cause the electronic device to perform the method provided by the above method embodiment. The electronic device may further include: The communication interface 330 is used for communicating with an external device and performing data interaction and transmission.

[0096] When the memory 310, the processor 320, and the communication interface 330 are implemented independently, the memory 310, the processor 320, and the communication interface 330 are connected to each other via a bus and can communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be classified into an address bus, a data bus, a control bus, etc. For ease of explanation, only one thick line is shown in FIG. 3, but this does not indicate only one bus or only one type of bus.

[0097] Optionally, in a specific implementation, when the memory 310, the processor 320, and the communication interface 330 are integrated on one chip, the memory 310, the processor 320, and the communication interface 330 can communicate with each other via an internal interface.

[0098] It should be understood that the processor may be a Central Processing Unit (CPU) or may be other general purpose processors, Digital Signal Processing (DSP), Application Specific Integrated Circuits (ASIC), Field Programmable Gate Arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware assemblies, etc. The general purpose processor may be a microprocessor or any conventional processor, etc. Additionally, the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0099] Additionally, optionally, the memory may include read-only memory and random access memory, and may further include non-volatile random access memory. The memory may be either volatile or non-volatile memory, or may include both volatile and non-volatile memory. Here, non-volatile memory may include Read-Only Memory (ROM), Programmable ROM (PROM), Erasable PROM (EPROM), Electrically EPROM (EEPROM), or flash memory. Volatile memory may include Random Access Memory (RAM), which acts as an external cache. By way of example and not limitation, many forms of RAM are available. For example, static random access memory (Static RAM, SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (Enhanced SDRAM, ESDRAM), synchlink DRAM (Synchlink DRAM, SLDRAM), and direct RAMBUS RAM (DR RAM).

[0100] The above-mentioned embodiments may be implemented in whole or in part in software, hardware, firmware, or any combination thereof. When implemented in software, the whole or in part may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, a process or function according to the embodiments of the present disclosure is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center via wire (e.g., coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, Bluetooth, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device including a server, a data center, etc. integrated with one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a Digital Versatile Disc (DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)). Note that the computer-readable storage medium referred to in this disclosure may be a non-volatile storage medium, in other words, a non-transitory storage medium.

[0101] Those skilled in the art can understand that all or part of the steps for realizing the above embodiments may be implemented by hardware, or may be implemented by instructing relevant hardware by a program, and the program may be stored in a computer-readable storage medium, and the storage medium may be a read-only memory, a magnetic disk, an optical disk, etc.

[0102] In the description of the embodiments of the present disclosure, the reference words "one embodiment", "several embodiments", "examples", "specific examples", or "several examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. And, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. Furthermore, a person skilled in the art may combine different embodiments or examples described in the present disclosure and features of different embodiments or examples to the extent that they are not mutually inconsistent.

[0103] In the description of the embodiments of the present disclosure, unless otherwise stated, " / " means "or", for example, A / B can mean either A or B. "And / or" in the present disclosure is only for describing the related relationship of related objects, and indicates that three types of relationships may exist, for example, A and / or B can indicate the following three types of situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0104] In the description of the embodiments of the present disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying a relative importance, nor should they be construed as implying a number of technical features shown. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present disclosure, "plurality" means two or more, unless otherwise specified.

[0105] The above are merely illustrative examples of the present disclosure, and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure should be included within the scope of protection of the present disclosure.

Claims

1. A method for controlling maintenance support by an unmanned aerial vehicle, comprising: Determining a target workpiece and its storage location according to a collaboration message obtained by a first terminal at a work location, the collaboration message being for indicating a target workpiece that is missing due to a maintenance operation, and the work location being located in a factory area; determining environmental information for drone flight based on the construction location, the environmental information indicating a degree of complexity of the environment or a difficulty of drone flight; determining a task type based on the environmental information; When the task type is a complex type, acquiring a historical route when a maintenance person enters the work location with the first terminal, including a case where the environmental information indicates that the work location is in an area where equipment is densely packed when the task type is a complex type, the first terminal includes a following drone that follows the maintenance person, and the historical route of the first terminal includes a historical flight route of the following drone; Planning and generating a second route based on the construction location and the storage location; determining a complex area based on the construction location; and determining a first trajectory point on the second path that enters the complex area; determining a second trajectory point on the historical route that is closest to the first trajectory point; A route of a portion of the second route located after the first trajectory point and a route of a portion of the history route located after the second trajectory point are merged to obtain a merged route; A portion of the second path located before the first trajectory point is joined with the merged path to obtain a flight path of a transportation task; and controlling the drone to deliver the target workpiece to the construction location based on the flight path. A method for controlling drone-based maintenance support.

2. Determining a target workpiece and its storage location in response to the collaboration message acquired by the first terminal at the construction location includes: In response to a collaboration message obtained by a first terminal at a construction location, determining workpiece information of a tool or part that is missing for a maintenance operation based on semantic information included in the collaboration message; determining a target workpiece and its storage location from inventory based on the workpiece information. A method for controlling maintenance support using an unmanned aerial vehicle according to claim 1.

3. Obtaining a flight path for the transport task includes: If the task type is a non-complex type, planning a plurality of first routes connecting the work location and the storage location by using an open area and / or a passage area in a preset map; Evaluating the travel time and safety of the first routes; selecting a path from the plurality of first paths based on the evaluation result and a timeliness requirement to obtain a flight path for the transport task; A method for controlling maintenance support using an unmanned aerial vehicle according to claim 1.

4. The control method includes: Estimating a completion status of a transportation task by the plurality of loaded drones based on a current state of the plurality of loaded drones; and determining a payload drone that can complete the transportation task and requires the shortest time based on the completion status. A method for controlling maintenance support using an unmanned aerial vehicle according to claim 1.

5. Planning a flight path for a transportation task based on the construction location and the storage location includes: generating a workpiece retrieval task based on a current location of the loading drone and the storage location; and when the workpiece removal task is completed, planning a flight path for a transport task based on the storage location and the construction location. A method for controlling maintenance support using an unmanned aerial vehicle according to claim 4.

6. A device for maintenance support using an unmanned aerial vehicle, a workpiece inquiry module for determining a target workpiece and its storage location according to a collaboration message obtained by a first terminal at a work location, the collaboration message being for indicating a target workpiece that is missing due to a maintenance operation, and the work location being located in a factory area; a path planning module, an information determination submodule for determining environmental information for drone flight based on the construction location, the environmental information indicating the complexity of the environment or the difficulty of drone flight; a type determination submodule for determining a task type based on the environment information; A planning sub-module, comprising: When the task type is a complex type, acquiring a historical route when a maintenance person enters the work location with the first terminal, including a case where the environmental information indicates that the work location is in an area where equipment is densely packed when the task type is a complex type, the first terminal includes a following drone that follows the maintenance person, and the historical route of the first terminal includes a historical flight route of the following drone; Planning and generating a second route based on the construction location and the storage location; determining a complex area based on the construction location; and determining a first trajectory point on the second path that enters the complex area; determining a second trajectory point on the historical route that is closest to the first trajectory point; A route of a portion of the second route located after the first trajectory point and a route of a portion of the history route located after the second trajectory point are merged to obtain a merged route; A portion of the second path located before the first trajectory point is joined with the merged path to obtain a flight path of a transportation task; a path planning module including a planning sub-module for and a control module for controlling the drone to deliver the target workpiece to the construction location based on the flight path. A drone-based maintenance support device.

7. The workpiece inquiry module includes: In response to a collaboration message obtained by a first terminal at a construction location, determining workpiece information of a tool or part that is missing for a maintenance operation based on semantic information included in the collaboration message; determining a target workpiece and its storage location from the inventory based on the workpiece information; The device for maintenance support using an unmanned aerial vehicle according to claim 6.

8. The Planning submodule is If the task type is a non-complex type, planning a plurality of first routes connecting the work location and the storage location by using an open area and / or a passage area in a preset map; Evaluating the travel time and safety of the first routes; and selecting a path from the plurality of first paths based on the evaluation result and a timeliness requirement to obtain a flight path of the transportation task. The device for maintenance support using an unmanned aerial vehicle according to claim 6.

9. The apparatus comprises: Estimating a completion status of a transportation task by the plurality of loaded drones based on a current state of the plurality of loaded drones; and determining a loading drone that can complete the transportation task and requires the shortest time based on the completion status. The device for maintenance support using an unmanned aerial vehicle according to claim 6.

10. The path planning module: generating a workpiece retrieval task based on a current location of the loading drone and the storage location; and when the workpiece removal task is completed, planning a flight path for a transport task based on the storage location and the construction location. The device for maintenance support using an unmanned aerial vehicle according to claim 9.

11. At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform a method according to any one of claims 1 to 5. Electronic devices.

12. A non-transitory computer readable storage medium for storing instructions that cause a computer to perform the method of any one of claims 1 to 5.

Images