Control method, apparatus, electronic device and computer-readable storage medium for drone-assisted maintenance

Abstract

To provide a control method, apparatus, electronic device, storage medium and program for drone-assisted maintenance.SOLUTION: The method includes: determining a target workpiece and a storage location thereof in response to an assistance message obtained by a first terminal located at a construction location, where the assistance message is for indicating the target workpiece lacking in a maintenance operation; planning a flight path for a transport task according to the construction location and the storage location; and controlling a drone to transport the target workpiece to the construction location according to the flight path. When a necessary workpiece is lacking for the maintenance operation, the workpiece can be transported quickly by the drone, thereby capable of assisting the maintenance operation and shortening the time required for the maintenance operation.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to the field of drone technology, and more particularly to the fields of drone control, path planning, and device maintenance technology.

Background Art

[0002] In conventional maintenance work, if a maintenance worker suddenly realizes that necessary tools or parts are lacking during the maintenance work, usually, the current construction process needs to be interrupted and they have to return to the warehouse or other storage areas to obtain the necessary items. As a result, the maintenance efficiency is significantly reduced. In particular, in a narrow space, a complex environment, or when access to the site is difficult, it has a great impact on the progress of maintenance and related production operations.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The present disclosure provides a control method, apparatus, electronic device, storage medium, and program for maintenance support by a drone to solve or alleviate one or more technical problems in the prior art.

Means for Solving the Problems

[0004] In a first aspect, the present disclosure provides a control method for maintenance support by a drone, the method comprising: determining a target workpiece and its storage location in response to a cooperation message acquired by a first terminal at a construction location, wherein the cooperation message is for indicating a target workpiece lacking for maintenance work; planning a flight path of a transportation task based on the construction location and the storage location; controlling the drone to transport the target workpiece to the construction location based on the flight path.

[0005] In a second aspect, the present disclosure provides an apparatus for maintenance support by a drone, the apparatus comprising: a workpiece inquiry module for determining a target workpiece and its storage location according to a cooperation message acquired by a first terminal at a construction location, wherein the cooperation message is for indicating a target workpiece lacking for maintenance work; a route planning module for planning a flight route of a transportation task based on the construction location and the storage location; a control module for controlling a drone to transport the target workpiece to the construction location based on the flight route.

[0006] In a third aspect, the present disclosure provides an electronic device, the device comprising: at least one processor; a memory communicatively connected to the at least one processor, wherein instructions executable by the at least one processor are stored in the memory, and when the instructions are executed by the at least one processor, any one of the methods in the embodiments of the present disclosure is executed.

[0007] In a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute any one of the methods in the embodiments of the present disclosure is provided.

[0008] In a fifth aspect, a program is provided, and when the program is executed by a processor, any one of the methods in the embodiments of the present disclosure is realized.

[0009] The beneficial effects of the solutions provided by the present disclosure at least include the following.

[0010] When there is a shortage of workpieces required for maintenance work, they can be quickly transported by a drone, thereby assisting the maintenance work and shortening the time required for the maintenance work.

[0011] The content described herein is not intended to describe the key points or important features of the embodiments of the present disclosure, nor should it be used to limit the scope of the present disclosure. For other features of the present disclosure, understanding is promoted through the following specification.

Brief Description of the Drawings

[0012] In the accompanying drawings, unless otherwise specified, the same reference numerals indicate the same or similar components or elements throughout the plurality of accompanying drawings. These accompanying drawings are not necessarily drawn to scale. It should be understood that these drawings show only some of the embodiments provided by the present disclosure and should not be regarded as limiting the scope of the present disclosure.

Figure 1

Figure 2

Figure 3

Modes for Carrying Out the Invention

[0013] Hereinafter, the present disclosure will be described in more detail with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar elements. Also, in the accompanying drawings, although various aspects of the embodiments are shown, these accompanying drawings are not necessarily drawn to scale unless otherwise explained.

[0014] Furthermore, to better illustrate the present disclosure, many specific details are described in the following specific embodiments. Those skilled in the art should understand that the present disclosure can be implemented similarly even without some details. In some examples, well-known methods, means, components, circuits, etc. to those skilled in the art are not described in detail so that the gist of the present disclosure can be made clear.

[0015] FIG. 1 is a flowchart of a control method for maintenance support by a drone provided according to an embodiment of the present disclosure. As shown in FIG. 1, the method includes at least the following steps.

[0016] In S101, according to a cooperation message acquired by a first terminal at a construction position, determining a target workpiece and its storage position, wherein the cooperation message is for indicating a target workpiece lacking for maintenance work.

[0017] The control method for maintenance support by a drone according to an embodiment of the present disclosure can be applied to a data processing device, whereby an execution system for executing this method can be obtained. For example, when the device is deployed and executed on a first server or other processing device, the control method for assisted maintenance by a drone can be realized.

[0018] The first server may be an independent server, or a server cluster or distributed system, or a cloud server providing underlying cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network piece services, cloud communications, middleware services, and big data and artificial intelligence platforms.

[0019] In an embodiment of the present disclosure, the first terminal is an electronic device carried by a maintenance worker to a maintenance work site, and the first terminal can communicate with a first server via a wireless network. Optionally, the first terminal may be a terminal device such as a smartphone, a tablet, a notebook computer, a smartwatch, a personal digital assistant (PDA), etc., or may be a device such as smart glasses, a following drone, a video collection device, etc. for assisting and recording the maintenance process, but is not limited thereto. In the first terminal, an application capable of supporting maintenance by a drone is installed and running.

[0020] The maintenance worker can generate a cooperation message via the first terminal and send it to the execution system. The cooperation message indicates the items lacking for the currently ongoing maintenance work and indicates that cooperation is required for obtaining the items. The cooperation message can include an audio message dictated by the maintenance worker, and the system can determine the lacking workpiece information through speech recognition and semantic analysis. Also, the cooperation message may be the lacking workpiece information determined by the system through semantic analysis based on a text message manually input by the maintenance worker. Furthermore, the cooperation message may be the lacking workpiece information directly selected by the maintenance worker through an operation such as clicking and selecting on the dialogue interface of a preset program running on the first terminal.

[0021] Note that in some complex construction environments, for example, inside devices such as intersecting pipes, reaction kettles, and stirrers, the space is very limited and the operation of the maintenance worker is inconvenient. In this case, the cooperation message can be generated more conveniently in the form of audio.

[0022] After determining the missing workpieces, the system can query the storage location of the workpieces through the inventory information it stores. It can also send the information of the target workpieces to the second server that runs the inventory management system and query the storage location of the target workpieces through the second server.

[0023] In S102, based on the construction location and the storage location, plan the flight path of the transportation task.

[0024] When the first terminal sends a cooperation message to the system, it can simultaneously send its own location, that is, the construction location, to the system as part of the cooperation message. The system plans the flight path using a pre-set map based on the construction location and the storage location.

[0025] For route planning, it is necessary to consider elements such as the terrain, obstacles, and flight restrictions between the storage location of the target workpieces and the construction location, and it can be realized by using the elevation information, building distribution, etc. of the map data.

[0026] In S103, based on the flight path, control the drone to transport the target workpieces to the construction location.

[0027] The system generates the planned flight path and sends it as a navigation command to the drone that executes the transportation task, thereby controlling the drone to fly along the planned flight path. The flight path must be designed to efficiently reach the storage location of the target workpieces and ensure flight safety to avoid collisions and other potential dangers.

[0028] After the drone loads the target workpiece, the flight control system starts to execute the transportation task based on the command and can control the drone to fly along the planned route. During flight, the drone can sense the surrounding environment and its own state in real time. This sensing can be achieved by installing sensors such as cameras and lidar. If new obstacles or other risks are discovered during flight, the flight control system can adjust the flight route based on existing related obstacle avoidance technologies such as route changes and obstacle avoidance measures.

[0029] After the drone arrives at the construction site, at least one of the operations such as hovering, landing, throwing, and lowering the suspension bin needs to be executed based on conditions such as the loading method of the target workpiece, the environment at the construction site, and the dimensions of the target workpiece to ensure that the maintenance personnel can safely pick up the target workpiece. Then, according to specific needs, the drone may take off again in the air, return to the stop position of the drone according to the previously planned route, or execute other tasks.

[0030] According to the solution of the embodiment of the present disclosure, when the maintenance work cannot continue due to a shortage of required workpieces, it can be immediately transported by the drone, thereby assisting the maintenance work and shortening the required time for the maintenance work. Stopping many important facilities for maintenance may lead to the suspension of work in the entire production process and cause extremely large economic losses. Therefore, by completing the maintenance timely and quickly, the losses can be reduced as much as possible.

[0031] In a possible embodiment, determining the target workpiece and its storage location according to the cooperation message obtained by the first terminal at the construction site in S101 includes the following steps.

[0032] In S1011, according to the cooperation message acquired by the first terminal at the construction site, based on the semantic information included in the cooperation message, workpiece information of tools or parts lacking for maintenance work is determined.

[0033] In S1012, based on the workpiece information, a target workpiece and its storage location are determined from the inventory.

[0034] In an embodiment of the present disclosure, after receiving the cooperation message transmitted by the first terminal at the construction site, the system semantically analyzes the voice message or the manually input text message in the cooperation message to identify the workpiece information of the tools or parts lacking for maintenance work. For example, voice recognition technology is used to convert voice information into text, and natural language processing (NLP) technology is used to extract important information.

[0035] After acquiring the workpiece information, the system queries the local workpiece information database or communicates with the second server that runs the inventory management system to determine the target workpiece and the storage location of its inventory. Optionally, the workpiece information includes information such as the number, name, and specifications of the workpiece, and the target workpiece can be accurately found.

[0036] According to the solution of the embodiment of the present disclosure, through semantic analysis and information query for the whole process, the system can understand the needs of the maintenance staff and identify the exact location of the necessary tools or parts in the inventory. It helps to improve the efficiency of maintenance work and can confirm that the unmanned aerial vehicle can quickly acquire and transport the missing workpieces.

[0037] In one possible embodiment, planning the flight path of the transportation task based on the construction site and the storage location in S102 includes the following steps.

[0038] In S1021, based on the construction location, determine the environmental information for drone flight.

[0039] In an embodiment of the present disclosure, the environmental information can be understood as information indicating the complexity of the environment. Usually, in an open area, the environmental complexity is lower than that in an indoor area, and in an indoor area, the environmental complexity of a passage area is lower than that of a factory area with production facilities. The more complex the environment is, the more difficult it is for the drone to fly. The environmental information for drone flight can be understood as information on the difficulty level of drone flight. Based on the construction location, determine the geographical information of the factory area where the construction location is located, the distribution positions of each building and facility, etc. from a preset map. Analyze the spatial layout of the factory area where the construction location is located, including the height of the building, the position and height of the facility, etc. For each building and facility, environmental rules can be preset in advance, for example, indicating which areas are flyable and open and which areas are flight restricted. Furthermore, for the areas where various facilities are located, for example, the flight difficulty level for open areas and areas more than 3 meters above the facilities is level 1, the flight difficulty level for the area where type A facilities are located is level 2, the flight difficulty level for the area where type B facilities are located is level 3, the flight difficulty level for flying between complex pipeline networks is level 4, and the flight difficulty level inside the pipeline or inside the facility is level 5, etc., and the flight difficulty level can be set.

[0040] In S1022, based on the environmental information, determine the task type.

[0041] Based on the environmental information, the flight difficulty level of the current task can be judged, and the task type can be determined. Exemplarily, when the construction location is in an outdoor open area or a large indoor area, the drone can fly in a straight line, so the task type may be a simple type. On the other hand, when the construction location is in an area where facilities are dense and various pipeline networks intersect, the task type is a complex type and may be related to complex operations such as obstacle avoidance. The task type can be classified according to whether the flight difficulty level is greater than a preset threshold (for example, level 4).

[0042] In the case of a drone having a plurality of different loading methods, when determining the task type based on environmental information, the drone type or the loading method can also be limited. For example, when the transportation task is of a complex type and the drone does not have the condition of descending, a drone with a small volume and high adaptability can be preferentially selected, and the loading method can preferentially select methods such as throwing and hovering.

[0043] In S1023, based on the task type, the construction position, and the storage position, plan the flight path of the transportation task.

[0044] For different task types, it is necessary to select an appropriate flight strategy. Then, based on the construction position and the storage position, plan the flight path on a preset map. For example, in an area with low flight difficulty, a high-speed straight flight strategy can be adopted, and in an area with high flight difficulty, a low-speed or high-altitude flight strategy can be adopted.

[0045] According to the solution of the embodiment of the present disclosure, in different environments, dynamically plan the optimal flight path based on map information and environmental rules to ensure that the drone can safely and efficiently transport the target workpiece in various scenarios.

[0046] In one possible implementation form, planning the flight path of the transportation task based on the task type, the construction position, and the storage position in S1023 further includes when the task type is a non-complex type, using the open area and / or passage area in the preset map to plan a plurality of first paths connecting the construction position and the storage position; evaluating the required time and safety of the plurality of first paths; selecting one path from the plurality of first paths based on the evaluation result and the timeliness requirement to obtain the flight path of the transportation task.

[0047] In an embodiment of the present disclosure, first, using preset map information, a plurality of routes connecting the construction position and the storage position are determined, which may be open areas or passage areas. The open area includes the open area above various facilities inside the factory, and the passage area includes pedestrian passages and equipment passages inside the factory.

[0048] Next, the required time of the route is evaluated, and for each connected route, the system estimates the flight required time. In this process, factors such as flight distance, flight speed, and assumed risk areas can be considered. For each connected route, it is necessary to evaluate whether the flight area is restricted and the safety including obstacles such as mobile equipment and workers that may appear. By comprehensively considering factors such as required time and safety, a comprehensive score can be generated for each connected route to obtain the evaluation result.

[0049] The timeliness requirement refers to the maximum allowable flight required time of the transportation task. As the timeliness requirement, it may be indicated by the maintenance personnel in the cooperation message, for example, terms such as "as soon as possible" and "quickly" in the voice message. Based on the evaluation result and the timeliness requirement, route selection is performed, and the final flight route is selected as the route with a reasonable flight required time and a high safety score.

[0050] According to the solution of the embodiment of the present disclosure, when the task is not complex, the system comprehensively considers factors such as required time and safety based on map information and preset rules, selects the optimal flight route, and ensures that the unmanned aerial vehicle can complete the transportation task of the target workpiece safely within the limited required time.

[0051] In one possible implementation form, planning the flight route of the transportation task based on the task type, construction position, and storage position in S1023 further includes when the task type is a complex type, obtaining the historical route of the first terminal, and planning and generating a second route based on the construction position and the storage position, and Including adjusting and optimizing a second path by using at least a part of the history path to obtain a flight path for a transportation task.

[0052] In an embodiment of the present disclosure, for a complex type of task, that is, when the area has a complex environment, usually, the preset map information does not include specific data regarding these areas. Therefore, the planned flight path is only a straight-line path from the entrance of the area to the construction location, and it may be difficult to freely move back and forth through the gaps between facilities. The drone executing the transportation task may need to perform obstacle avoidance multiple times in this area in order to reach the construction location. If the environment is too complex, the drone executing the transportation task may take too much time or may have difficulty reaching the construction site. Thus, the system can obtain the history path when the maintenance personnel enter the construction location with the first terminal, and generate the flight path of the transportation task with the assistance of the history path. The history path can be obtained by the recording of the first terminal device or other position tracking technologies.

[0053] Based on the construction location and the storage location, the system planning and generating an initial second path may be planning a basic path based on information such as a preset map and open passages.

[0054] Using at least a part of the history path, the system can adjust and optimize the initial second path, specifically including trajectory matching, dynamic adjustment, and obstacle avoidance processing.

[0055] In trajectory matching, match the history path with the second path to find adjacent or similar parts.

[0056] In dynamic adjustment, dynamically adjust the second path according to the actual progress situation within the history path to adapt to the actual situation.

[0057] In obstacle avoidance processing, if there is an obstacle avoidance trajectory in the history path, use this information to adjust the second path to avoid obstacles.

[0058] According to the solution of the embodiment of the present disclosure, by using the historical path information and combining it with the planned second path, and through matching and adjustment, the final flight path is generated, so that it can better adapt to the complex construction environment and improve the accuracy and efficiency of the drone navigation.

[0059] In one possible embodiment, the first terminal includes a following drone, and the historical path of the first terminal includes the historical flight path of the following drone.

[0060] In the embodiment of the present disclosure, the historical flight path formed during the process of the following drone following the maintenance personnel and entering the complex construction environment to reach the construction position can be used as a reference for the flight path of the drone transporting the target workpiece. By planning the flight path of the transportation task with reference to this historical flight path, the drone executing the transportation task is guided to learn the running trajectory of the following drone in this space, so that the transporting drone can better avoid entering branch paths or colliding in this complex space, and ultimately ensure the rapid delivery of the target workpiece.

[0061] In addition, the following drone can follow the maintenance personnel using existing related following technologies, which is helpful for operations such as process recording, remote calls, and equipment history information queries by the maintenance personnel. When following, the following drone usually only needs to reach the walking speed of a human, and the flight speed is relatively low. Therefore, even if it encounters an obstacle during flight, there is sufficient time to avoid the obstacle. The historical flight path obtained thereby helps to reduce the obstacle avoidance operation of the drone executing the transportation task, and ultimately can arrive at the construction position at a faster speed and improve the efficiency of the maintenance work.

[0062] The following drone needs to record its own flight trajectory during the process of executing the maintenance work to form a historical flight path. The flight trajectory of the following drone can be sent to the system in real time, or it can also send a cooperation message and send it to the system.

[0063] In one possible embodiment, using at least a portion of the history path to adjust and optimize the second path to obtain the flight path of the transportation task specifically includes: determining a complex area based on the construction location; determining a first trajectory point for entering the complex area in the second path; determining a second trajectory point in the history path that is closest to the first trajectory point; fusing the path of the portion of the second path that is located after the first trajectory point and the path of the portion of the history path that is located after the second trajectory point to obtain a fused path; connecting the path of the portion of the second path that is located before the first trajectory point to the fused path to obtain the flight path of the transportation task.

[0064] In the embodiments of the present disclosure, determining a complex area based on the construction location includes determining a complex area that needs to fly in the air, including narrow passages, high obstacles, etc., based on the construction location and environmental information.

[0065] After that, in the second path, determine a first trajectory point for entering the complex area, that is, an entry point for flying over the complex area. Next, find the second trajectory point in the history path that is closest to the first trajectory point. That is, it is possible to find a trajectory point in the history path that can be docked with the second path.

[0066] Fuse the path of the portion of the second path that is located after the first trajectory point and the path of the portion of the history path that is located after the closest second trajectory point to form a fused path. The purpose of this step is to utilize the experience information in the history path to ensure the rationality and safety of the fused path.

[0067] Connect the path of the portion located in front of the first trajectory point in the second path and the fusion path to obtain the final flight path. In this step, while maintaining the continuity of the original path, it is ensured that the guidance path is used to efficiently navigate complex areas.

[0068] In flight path planning, to fuse the historical path and the second path, any of the following fusion methods can be adopted.

[0069] First, weighted fusion. That is, weights can be assigned to the two paths for fusion. The weights can be assigned based on factors such as the reliability and accuracy of the historical path. For example, if the historical path is the historical flight path of a following drone, it has a higher reference value and can be assigned a higher weight. If it is the historical path of a device such as a mobile phone or PDA, a lower weight can be assigned.

[0070] Second, path interpolation. That is, interpolate the key points of the historical path and the second path to generate a new path. This method has some similar parts between the historical path and the new path, and can be smoothly fused at these similar parts. The specific procedure is as follows.

[0071] 1. Analyze the historical path. First, analyze the historical path, identify the curved and turning parts in it, and this process can be completed by detecting the changes in the curvature and direction of the path. The parts where the curvature is greater than the threshold generally correspond to curves, and the parts where the direction changes significantly may represent turns.

[0072] 2. Set key points. Set key points at the positions of curves and turns. The key points can be set at the highest curvature points of the path, the turning points of the path, or other scenario-specific positions. The key points are used for subsequent interpolation.

[0073] 3. Select an interpolation algorithm. To generate a smooth curve between the set key points, select an appropriate interpolation algorithm. Common interpolation methods include linear interpolation, spline interpolation, Bezier curves, etc. Select the interpolation algorithm according to the characteristics of the path and the requirements of smoothness.

[0074] 4. Perform interpolation processing. Using the selected interpolation algorithm, generate an interpolation path for each pair of key points to create a new path. The path is created to be as compatible as possible with the curves and turns of the historical path while maintaining smoothness between the key points.

[0075] 5. Integrate the new path. Integrate the new path generated by interpolation with other parts of the original path to form a complete planned path.

[0076] Thirdly, there is path guidance. That is, regarding the historical path as a guidance path, it can be adjusted based on the guidance path when generating a new path. For example, using a path planning algorithm such as the A* algorithm, a search can be performed based on the guidance path to obtain a new path.

[0077] In one possible embodiment, the method further includes estimating the completion status of the transportation task by a plurality of loaded drones based on the current states of the plurality of loaded drones, and determining a loaded drone that can complete the transportation task based on the completion status and requires the shortest required time.

[0078] In an embodiment of the present disclosure, a payload drone is a drone with a carrying capacity and can carry and transport articles or loads. Such drones are usually equipped with a special load platform that can be connected to various loads in order to achieve various types of tasks. For example, the drone may be equipped with fixtures such as suction cups and grippers, and can carry articles by suction or gripping. In addition, the drone may be provided with a special cargo compartment. For example, a cargo compartment can be installed at the bottom of the drone, or in the case of a dual-battery model, one of the battery compartments can be used for installing the cargo compartment for transporting the cargo. Further, a hook may be designed at the bottom or other positions of the drone, and various devices and articles such as a cabin that can be lifted and lowered using a cable can be suspended.

[0079] Obtain the current states of a plurality of payload drones, including information such as power amount, load status, and current position. Based on the estimation results of each drone, select the drone that can complete the task and requires the shortest time. Through the selection, select the payload drone that can complete the task fastest and ensure the success of the task. Thereafter, it is necessary to assign a transportation task to the selected drone and notify it to execute the task.

[0080] According to the solution of the embodiment of the present disclosure, it is ensured that the optimal one for completing the task is selected from a plurality of payload drones so as to maximize the overall transportation efficiency in consideration of factors such as required time and power amount.

[0081] In one possible embodiment, planning the flight path of the transportation task based on the construction position and the storage position in S102 includes generating a workpiece extraction task based on the current position of the payload drone and the storage position, and when the workpiece extraction task is completed, planning the flight path of the transportation task based on the storage position and the construction position.

[0082] In an embodiment of the present disclosure, before controlling the drone to execute the transportation task, since the drone and the storage location of the target workpiece are usually not in the same place, it is necessary to generate a workpiece extraction task based on the current position of the drone-mounted and the storage location of the target workpiece.

[0083] After the drone reaches the target position, a specific workpiece extraction operation is executed. This may include using tools such as robotic arms and chucks to extract parts, and the specific workpiece extraction method depends on the nature of the target workpiece.

[0084] When the workpiece extraction task is successful, the drone updates its status and notifies the system that the task has been completed.

[0085] FIG. 2 is a schematic diagram showing the configuration of a maintenance support device by a drone provided according to an embodiment of the present disclosure. As shown in FIG. 2, this device includes a workpiece inquiry module 201 for determining a target workpiece and its storage location according to a cooperation message acquired by a first terminal at a construction location, the cooperation message being for indicating a target workpiece lacking for maintenance work, the workpiece inquiry module 201, a route planning module 202 for planning a flight route of the transportation task based on the construction location and the storage location, a control module 203 for controlling the drone to transport the target workpiece to the construction location based on the flight route.

[0086] In one possible embodiment, the workpiece inquiry module 201 determines workpiece information of tools or parts lacking for maintenance work according to the cooperation message acquired by a first terminal at a construction location and based on the semantic information included in the cooperation message, It is used to determine the target workpiece and its storage location from inventory based on workpiece information.

[0087] In one possible embodiment, the route planning module 202 includes an information determination sub-module for determining environmental information for drone flight based on the construction location, a type determination sub-module for determining the task type based on the environmental information, and a planning sub-module for planning the flight route of the transportation task based on the task type, the construction location, and the storage location.

[0088] In one possible embodiment, the planning sub-module when the task type is a non-complex type, plans a plurality of first routes connecting the construction location and the storage location using the open area and / or the passage area in the preset map, evaluates the required time and safety of the plurality of first routes, and selects one route from the plurality of first routes based on the evaluation result and the timeliness requirement to obtain the flight route of the transportation task.

[0089] In one possible embodiment, the planning sub-module when the task type is a complex type, obtains the historical route of the first terminal, plans and generates a second route based on the construction location and the storage location, and uses at least a part of the historical route to adjust and optimize the second route to obtain the flight route of the transportation task.

[0090] In one possible embodiment, the first terminal includes a following drone, and the historical route of the first terminal includes the historical flight route of the following drone.

[0091] In one possible embodiment, the planning sub-module determines a complex area based on the construction location, Determining a first trajectory point for entering a complex area in the second path; Determining a second trajectory point closest to the first trajectory point in the historical path; Fusing a part of the path in the second path that is located after the first trajectory point and a part of the path in the historical path that is located after the second trajectory point to obtain a fused path; It is further used for connecting the path of the part located before the first trajectory point in the second path to the fused path to obtain the flight path of the transportation task.

[0092] In one possible embodiment, the apparatus Estimates the completion status of the transportation task by a plurality of loaded drones based on the current states of the plurality of loaded drones; And further includes a selection module for determining a loaded drone that can complete the transportation task and requires the shortest required time based on the completion status.

[0093] In one possible embodiment, the path planning module 202 Generates a workpiece picking task based on the current position and storage position of the loaded drone; When the workpiece picking task is completed, it is further used for planning the flight path of the transportation task based on the storage position and the construction position.

[0094] For the specific functions and exemplary descriptions of each module and sub-module of the apparatus according to the embodiments of the present disclosure, reference can be made to the relevant descriptions of the corresponding steps in the embodiments of the above-described method, and details will not be repeated here.

[0095] FIG. 3 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 3, the electronic device includes a memory 310 and a processor 320, and a computer program executable by the processor 320 is stored in the memory 310. The number of the memory 310 and the processor 320 may be one or more. The memory 310 can store one or more computer programs, and when the one or more computer programs are executed by the electronic device, the electronic device is caused to execute the method provided by the embodiment of the above method. The electronic device may further include the following. A communication interface 330 is used for communicating with an external device and performing data interaction and transmission.

[0096] When the memory 310, the processor 320, and the communication interface 330 are independently implemented, the memory 310, the processor 320, and the communication interface 330 are connected to each other via a bus and can communicate with each other. The bus may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be classified into an address bus, a data bus, a control bus, etc. For ease of explanation, only a single thick line is shown in FIG. 3, but it does not represent only a single bus or a single type of bus.

[0097] Optionally, in a specific implementation form, when the memory 310, the processor 320, and the communication interface 330 are integrated on one chip, the memory 310, the processor 320, and the communication interface 330 can communicate with each other via an internal interface.

[0098] The above-mentioned processor may be a Central Processing Unit (CPU), and it should be understood that it may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware assemblies, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. Note that the processor may be a processor that supports the Advanced RISC Machines (ARM) architecture.

[0099] Furthermore, optionally, the memory may include a read-only memory and a random access memory, and may further include a non-volatile random access memory. The memory can be either a volatile memory or a non-volatile memory, or can include both a volatile memory and a non-volatile memory. Here, the non-volatile memory can include ROM (Read-Only Memory), PROM (Programmable ROM), EPROM (Erasable PROM), EEPROM (Electrically EPROM), or flash memory. The volatile memory can include a random access memory (Random Access Memory, RAM) that functions as an external cache. By way of example and not limitation, many forms of RAM are available. For example, static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic Random Access Memory, DRAM), synchronous DRAM (Synchronous DRAM, SDRAM), double data rate SDRAM (Double Data Rate SDRAM, DDR SDRAM), enhanced SDRAM (Enhanced SDRAM, ESDRAM), synchlink DRAM (Synchlink DRAM, SLDRAM), and direct RAMBUS RAM (Direct RAMBUS RAM, DR RAM).

[0100] In the above embodiments, they may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the whole or part may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present disclosure are generated in whole or in part. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, Bluetooth®, microwave, etc.). The computer-readable storage medium may be any available medium accessible by a computer, or a data storage device including a server, a data center, etc. integrated with one or more available media. The available media may be a magnetic medium (such as a floppy (registered trademark) disk, hard disk, magnetic tape), an optical medium (such as a Digital Versatile Disc (DVD)), or a semiconductor medium (such as a Solid State Disk (SSD)), etc. It should be noted that the computer-readable storage medium referred to in the present disclosure may be a non-volatile storage medium, in other words, a non-transitory storage medium.

[0101] Those skilled in the art can understand that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by instructing relevant hardware through a program. The program may be stored in a computer-readable storage medium, and the above storage medium may be a read-only memory, a magnetic disk, an optical disk, or the like.

[0102] In the description of the embodiments of the present disclosure, the descriptions of reference terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or features described in relation to the embodiment or example are included in at least one embodiment or example of the present disclosure. Moreover, the specific features, structures, materials, or features described can be combined in any one or more embodiments or examples in an appropriate manner. Furthermore, those skilled in the art may combine different embodiments or examples described in the present disclosure and the features of different embodiments or examples as long as they do not conflict with each other.

[0103] In the description of the embodiments of the present disclosure, " / " represents the meaning of "or" unless otherwise described. For example, A / B may represent either A or B. "And / or" in the present disclosure only explains the relationship of related objects and indicates that there may be three types of relationships. For example, A and / or B can represent the following. There are three situations where A exists alone, A and B exist simultaneously, and B exists alone.

[0104] In the description of the embodiments of the present disclosure, the terms "first" and "second" are used only for the purpose of description and should not be construed as indicating or implying relative importance, nor should they be construed as implying the number of technical features shown. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present disclosure, "a plurality" means two or more unless otherwise described.

[0105] The above are only exemplary embodiments of the present disclosure and do not limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and scope of the principles of the present disclosure should all be included within the protection scope of the present disclosure.

Claims

1. A method for controlling maintenance support by an unmanned aerial vehicle, comprising: Determining a target workpiece and its storage location according to a collaboration message obtained by a first terminal at a work location, the collaboration message being for indicating a target workpiece that is missing due to a maintenance operation, and the work location being located in a factory area; determining environmental information for flying the unmanned aerial vehicle based on the construction location; determining a task type based on the environmental information; When the task type is a complex type, acquiring a historical route of the first terminal, including a case where the environmental information indicates that the construction location is in an area where equipment is densely packed when the task type is a complex type, the first terminal includes a following drone, and the historical route of the first terminal includes a historical flight route of the following drone; Planning and generating a second route based on the construction location and the storage location; determining a complex area based on the construction location; and determining a first trajectory point on the second path that enters the complex area; determining a second trajectory point on the historical route that is closest to the first trajectory point; A route of a portion of the second route located after the first trajectory point and a route of a portion of the history route located after the second trajectory point are merged to obtain a merged route; A portion of the second path located before the first trajectory point is joined with the merged path to obtain a flight path of a transportation task; and controlling the drone to deliver the target workpiece to the construction location based on the flight path. A method for controlling drone-based maintenance support.

2. Determining a target workpiece and its storage location in response to the collaboration message acquired by the first terminal at the construction location includes: In response to a collaboration message obtained by a first terminal at a construction location, determining workpiece information of a tool or part that is missing for a maintenance operation based on semantic information included in the collaboration message; determining a target workpiece and its storage location from inventory based on the workpiece information. A method for controlling maintenance support using an unmanned aerial vehicle according to claim 1.

3. Planning a flight path for a transportation task based on the task type, the construction location, and the storage location includes: If the task type is a non-complex type, planning a plurality of first routes connecting the work location and the storage location by using an open area and / or a passage area in a preset map; Evaluating the travel time and safety of the first routes; selecting a path from the plurality of first paths based on the evaluation result and a timeliness requirement to obtain a flight path for the transport task; A method for controlling maintenance support using an unmanned aerial vehicle according to claim 1.

4. The control method includes: Estimating a completion status of a transportation task by the plurality of loaded drones based on a current state of the plurality of loaded drones; and determining a payload drone that can complete the transportation task and requires the shortest time based on the completion status. A method for controlling maintenance support using an unmanned aerial vehicle according to claim 1.

5. Planning a flight path for a transportation task based on the construction location and the storage location includes: generating a workpiece retrieval task based on a current location of the loading drone and the storage location; and when the workpiece removal task is completed, planning a flight path for a transport task based on the storage location and the construction location. A method for controlling maintenance support using an unmanned aerial vehicle according to claim 4.

6. A device for maintenance support using an unmanned aerial vehicle, a workpiece inquiry module for determining a target workpiece and its storage location according to a collaboration message obtained by a first terminal at a work location, the collaboration message being for indicating a target workpiece that is missing due to a maintenance operation, and the work location being located in a factory area; a path planning module, an information determination submodule for determining environmental information for drone flight based on the construction location; a type determination submodule for determining a task type based on the environment information; A planning sub-module, comprising: When the task type is a complex type, acquiring a historical route of the first terminal, including a case where the environmental information indicates that the construction location is in an area where equipment is densely packed when the task type is a complex type, the first terminal includes a following drone, and the historical route of the first terminal includes a historical flight route of the following drone; Planning and generating a second route based on the construction location and the storage location; determining a complex area based on the construction location; and determining a first trajectory point on the second path that enters the complex area; determining a second trajectory point on the historical route that is closest to the first trajectory point; A route of a portion of the second route located after the first trajectory point and a route of a portion of the history route located after the second trajectory point are merged to obtain a merged route; A portion of the second path located before the first trajectory point is joined with the merged path to obtain a flight path of a transportation task; a path planning module including a planning sub-module for and a control module for controlling the drone to deliver the target workpiece to the construction location based on the flight path. A drone-based maintenance support device.

7. The workpiece inquiry module includes: In response to a collaboration message obtained by a first terminal at a construction location, determining workpiece information of a tool or part that is missing for a maintenance operation based on semantic information included in the collaboration message; determining a target workpiece and its storage location from the inventory based on the workpiece information; The device for maintenance support using an unmanned aerial vehicle according to claim 6.

8. The Planning submodule is If the task type is a non-complex type, planning a plurality of first routes connecting the work location and the storage location by using an open area and / or a passage area in a preset map; Evaluating the travel time and safety of the first routes; and selecting a path from the plurality of first paths based on the evaluation result and a timeliness requirement to obtain a flight path of the transportation task. The device for maintenance support using an unmanned aerial vehicle according to claim 6.

9. The apparatus comprises: Estimating a completion status of a transportation task by the plurality of loaded drones based on a current state of the plurality of loaded drones; and determining a loading drone that can complete the transportation task and requires the shortest time based on the completion status. The device for maintenance support using an unmanned aerial vehicle according to claim 6.

10. The path planning module: generating a workpiece retrieval task based on a current location of the loading drone and the storage location; and when the workpiece removal task is completed, planning a flight path for a transport task based on the storage location and the construction location. The device for maintenance support using an unmanned aerial vehicle according to claim 9.

11. At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform a method according to any one of claims 1 to 5. Electronic devices.

12. A non-transitory computer readable storage medium for storing instructions that cause a computer to perform the method of any one of claims 1 to 5.

Images