Online Path Planning Method, Device and Computer Equipment for UAV to Detect Scene Update Changes in Real Time

Abstract

The present application relates to an online path planning method, device and computer device for an unmanned aerial vehicle (UAV) to detect real-time scene update changes. The method includes: obtaining a first sampling point and a first probability included in an initial scene reconstruction model; obtaining a first importance degree of each first candidate view according to the first sampling point and the first probability, and obtaining a target view from the first candidate views; generating a prior flight path of the UAV according to each target view; during the flight of the UAV along the prior flight path, if a scene change area is detected, obtaining a second sampling point and a second probability, obtaining a second importance degree of each second candidate view according to the second sampling point and the second probability, and obtaining a target flight view from the second candidate views; and adjusting the prior flight path according to the target flight view to obtain a real-time flight path. By using this method, the length of the UAV flight path can be reduced, and thus the efficiency of updating the three-dimensional model of the scene can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of unmanned aerial vehicle (UAV) control, and particularly to a method, device, and computer device for online path planning of a UAV for real-time detection of scene update changes. Background Art

[0002] With the development of UAV control technology, a technology for realizing three-dimensional (3D) scene reconstruction of an area using a UAV has emerged. This technology can collect complete scene data with high precision and high coverage rate through the UAV to construct an initial 3D scene model of the scene area. Subsequently, a regular flight mission of the UAV can be set to collect data for the scene area where changes have occurred to regularly update the initial 3D scene model.

[0003] The UAV flight control mission set for the regular update of the initial 3D scene model depends on the path planning of the UAV. In traditional technologies, in the UAV flight mission for updating the 3D scene model, the UAV usually explores and reconstructs the entire scene area comprehensively during the flight process.

[0004] However, this UAV flight control method will cause the UAV to fly repeatedly over the entire scene area, resulting in an overly long UAV flight path length, and thus reducing the efficiency of updating the 3D scene model. Summary of the Invention

[0005] Based on this, in view of the above technical problems, it is necessary to provide a method, device, computer device, computer-readable storage medium, and computer program product for online path planning of a UAV for real-time detection of scene update changes, which can reduce the UAV flight path length to improve the efficiency of updating the 3D scene model.

[0006] In a first aspect, the present application provides a method for online path planning of a UAV for real-time detection of scene update changes, including:

[0007] Obtaining an initial scene reconstruction model of a target area, obtaining first sampling points included in the initial scene reconstruction model, and the first probability of each first sampling point being located in a scene change area of the target area;

[0008] Obtaining a first candidate view set associated with the target area, obtaining the first importance degree of each first candidate view included in the first candidate view set according to the first sampling points and the first probability, and obtaining a target view from the first candidate views according to the first importance degree; the view set composed of the target views is the view set with the smallest number of views and the largest coverage range for the target area;

[0009] Generate a prior flight path for the drone according to each of the target perspectives; the prior flight path is used to control the estimated flight of the drone in the target area.

[0010] During the process of the drone flying in the target area according to the prior flight path, if a scene change area is detected, obtain second sampling points and the second probabilities of each of the second sampling points located in the scene change area.

[0011] Obtain a second candidate perspective set associated with the target area, obtain the second importance degrees of each second candidate perspective included in the second candidate perspective set according to the second sampling points and the second probabilities, and obtain a target flight perspective from the second candidate perspectives according to the second importance degrees.

[0012] Adjust the prior flight path according to the target flight perspective to obtain a real-time flight path; the real-time flight path is used to control the real-time flight of the drone in the target area to update the initial scene reconstruction model.

[0013] In one embodiment, the obtaining the first importance degrees of each first candidate perspective included in the first candidate perspective set according to the first sampling points and the first probabilities includes: obtaining a current first candidate perspective and the target first sampling points observable by the current first candidate perspective; the current first candidate perspective is any one of the first candidate perspectives; obtaining the first sample change scores corresponding to the target first sampling points according to the first probabilities corresponding to the target first sampling points; and obtaining the first importance degree of the current first candidate perspective based on the first sample change scores.

[0014] In one embodiment, the obtaining the first sample change scores corresponding to the target first sampling points according to the first probabilities corresponding to the target first sampling points includes: obtaining a current first sampling point and the number of first candidate perspectives observable by the current first sampling point; the current first sampling point is any one of the first sampling points; and obtaining the first sample change score corresponding to the current first sampling point according to the number and the first probability corresponding to the current first sampling point.

[0015] In one embodiment, obtaining the first sampling points included in the initial scene reconstruction model and the first probability that each of the first sampling points is located in a scene change area within the target area includes: obtaining the first sampling points included in the initial scene reconstruction model and the scene area type labels corresponding to each of the first sampling points; obtaining the prior probabilities respectively corresponding to each of the scene area type labels; the prior probability is used to represent the possibility of a scene change occurring in the area corresponding to each of the scene area type labels; taking the prior probability corresponding to each of the first sampling points as the first probability corresponding to each of the first sampling points.

[0016] In one embodiment, obtaining the second importance degree of each second candidate view included in the second candidate view set according to the second sampling points and the second probability includes: obtaining the current second candidate view; the current second candidate view is any one of each of the second candidate views; obtaining the second sample change scores corresponding to the current second candidate view for each of the second sampling points according to the second sampling points and the second probability; obtaining the second importance degree of the current second candidate view according to the second sample change scores corresponding to the current second candidate view for each of the second sampling points.

[0017] In one embodiment, obtaining the second sample change scores corresponding to the current second candidate view for each of the second sampling points according to the second sampling points and the second probability includes: obtaining the current second sampling point and the views that have been visited during the flight of the drone in the target area; the current second sampling point is any one of each of the second sampling points; obtaining the visibility information of the current second sampling point according to the views that have been visited and the current second candidate view; the visibility information is used to represent whether the current second sampling point can be observed by the current second candidate view and the views that have been visited; obtaining the second sample change scores corresponding to the current second candidate view for the current second sampling point according to the second probability and the visibility information.

[0018] In one embodiment, obtaining the second sampling points and the second probability that each of the second sampling points is located in the scene change area includes: obtaining the sampling points included in the scene change area, and taking the newly added sampling points included in the scene change area and the first sampling points other than the newly added sampling points as the second sampling points; setting the second probability corresponding to the newly added sampling points to 1, and taking the first probability corresponding to the first sampling points other than the newly added sampling points as the second probability corresponding to the first sampling points other than the newly added sampling points.

[0019] In one of the embodiments, the obtaining of the first candidate view set associated with the target area includes: performing Poisson disk sampling on the initial scene reconstruction model to obtain the first candidate view set; the obtaining of the second candidate view set associated with the target area includes: excluding the visited views during the flight of the drone in the target area from the target views to obtain an unvisited view set; performing Poisson disk sampling on the scene change area to obtain a candidate view set for the change area; and taking the union between the unvisited view set and the candidate view set for the change area as the second candidate view set.

[0020] In a second aspect, the present application also provides a device for online path planning of a drone for real-time detection of scene update changes, including:

[0021] A first sampling and obtaining module, configured to obtain an initial scene reconstruction model of a target area, obtain first sampling points included in the initial scene reconstruction model, and obtain first probabilities of each of the first sampling points being located in a scene change area in the target area;

[0022] A target view screening module, configured to obtain a first candidate view set associated with the target area, obtain first importance degrees of each of the first candidate views included in the first candidate view set according to the first sampling points and the first probabilities, and obtain target views from the first candidate views according to the first importance degrees; the view set composed of the target views is the view set with the smallest number of views and the largest coverage range for the target area;

[0023] A prior path generation module, configured to generate a prior flight path of the drone according to each of the target views; the prior flight path is used to control the estimated flight of the drone in the target area;

[0024] A second sampling and obtaining module, configured to, during the flight of the drone in the target area according to the prior flight path, if a scene change area is detected, obtain second sampling points and second probabilities of each of the second sampling points being located in the scene change area;

[0025] A flight view screening module, configured to obtain a second candidate view set associated with the target area, obtain second importance degrees of each of the second candidate views included in the second candidate view set according to the second sampling points and the second probabilities, and obtain target flight views from the second candidate views according to the second importance degrees;

[0026] A real-time path generation module, configured to adjust the prior flight path according to the target flight perspective to obtain a real-time flight path; the real-time flight path is used to control the real-time flight of the drone in the target area to update the initial scene reconstruction model.

[0027] In a third aspect, the present application further provides a computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of the method according to any one of the embodiments in the first aspect are implemented.

[0028] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method according to any one of the embodiments in the first aspect are implemented.

[0029] In a fifth aspect, the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of the embodiments in the first aspect are implemented.

[0030] The above-mentioned online path planning method, device, computer equipment, storage medium and computer program product for real-time detection of scene update changes obtain an initial scene reconstruction model of a target area, and obtain first sampling points included in the initial scene reconstruction model, and the first probability of each first sampling point being in a scene change area in the target area; obtain a first candidate view set associated with the target area, obtain the first importance degree of each first candidate view included in the first candidate view set according to the first sampling points and the first probability, and obtain a target view from the first candidate views according to the first importance degree; the view set composed of the target views is the view set with the smallest number of views and the largest coverage range for the target area; generate a prior flight path of the drone according to each target view; the prior flight path is used to control the estimated flight of the drone in the target area; during the flight of the drone in the target area according to the prior flight path, if a scene change area is detected, obtain second sampling points and the second probability of each second sampling point being in the scene change area; obtain a second candidate view set associated with the target area, obtain the second importance degree of each second candidate view included in the second candidate view set according to the second sampling points and the second probability, and obtain a target flight view from the second candidate views according to the second importance degree; adjust the prior flight path according to the target flight view to obtain a real-time flight path; the real-time flight path is used to control the real-time flight of the drone in the target area to update the initial scene reconstruction model.Before the UAV takes off, this application can obtain the first probability that each first sampling point belongs to the scene change area according to the initial scene reconstruction model of the target area, and combine the first probability to obtain the first importance degree of each first candidate perspective in the first candidate perspective set associated with the target area. Then, the target perspective can be screened out according to the first importance degree, and the perspective set composed of the screened target perspectives can meet the conditions of the minimum number of perspectives and the largest coverage range for the target area. After that, the prior flight path can be generated according to the target perspective, so that the UAV can fly in the target area according to the prior flight path. And if a scene change area is detected during the flight, the second probability that each second sampling point belongs to the scene change area can be obtained in real time. Then, in combination with the first probability, the second importance degree of each second candidate perspective in the second candidate perspective set associated with the target area can be obtained, so as to screen out the target flight perspective according to the second importance degree, and then adjust the prior flight path according to the target flight perspective to obtain the real-time flight path, so as to control the UAV to fly in real time. Compared with the prior art where the UAV flight needs to comprehensively explore the entire scene area, before the UAV flight in this application, the importance degree of each first candidate perspective can be obtained according to the first probability that the first sampling point belongs to the scene change area, so as to generate the prior flight path. And during the flight, when a scene change area is detected, the importance degree of each second candidate perspective can be obtained according to the second probability that the second sampling point belongs to the scene change area, so as to obtain the target flight perspective to adjust the prior flight path to obtain the real-time flight path. In this way, the flight path can be generated according to the probability that the sampling point belongs to the scene change area, so that the UAV can be avoided from flying repeatedly over the entire scene area, but only detecting the areas with high probability of change specifically, thereby reducing the length of the UAV flight path and improving the efficiency of updating the three-dimensional model of the scene. Description of the Drawings

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0032] Figure 1 It is a schematic flowchart of the online path planning method for a UAV to detect scene update changes in real time in an embodiment;

[0033] Figure 2 It is a schematic flowchart of obtaining the first importance degree in an embodiment;

[0034] Figure 3Schematic flowchart of obtaining the second importance degree in an embodiment;

[0035] Figure 4 Schematic flowchart of obtaining the second sample change score in an embodiment;

[0036] Figure 5 Schematic diagram of the perspective generation area in an embodiment;

[0037] Figure 6 Basic flowchart of the online path planning method for an unmanned aerial vehicle in an embodiment;

[0038] Figure 7 Flowchart of change area detection and extraction in an embodiment;

[0039] Figure 8 Structural block diagram of the online path planning device for an unmanned aerial vehicle that detects scene update changes in real time in an embodiment;

[0040] Figure 9 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0041] In order to make the objectives, technical solutions, and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] In one embodiment, as Figure 1 shown, an online path planning method for an unmanned aerial vehicle that detects scene update changes in real time is provided. In this embodiment, the application of this method to a terminal is used as an example for illustration. In this embodiment, the method includes the following steps:

[0043] Step S101, obtain an initial scene reconstruction model of a target area, obtain first sampling points included in the initial scene reconstruction model, and the first probability of each first sampling point in a scene change area in the target area.

[0044] Among them, the target area refers to the area where three-dimensional scene reconstruction needs to be performed, and the initial scene reconstruction model refers to the scene reconstruction model currently generated for three-dimensional scene reconstruction of the target area. For example, it can be a scene reconstruction model generated from data collected in the previous UAV flight mission. The first sampling point refers to the sampling point obtained by sampling from the initial scene reconstruction model, and the first probability refers to the probability that the first sampling point is in the scene change area in the target area, which can be the prior probability of the first sampling point for the scene change area before the UAV flight.

[0045] Specifically, before the drone executes the current flight mission, the terminal can pre-obtain the initial scene reconstruction model of the target area, sample the initial scene reconstruction model to obtain the first sampling points, and at the same time calculate the probability that each first sampling point is located in the scene change area of the target area as the first probability corresponding to each first sampling point.

[0046] Step S102: Obtain the first candidate view set associated with the target area, obtain the first importance degree of each first candidate view included in the first candidate view set according to the first sampling points and the first probabilities, and obtain the target view from the first candidate views according to the first importance degree; the view set composed of the target views is the view set with the smallest number of views and the largest coverage range for the target area.

[0047] The first candidate view set refers to the view set composed of the first candidate views. The first candidate view refers to the candidate flight view generated before the drone executes the current flight mission, and the first importance degree refers to the importance degree of each first candidate view in the current drone flight mission. The greater the importance degree, the higher the priority of being selected as the target view, and the target view is the view screened from the first candidate views to reduce the redundancy of views. That is to say, the screening goal of the target view is to minimize the number of views as much as possible while ensuring the maximum coverage, that is, the view set composed of the target views has the smallest number of views and the largest coverage range for the target area.

[0048] Specifically, the terminal can also obtain the first candidate view set associated with the target area, and combine the first sampling points collected in step S101 and the first probabilities corresponding to each first sampling point to calculate the importance degree of each first candidate view included in the first candidate view set, that is, the first importance degree. Then, the terminal can further obtain the target view from the first candidate views according to the importance degree of the first candidate view, so that the view set composed of the screened target views can meet the requirements of the smallest number of views and the largest coverage range for the target area, thereby effectively reducing the redundancy of views.

[0049] Step S103: Generate the prior flight path of the drone according to each target view; the prior flight path is used to control the estimated flight of the drone in the target area.

[0050] The prior flight path refers to the flight path pre-generated before the drone executes the current flight mission, and this flight path can represent the estimated flight trajectory of the drone in the target area. Specifically, after obtaining multiple target views, the multiple target views can be connected to generate the prior flight path of the drone for the current flight mission.

[0051] Step S104, during the flight of the drone in the target area according to the prior flight path, if a scene change area is detected, obtain second sampling points and the second probabilities of each second sampling point located in the scene change area.

[0052] The second sampling points are the sampling points used to generate the real-time flight path when the drone executes the current flight mission and detects a scene change area, and the second probability refers to the probability that the second sampling point is located in the scene change area of the target area. Specifically, after generating the prior flight path, the drone can fly in the target area according to the prior flight path. If a scene change area is detected during the flight, the terminal can also correct the prior flight path, that is, re-obtain the second sampling points and the second probabilities of each second sampling point located in the scene change area.

[0053] Step S105, obtain the second candidate view set associated with the target area, obtain the second importance degrees of each second candidate view included in the second candidate view set according to the second sampling points and the second probabilities, and obtain the target flight view from the second candidate views according to the second importance degrees.

[0054] The second candidate view set refers to the view set composed of second candidate views. The second candidate view refers to the candidate flight view generated after the drone detects a scene change area during the execution of the current flight mission, and the second importance degree refers to the importance degree of each second candidate view in the current drone flight mission. The greater the importance degree, the higher the priority of being selected as the target flight view, and the target flight view is the view screened from the second candidate views, and this view can be used as the next view for the real-time flight of the drone.

[0055] Specifically, after the drone detects a scene change area during the flight, the terminal can also obtain the second candidate view set associated with the target area, and combine the second sampling points collected in step S104 and the second probabilities corresponding to each second sampling point to calculate the importance degrees of each second candidate view included in the second candidate view set, that is, the second importance degrees. Then, the terminal can further obtain the next view for the real-time flight of the drone from the second candidate views according to the importance degrees of the second candidate views, that is, obtain the target flight view.

[0056] Step S106, adjust the prior flight path according to the target flight view to obtain the real-time flight path; the real-time flight path is used to control the real-time flight of the drone in the target area to update the initial scene reconstruction model.

[0057] The real-time flight path refers to the flight path after the prior flight path is corrected when the UAV detects a scene change area during flight. This path can be used to control the real-time flight of the UAV, thereby updating the initial scene reconstruction model. Specifically, when the target flight perspective is obtained, the prior flight path of the UAV can be adjusted to obtain the real-time flight path, and the UAV can fly according to the corrected real-time flight path, thereby realizing the update of the initial scene reconstruction model.

[0058] For example, before the UAV takes off, the generated prior flight path can be: perspective A - perspective B - perspective C - perspective D. When the UAV is flying according to the prior flight path, it can fly from perspective A to perspective B. If a scene change area is detected at perspective B, the target flight perspective can be obtained as the next flight perspective, such as perspective E. At this time, the flight path will be corrected to the real-time flight path, that is, perspective A - perspective B - perspective E.

[0059] In the above online path planning method for an unmanned aerial vehicle (UAV) to detect real-time scene updates and changes, an initial scene reconstruction model of the target area is obtained, and first sampling points included in the initial scene reconstruction model, as well as the first probability of each first sampling point being located in the scene change area of the target area, are obtained; a first candidate view set associated with the target area is obtained, and according to the first sampling points and the first probability, the first importance degree of each first candidate view included in the first candidate view set is obtained, and the target view is obtained from the first candidate views according to the first importance degree; the view set composed of the target views is the view set with the minimum number of views and the largest coverage range for the target area; a prior flight path of the UAV is generated according to the target views; the prior flight path is used to control the estimated flight of the UAV in the target area; during the flight of the UAV in the target area according to the prior flight path, if a scene change area is detected, second sampling points, as well as the second probability of each second sampling point being located in the scene change area, are obtained; a second candidate view set associated with the target area is obtained, and according to the second sampling points and the second probability, the second importance degree of each second candidate view included in the second candidate view set is obtained, and the target flight view is obtained from the second candidate views according to the second importance degree; the prior flight path is adjusted according to the target flight view to obtain a real-time flight path; the real-time flight path is used to control the real-time flight of the UAV in the target area to update the initial scene reconstruction model.Before the UAV flies, this application can obtain the first probability that each first sampling point belongs to the scene change area according to the initial scene reconstruction model of the target area, and combine the first probability to obtain the first importance degree of each first candidate view in the first candidate view set associated with the target area. Then, the target view is screened according to the first importance degree, and the view set composed of the screened target views can meet the conditions of the minimum number of views and the largest coverage range for the target area. After that, the prior flight path can be generated according to the target view, so that the UAV can fly in the target area according to the prior flight path. And if a scene change area is detected during the flight, the second sampling point and the second probability that each second sampling point belongs to the scene change area can be obtained in real time. Then, the second importance degree of each second candidate view in the second candidate view set associated with the target area can be obtained by combining the first probability. Thus, the target flight view is screened according to the second importance degree, and then the prior flight path is adjusted according to the target flight view to obtain the real-time flight path, so as to control the real-time flight of the UAV. Compared with the prior art that the UAV flight needs to comprehensively explore the entire scene area, before the UAV flies in this application, the importance degree of each first candidate view can be obtained according to the first probability that the first sampling point belongs to the scene change area, so as to generate the prior flight path. And during the flight, when a scene change area is detected, the importance degree of each second candidate view can be obtained according to the second probability that the second sampling point belongs to the scene change area, so as to obtain the target flight view to adjust the prior flight path to obtain the real-time flight path. By this method, the flight path can be generated according to the probability that the sampling point belongs to the scene change area. Therefore, it is possible to avoid the UAV from flying repeatedly over the entire scene area and only conduct targeted detection on the areas with high probability of change, thereby reducing the length of the UAV flight path and improving the efficiency of updating the three-dimensional model of the scene.

[0060] In one embodiment, as Figure 2 shown, step S102 may further include:

[0061] Step S201, obtaining the current first candidate view and the target first sampling points observable by the current first candidate view; the current first candidate view is any one of the first candidate views.

[0062] Among them, the current first candidate perspective refers to any one of the first candidate perspectives included in the first candidate perspective set, and the target first sampling point refers to the first sampling point that can be observed from the current first candidate perspective. In this embodiment, the terminal can pre-obtain the visibility relationship between each first candidate perspective and the first sampling point, and this visibility relationship can represent whether a certain first candidate perspective can observe a certain first sampling point. Therefore, based on the above visibility relationship, the first sampling point that can be observed from the current first candidate perspective can be determined as the target first sampling point.

[0063] Step S202: Obtain the first sample change scores corresponding to each target first sampling point according to the first probabilities corresponding to each target first sampling point.

[0064] Step S203: Based on each first sample change score, obtain the first importance degree of the current first candidate perspective.

[0065] The first sample change score refers to the sample change scores corresponding to each target first sampling point, which can be used to represent the change possibility of the sample. In this embodiment, after the first probabilities corresponding to each target first sampling point, the first sample change score corresponding to each target first sampling point can also be calculated based on the above first probabilities, and then each first sample change score is used to obtain the first importance degree of the current first candidate perspective.

[0066] For example, candidate perspective and sampling point The visibility relationship between them can be represented by the following formula:

[0067]

[0068] The current first candidate perspective The first importance degree of Can be calculated by the following formula:

[0069]

[0070] Among them, Represents the first sample change scores corresponding to each first sampling point respectively, while Represents each first sampling point The set of sample points composed of. It can be seen that only when Is 1, that is, Can be When observed, the first sample change score is used to calculate the first importance degree, that is, only the first sample change scores corresponding to the target first sampling points that can be observed from the current first candidate perspective are used to calculate the importance degree of the current first candidate perspective.

[0071] In this embodiment, it is also possible to determine the target first sampling points observable from each first candidate perspective, and further obtain the corresponding first sample change scores based on the first probabilities corresponding to the target first sampling points, so as to obtain the first importance degrees of the respective first candidate perspectives. In this way, the accuracy of obtaining the first importance degrees can be improved.

[0072] Further, step S202 may further include: obtaining the current first sampling point and the number of first candidate perspectives that can observe the current first sampling point; the current first sampling point is any one of the first sampling points; and obtaining the first sample change score corresponding to the current first sampling point according to the number and the first probability corresponding to the current first sampling point.

[0073] The current first sampling point refers to any one of the first sampling points. In this embodiment, since the terminal can pre-obtain the visibility relationship between each first candidate perspective and the first sampling points, it is also possible to obtain the number of first candidate perspectives that can observe the current first sampling point based on this visibility relationship. For example, if the first sampling point 1 can be observed by the first candidate perspective A and the first candidate perspective B, then the number of first candidate perspectives that can observe the first sampling point 1 is 2. Then, this number and the first probability corresponding to the current first sampling point can be used to calculate the first sample change score corresponding to the current first sampling point.

[0074] For example, the current first sampling point corresponding first sample change score can be calculated by the following formula:

[0075]

[0076] where, represents the number of first candidate perspectives that observe the current first sampling point and can be calculated by the following formula:

[0077]

[0078] where, represents the scaling parameter, represents the first probability corresponding to the current first sampling point and represents the visibility relationship between the current first sampling point and each first candidate perspective

[0079] ​In this embodiment, the first sample change score corresponding to the current first sampling point can also be obtained according to the number of first candidate perspectives of the current first sampling point and the first probability corresponding to the current first sampling point. This method can ensure that the change possibility of the sample changes with the number of perspectives observing it. For example, the change possibility of the sample gradually decreases as it is observed by more perspectives, thereby further improving the accuracy of obtaining the first sample change score.

[0080] In addition, step S101 may further include: obtaining the first sampling points included in the initial scene reconstruction model and the scene area type labels corresponding to each first sampling point; obtaining the prior probabilities corresponding to each scene area type label respectively; the prior probability is used to represent the possibility of scene change occurring in the area corresponding to each scene area type label; taking the prior probability corresponding to each first sampling point as the first probability corresponding to each first sampling point.

[0081] The scene area type label can represent the type of the scene area where each first sampling point is located, such as a road or a building, etc. The prior probability refers to the probability of the possibility of scene change occurring in the area corresponding to the scene area type label. In this embodiment, the corresponding relationship between each scene type label and the prior probability can be pre-constructed, and then this corresponding relationship can be used to obtain the first probability corresponding to each first sampling point.

[0082] Specifically, the terminal can first sample the initial scene reconstruction model to obtain the first sampling points included in the initial scene reconstruction model, and then identify the scene area type labels corresponding to the first sampling points, and combine the prior probabilities corresponding to each scene area type label respectively to obtain the prior probabilities corresponding to each first sampling point, and take them as the first probabilities corresponding to each first sampling point.

[0083] For example, the prior probability 1 corresponding to the road label and the prior probability 2 corresponding to the building label can be pre-constructed. When the scene area type label corresponding to the first sampling point 1 is the road label, then the first probability corresponding to the first sampling point is the prior probability 1, and if the scene area type label corresponding to the first sampling point 1 is the building label, then the first probability corresponding to the first sampling point is the prior probability 2.

[0084] In this embodiment, the corresponding relationship between the scene area type label and the prior probability can also be pre-constructed, so that the first probability corresponding to the first sampling point can be obtained according to the scene area type label corresponding to the first sampling point. In this way, the accuracy of calculating the prior probability can be improved.

[0085] In one embodiment, as Figure 3 shown, step S105 may further include:

[0086] Step S301: Obtain the current second candidate perspective; The current second candidate perspective is any one of the second candidate perspectives.

[0087] Step S302: According to the second sampling points and the second probabilities, obtain the second sample change scores corresponding to the current second candidate perspective for each second sampling point.

[0088] The current second candidate perspective refers to any one of the second candidate perspectives included in the set of second candidate perspectives, and the second sample change score refers to the sample change score corresponding to the current second candidate perspective for each second sampling point, which can be used to characterize the change possibility of the sample.

[0089] Specifically, the terminal can also use any one of the second candidate perspectives as the current second candidate perspective, and combine each second sampling point and the corresponding second probability to calculate the second sample change score corresponding to the current second candidate perspective for each second sampling point.

[0090] Step S303: According to the second sample change scores corresponding to the current second candidate perspective for each second sampling point, obtain the second importance degree of the current second candidate perspective.

[0091] After obtaining the second sample change score corresponding to the current second candidate perspective for each second sampling point, the above second sample change scores can be further summed up to obtain the second importance degree of the current second candidate perspective.

[0092] The current second candidate perspective The second importance degree can be calculated through the following formula:

[0093]

[0094] where, represents the second sample change scores corresponding to the current second candidate perspective for each second sampling point, and represents the set of sample points composed of each second sampling point corresponding to the current second candidate perspective respectively. It can be seen that in this embodiment, the second importance degree of the current second candidate perspective can be obtained by summing up the second sample change scores corresponding to the current second candidate perspective for each second sampling point. represents the second sample change scores corresponding to the current second candidate perspective for each second sampling point, and represents the set of sample points composed of each second sampling point

[0095] In this embodiment, the second sample change scores corresponding to the current second candidate perspective for each second sampling point can also be calculated through each second sampling point and the second probability, so as to obtain the second importance degree of the current second candidate perspective by using each second sample change score. In this way, the accuracy of calculating the second importance degree can be improved.

[0096] Further, as Figure 4 shown, step S302 may further include:

[0097] Step S401, obtaining the current second sampling point and the visited viewpoints during the flight of the drone in the target area; the current second sampling point is any one of the second sampling points.

[0098] The current second sampling point refers to any one of the second sampling points, and the visited viewpoints refer to the viewpoints that the drone has visited during the flight in the target area. Taking the prior flight path of the drone as Viewpoint A - Viewpoint B - Viewpoint C - Viewpoint D as an example, during the flight of the drone along the prior flight path, if a scene change area is detected after flying from Viewpoint A to Viewpoint B, then the visited viewpoints at this time can include Viewpoint A and Viewpoint B. After that, if the obtained target flight viewpoint, that is, the next flight viewpoint, is Viewpoint E, then the flight path will be corrected to the real-time flight path, that is, Viewpoint A - Viewpoint B - Viewpoint E, and the visited viewpoints will include Viewpoint A, Viewpoint B, and Viewpoint E.

[0099] Specifically, the terminal can take any one of the second sampling points as the current second sampling point and can determine the visited viewpoints during the flight of the drone in the target area.

[0100] Step S402, obtaining the visibility information of the current second sampling point according to the visited viewpoints and the current second candidate viewpoint; the visibility information is used to characterize whether the current second sampling point can be observed by the current second candidate viewpoint and the visited viewpoints.

[0101] The visibility information of the current second sampling point refers to the information used to characterize whether the current second sampling point can be observed by the current second candidate viewpoint and the visited viewpoints. This information can be obtained according to the visibility relationship between the current second sampling point and the candidate viewpoints, that is, it can include the visibility relationship between the current second sampling point and the current second candidate viewpoint, and the visibility relationship between the current second sampling point and each visited viewpoint.

[0102] Specifically, after determining each visited viewpoint and the current second candidate viewpoint, the visibility relationship between the above viewpoints and the current second sampling point can be obtained, so as to obtain the visibility information of the current second sampling point.

[0103] Step S403, obtaining the second sample change score of the current second sampling point corresponding to the current second candidate viewpoint according to the second probability and the visibility information.

[0104] Finally, the terminal can calculate the second sample change score corresponding to the current second sampling point for the current second candidate view by combining the second probability corresponding to the current second sampling point and the above visibility information.

[0105] For example, the current second sampling point corresponding to the current second candidate view of the second sample change score can be calculated by the following formula:

[0106]

[0107] where represents the sample change score corresponding to the current second sampling point corresponding to the unvisited current second candidate view while represents the sample change score corresponding to the current second sampling point corresponding to each visited view That is, it is composed of these two parts.

[0108] And can be calculated by the following formula:

[0109]

[0110] where represents the scaling parameter, represents the second probability corresponding to the current second sampling point while represents the visibility relationship between the current second sampling point and the current second candidate view That is, it is composed of these two parts.

[0111] While can be calculated by the following formula:

[0112]

[0113] where represents the scaling parameter, while represents the visibility relationship between the current second sampling point and any visited view That is, it is composed of these two parts.

[0114] In this embodiment, the visibility information of the current second sampling point can also be obtained by acquiring the visibility relationships between the current second sampling point and each visited perspective, as well as between the current second sampling point and the current second candidate perspective. Then, by combining the visibility information of the current second sampling point with the second probability corresponding to the current second sampling point, the second sample change score corresponding to the current second sampling point for the current second candidate perspective can be calculated. In this way, the accuracy of calculating the second sample change score can be further improved.

[0115] In addition, step S104 may further include: acquiring the sampling points included in the scene change region, and using the newly added sampling points included in the scene change region and the first sampling points other than the newly added sampling points as the second sampling points; setting the second probability corresponding to the newly added sampling points to 1, and using the first probability corresponding to the first sampling points other than the newly added sampling points as the second probability corresponding to the first sampling points other than the newly added sampling points.

[0116] In this embodiment, the second sampling points can be composed of two parts, namely, the newly added sampling points obtained by sampling the scene change region after detecting the scene change region during the flight of the drone, and the part of the first sampling points other than the newly added sampling points. And the second probabilities corresponding to the two parts of sampling points are different. Since the newly added sampling points must be located in the scene change region, the second probability corresponding to this part of sampling points is set to 1. For the first sampling points other than the newly added sampling points, the first probability corresponding to the first sampling points is used as the second probability of this part of the second sampling points.

[0117] For example, the second sampling point The second probability can be represented by the following formula:

[0118]

[0119] where is the prior probability, that is, the first probability.

[0120] In this embodiment, the newly added sampling points included in the scene change region and the first sampling points other than the newly added sampling points can both be used as the second sampling points, and the second probability for the newly added sampling points is 1, while the second probability for the first sampling points other than the newly added sampling points remains the original first probability. In this way, the accuracy of obtaining the second sampling points and the second probability can be improved.

[0121] In one embodiment, step S102 may further include: performing Poisson disk sampling on the initial scene reconstruction model to obtain a first candidate view set; step S105 may further include: excluding the already visited views during the flight of the drone in the target area from the target views to obtain an unvisited view set; performing Poisson disk sampling on the scene change area to obtain a candidate visual set for the change area; and taking the union between the unvisited view set and the candidate visual set for the change area as the second candidate view set.

[0122] In this embodiment, the acquisition of the first candidate view set can be obtained by performing Poisson disk sampling on the initial scene reconstruction model, while the second candidate view set can be composed of the union of two sets, namely the unvisited view set and the candidate visual set for the change area. Among them, the unvisited view set refers to the set composed of unvisited views, and this set can be obtained by excluding the already visited views during the flight of the drone in the target area from the target views. The candidate visual set for the change area is obtained by performing Poisson disk sampling on the scene change area.

[0123] Continuing with the prior flight path of the drone as an example of view A - view B - view C - view D, during the flight of the drone according to the prior flight path, if a scene change area is detected after flying from view A to view B, at this time the target views can include view A, view B, view C, and view D, where the already visited views are view A and view B, then the unvisited view set includes view C and view D. Also, the terminal can perform Poisson disk sampling on the scene change area to obtain a candidate visual set for the change area, for example, it can include view C and view E. Then, at this time, the second candidate view set can be the union of the above two view sets, that is, the second candidate views included can be view C, view D, and view E.

[0124] In this embodiment, Poisson disk sampling can be performed on the initial scene reconstruction model to obtain the first candidate view set, and the union between the unvisited view set and the candidate visual set for the change area is taken as the second candidate view set. By this means, the integrity of candidate view screening can be ensured.

[0125] In one embodiment, an online path planning method for drones facing scenario updates is also provided. By introducing a heuristic variability method and based on previous scenario reconstruction data and change probability statistics, it preferentially selects areas that are likely to change for exploration. This can effectively avoid repeated exploration of static areas, significantly reduce flight time and computational overhead, and achieve rapid detection and response to urban environmental changes. By combining the detection of changing areas with real-time path planning, the system can dynamically adjust the flight trajectory to ensure that only the changing areas are focused on for exploration and update, thereby improving the real-time performance and accuracy of change detection. And a dual strategy of prior path and real-time path is adopted, combining statistical prior data and real-time change detection to ensure that when performing three-dimensional scenario updates, the information acquisition efficiency can be maximized and unnecessary data collection can be reduced. The principle of this method is as follows:

[0126] First of all, the core of this embodiment is to introduce a changeability heuristic to guide the trajectory planning. By analyzing the previously reconstructed data (such as the three-dimensional scenario at time T1) and change probability statistics, the system can evaluate the change possibility of each area. This changeability heuristic calculates the change probability of each area based on historical data and scenario statistics, thus providing a basis for trajectory planning. The system divides the trajectory into two parts: prior path and real-time path. The prior path is planned based on the model reconstructed at T1. By preferentially exploring areas with higher change probabilities, it avoids repeated exploration of static areas, thereby reducing unnecessary flight time and consumption of computational resources.

[0127] When the drone flies along the prior path, if a changing area is found, real-time path planning will be triggered. Real-time path planning can dynamically adjust the path according to the image differences captured during flight, preferentially explore the changing area, and continue to adjust the path through online updates to ensure full coverage of the changing area. During the exploration of the changing area, the system combines image feature matching and three-dimensional reconstruction methods to generate an accurate point cloud of the changing area, and uses the convex hull algorithm to extract the geometric information of the changing area for subsequent three-dimensional reconstruction and update.

[0128] To further improve efficiency, real-time path planning also takes into account the redundancy of the explored areas. By reducing the acquisition of repeated perspectives, it ensures that the changing areas are fully explored without wasting time on static areas. During the real-time path planning process, the system continuously updates the target area and uses the changeability heuristic to select the next optimal perspective to achieve efficient exploration of the changing area.

[0129] This embodiment can be specifically implemented through the following process:

[0130] The candidate viewpoints of the prior path and the real-time path are designed to fully observe all regions of the target surface T on the predefined safety height plane h. For the prior path, T corresponds to the reconstructed surface of T1, while for the real-time path, T represents the convex hull of M detected changed regions, which is initially empty and gradually expands if there are changes.

[0131] The region for generating new candidate viewpoints is determined by expanding a padding margin on the target T to ensure sufficient coverage of all sampling points along the edge of the target convex hull. Within this expanded region, Poisson disk sampling is used to generate candidate viewpoints , where each candidate viewpoint contains its 3D position and orientation. To accommodate drones equipped with multiple cameras (e.g., five cameras) or single-camera systems, five candidate viewpoints are generated at the same location, each with a different orientation. However, this introduces redundancy, so it is necessary to prioritize and streamline these viewpoints. For the prior path, the goal is to reduce redundancy while maintaining coverage. For the real-time path, the focus is on selecting the next best viewpoint to explore the entire changed region. To achieve this goal, this embodiment defines a heuristic method to quantify the importance of each viewpoint, thus guiding the two path planning tasks.

[0132] As Figure 5 shown, given that the main goal is to detect changes rather than perform a complete reconstruction, in the experiment, all candidate viewpoints are generated on the safety height plane set to h = 120 meters. The viewpoint generation region is determined by expanding a padding margin around the target T, as Figure 5 shown. This padding ensures that the generated viewpoints can fully cover all sample points on the edge of the target T . The padding size is calculated based on the field of view angle of the camera and the safety height h, with the formula , where d is a small constant to ensure sufficient coverage. In the settings of this embodiment, d is set to the sampling radius used in the Poisson disk sampling step.

[0133] The goal of this heuristic method is to prioritize the detection of surface regions with a higher probability of change. The target surface T is uniformly discretized into N sample points , where each sample point is associated with a score , and:

[0134]

[0135] If a sample is confirmed to be part of a changed region in real-time path planning, its score , indicating the highest change probability. Otherwise, are the prior probabilities. These probabilities are calculated using the statistics of the WUSU dataset, which includes 12 types of urban structures. By analyzing the change rates of these categories, they are mapped to 7 semantic labels in the UrbanBIS dataset.

[0136] Visibility function determines whether a sample point is visible from a perspective :

[0137]

[0138] A sample point has a coverage that is the number of perspectives from which the point is observed:

[0139]

[0140] For real-time path planning, the perspective V is divided into an accessed sequence and an unaccessed sequence . The next best perspective is selected from to maximize the change probability. For prior path planning, all perspectives are initially unaccessed, .

[0141] The importance of a perspective is defined by the change probability of the samples it observes. To this end, first define the change probability of a sample relative to a perspective as:

[0142]

[0143] where and are scaling parameters that control the contributions of unaccessed and accessed perspectives, respectively. The negative factor associated with the accessed perspective reflects the diminishing marginal benefit: as a sample is observed by more accessed perspectives, its change probability gradually stabilizes. This conforms to the idea that once a sample is observed, its "changed" or "unchanged" state is gradually confirmed, and each additional observation yields less new information. Therefore, the change probability decreases, highlighting the inefficiency of redundant observations.

[0144] The overall process can be as shown in Figure 6As shown, the input is a labeled T1 reconstruction model, where each semantic label is associated with a prior probability. First, the prior path is planned by minimizing redundancy while maximizing the potential for change and coverage. When the drone flies along the prior path and detects a changed area, real-time path planning is triggered to explore the entire target changed area by analyzing the difference image. The output of the system is a set of convex hulls that contain all detected changed areas. Specifically, it can include the following parts:

[0145] (1) Prior path planning:

[0146] Starting from the reconstructed scene of T1 a sample set is derived, which represents discretized target points related to the probability of change. Using these inputs, a dense set of candidate viewpoints V is generated as described above. The main goal of prior path planning is to identify a minimal subset of viewpoints that, while reducing redundancy, ensures maximum coverage . This goal is inspired by the max-min optimization framework. The trajectory connecting these selected viewpoints forms the prior aerial path. To this end, an iterative optimization process is adopted to optimize the set of viewpoints by removing redundant viewpoints and maximizing the target coverage. This optimization problem can be expressed as:

[0147]

[0148] where the first term minimizes redundancy and the second term maximizes coverage. The set of viewpoints obtained after solving this optimization problem also serves as the initial candidate set for real-time path planning. Throughout the process, all viewpoints are considered unvisited, i.e., , and .

[0149] Redundancy is defined based on the probability of change. If a sample observed by a viewpoint (1) has been well covered by other viewpoints or (2) has a low probability of change, resulting in less importance of its observation, then that viewpoint is considered redundant. Therefore, the redundancy of a viewpoint can be quantified as the negative value of its heuristic importance, i.e., , the higher the redundancy, the lower the importance. The heuristic importance of a viewpoint is derived based on the probability of change of the samples it observes. The probability of change of sample is determined based on the set of viewpoints observing it. The probability of change is defined as:

[0150]

[0151] where, for any there is 。This formula ensures that the variability of a sample gradually decreases as it is observed from more perspectives. Then, the heuristic importance of a perspective is calculated based on the samples it observes:

[0152]

[0153] To construct a continuous trajectory, all the selected perspectives in are visited, framing the problem as a Traveling Salesman Problem (TSP). The solution ensures an efficient flight path that preferentially explores regions with a higher probability of change, which are based on previous statistics.

[0154] (2) Detection and extraction of changed regions:

[0155] As Figure 7 shown, this is an example of a real-time path applied to a scene in UrbanBIS. The T1 reconstruction model (upper left) serves as a prior, showing three existing buildings, while the T2 model (lower left) serves as the ground truth, where one building has been demolished. When the drone detects a change in the scene, it dynamically adjusts the path (middle) by identifying differences in the captured 2D images. Subsequently, based on the updated real-time path, the detected point cloud (lower right) and the convex hull of the changed region (upper right) are generated.

[0156] At T2, the drone will execute sequentially according to the planned path . At each specified perspective in this sequence, the drone captures a 2D image in the perspective direction . Then, each captured image is compared with the corresponding rendered image generated by the T1 reconstruction model, ensuring consistent perspectives and directions.

[0157] To detect and identify changed regions at the image level, a learning-based image feature matching model DKM is used and enhanced by a generalized self-training framework GIM. This framework is pre-trained using large-scale Internet videos, ensuring robustness in multiple scenarios. By comparing image pairs , regions without feature matches are identified as changed regions .

[0158] To extract the corresponding 3D information, a pre-trained multi-view stereo (MVS) reconstruction method DUSt3R is applied. The detected changed regions are used as masks to exclude unchanged regions, resulting in a 3D point cloud representing the identified changed regions. When multiple changed regions are detected, they are added to the processing queue and processed sequentially.

[0159] To ensure the performance of DUSt3R during path planning, the reconstruction only includes the eight most recent images from the most recent view sequence However, this limitation may lead to an incomplete point cloud. To enhance the current target area with the newly generated point cloud data , their overlap is evaluated by the intersection over union (IoU) metric. If where is a configurable similarity threshold, the target area is updated to . Otherwise, the target area will be replaced by the new point cloud, . .

[0160] At each view, the convex hull of the detected point cloud is extracted as the detected changed area. This convex hull serves as the target area for further exploration and provides information for real-time path planning decisions to dynamically determine the next best view.

[0161] (3) Real-time path planning for the changed area

[0162] Once the changed area is identified , the system switches to the real-time planning mode to dynamically and comprehensively explore the area. If multiple changed areas are detected, the system will preferentially select the changed area closest to the current location of the UAV as the next exploration target. Given the sequence of visited views and their corresponding change targets (if any), the goal of real-time planning is to select the next best view that maximizes the exploration of . As the exploration progresses, this process dynamically updates the target area to .

[0163] Similar to the prior path planning, the changed area is first sampled to obtain where . Let be the new unvisited view to be added to the flight path. To evaluate the contribution of a candidate view, first consider the change likelihood of all samples observable from that view. For a sample , with respect to the candidate next view , considering the sequence of visited views , its change likelihood is defined as:

[0164]

[0165] Candidate view The total change probability gain (or heuristic importance) is calculated as the additional information it provides beyond what is obtained from the sequence of visited views:

[0166]

[0167] Candidate view comes from two sources: (1) unvisited views from the prior path ; (2) a set of real-time views generated using Poisson disk sampling . These two sets are combined and views that can observe at least one sample are filtered out :

[0168]

[0169] For each , its change probability gain is calculated in parallel. To prevent extreme trajectory deviations, the system selects the top K candidate views with the highest change probability gains and selects the one closest to the current view as the next best view. In the experiment, K is set to 10. Then, the real-time path is constructed by sequentially connecting the visited path with the selected next best view. When all samples have been observed by the visited views , the exploration is considered complete. If multiple change regions are detected, the system will continue to process the next closest target , and repeat this process.

[0170] After all detected change regions have been explored, the system will revisit any remaining unobserved regions. Unvisited views from the prior path are filtered to include only those that can observe unvisited samples. Then, these views are connected into a continuous trajectory using the Traveling Salesman Problem (TSP) formulation, just as in the prior path planning phase, to ensure effective coverage of the remaining scene. By dynamically exploring the identified change regions and revisiting the unexplored regions, the real-time path planning strategy ensures the comprehensiveness and efficiency of scene updates.

[0171] This embodiment solves the problem of low efficiency in the prior art for urban scene updating through an online trajectory planning method that combines prior probability and real-time detection of changed areas. The key idea is to use the reconstruction model and change probability statistical data at time T1 in prior path planning to evaluate the change possibility of each area, and based on this, plan the initial flight path. In real-time path planning, when a changed area is detected, a new flight path is dynamically generated by analyzing the difference image to thoroughly explore the target changed area. This method realizes more efficient and accurate urban scene updating by reducing unnecessary flight time and computational overhead. Specifically, prior path planning optimizes the initial flight path by minimizing redundancy and maximizing the coverage of potential changed areas; real-time path planning ensures a comprehensive exploration of the detected changed areas by selecting the next best viewing angle. In addition, by combining statistical priors with real-time decision-making, the present invention achieves high-quality scene updating while significantly reducing resource consumption, making it more scalable and effective in large-scale complex urban environments.

[0172] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same moment, but can be executed at different moments, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0173] Based on the same inventive concept, the embodiments of the present application also provide a device for online planning of an unmanned aerial vehicle path for real-time detection of scene update changes, which is used to implement the above-mentioned online planning method for an unmanned aerial vehicle path for real-time detection of scene update changes. The implementation solutions provided by this device to solve problems are similar to the implementation solutions described in the above method. Therefore, the specific limitations in one or more embodiments of the device for online planning of an unmanned aerial vehicle path for real-time detection of scene update changes provided below can refer to the limitations on the online planning method for an unmanned aerial vehicle path for real-time detection of scene update changes in the above text, and will not be repeated here.

[0174] In one embodiment, as Figure 8As shown in the figure, an online path planning device for an unmanned aerial vehicle (UAV) that detects real-time scene update changes is provided, including: a first sampling acquisition module 801, a target view screening module 802, a prior path generation module 803, a second sampling acquisition module 804, a flight view screening module 805, and a real-time path generation module 806, where:

[0175] The first sampling acquisition module 801 is configured to obtain an initial scene reconstruction model of a target area, obtain first sampling points included in the initial scene reconstruction model, and obtain a first probability of each first sampling point in a scene change area in the target area;

[0176] The target view screening module 802 is configured to obtain a first candidate view set associated with the target area, obtain a first importance degree of each first candidate view included in the first candidate view set according to the first sampling points and the first probability, and obtain a target view from the first candidate views according to the first importance degree; the view set composed of the target views is the view set with the smallest number of views and the largest coverage range for the target area;

[0177] The prior path generation module 803 is configured to generate a prior flight path of the UAV according to each target view; the prior flight path is used to control the estimated flight of the UAV in the target area;

[0178] The second sampling acquisition module 804 is configured to, during the flight of the UAV in the target area according to the prior flight path, if a scene change area is detected, obtain second sampling points and a second probability of each second sampling point in the scene change area;

[0179] The flight view screening module 805 is configured to obtain a second candidate view set associated with the target area, obtain a second importance degree of each second candidate view included in the second candidate view set according to the second sampling points and the second probability, and obtain a target flight view from the second candidate views according to the second importance degree;

[0180] The real-time path generation module 806 is configured to adjust the prior flight path according to the target flight view to obtain a real-time flight path; the real-time flight path is used to control the real-time flight of the UAV in the target area to update the initial scene reconstruction model.

[0181] In one embodiment, the target view screening module 802 is further configured to obtain a current first candidate view and target first sampling points observable by the current first candidate view; the current first candidate view is any one of the first candidate views; obtain a first sample change score corresponding to each target first sampling point according to the first probability corresponding to each target first sampling point; and obtain the first importance degree of the current first candidate view based on each first sample change score.

[0182] In one embodiment, the target perspective screening module 802 is further configured to obtain a current first sampling point and the number of first candidate perspectives that can observe the current first sampling point; the current first sampling point is any one of the first sampling points; and obtain a first sample change score corresponding to the current first sampling point according to the number and the first probability corresponding to the current first sampling point.

[0183] In one embodiment, the first sampling acquisition module 801 is further configured to obtain the first sampling points included in the initial scene reconstruction model and the scene area type labels corresponding to the first sampling points; obtain the prior probabilities respectively corresponding to the scene area type labels; the prior probabilities are used to represent the possibility of scene changes occurring in the areas corresponding to the scene area type labels; and use the prior probabilities corresponding to the first sampling points as the first probabilities corresponding to the first sampling points.

[0184] In one embodiment, the flight perspective screening module 805 is further configured to obtain a current second candidate perspective; the current second candidate perspective is any one of the second candidate perspectives; obtain a second sample change score corresponding to each second sampling point with respect to the current second candidate perspective according to the second sampling point and the second probability; and obtain the second importance degree of the current second candidate perspective according to the second sample change scores corresponding to each second sampling point with respect to the current second candidate perspective.

[0185] In one embodiment, the flight perspective screening module 805 is further configured to obtain a current second sampling point and the accessed perspectives during the flight of the drone in the target area; the current second sampling point is any one of the second sampling points; obtain the visibility information of the current second sampling point according to the accessed perspectives and the current second candidate perspective; the visibility information is used to represent whether the current second sampling point can be observed by the current second candidate perspective and the accessed perspectives; and obtain a second sample change score corresponding to the current second sampling point with respect to the current second candidate perspective according to the second probability and the visibility information.

[0186] In one embodiment, the second sampling acquisition module 804 is further configured to obtain the sampling points included in the scene change area, and use the newly added sampling points included in the scene change area and the first sampling points other than the newly added sampling points as the second sampling points; set the second probability corresponding to the newly added sampling points to 1, and use the first probability corresponding to the first sampling points other than the newly added sampling points as the second probability corresponding to the first sampling points other than the newly added sampling points.

[0187] In one embodiment, the target view filtering module 802 is further configured to perform Poisson disk sampling on the initial scene reconstruction model to obtain a first candidate view set; the flight view filtering module 805 is further configured to remove the visited views during the flight of the drone in the target area from the target views to obtain an unvisited view set; perform Poisson disk sampling on the scene change area to obtain a candidate visual set for the change area; and use the union between the unvisited view set and the candidate visual set for the change area as the second candidate view set.

[0188] Each module in the above-mentioned drone path online planning device for real-time detecting scene update changes can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0189] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as Figure 9 shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program, when executed by the processor, implements a method for online planning of a drone path for real-time detecting scene update changes. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the outer shell of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0190] Those skilled in the art can understand that Figure 9The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0191] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0192] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0193] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0194] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0195] Those of ordinary skill in the art can understand that all or part of the processes in the above-described embodiment methods can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above various methods. Among them, any reference to a memory, database, or other medium used in the various embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAMs), magnetoresistive random access memories (MRAMs), ferroelectric random access memories (FRAMs), phase change memories (PCMs), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the various embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the various embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0196] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0197] The above-described embodiments only represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. An online path planning method for an unmanned aerial vehicle to detect real-time scene update changes, characterized in that, The method includes: Obtaining an initial scene reconstruction model of a target area, obtaining first sampling points included in the initial scene reconstruction model, and first probabilities of each of the first sampling points being located in a scene change area of the target area; Obtaining a first candidate view set associated with the target area, obtaining first importance degrees of each of the first candidate views included in the first candidate view set according to the first sampling points and the first probabilities, and obtaining a target view from the first candidate views according to the first importance degrees; the view set composed of the target views is the view set with the smallest number of views and the largest coverage range for the target area; Generating a prior flight path of the unmanned aerial vehicle according to each of the target views; the prior flight path is used to control the estimated flight of the unmanned aerial vehicle in the target area; During the flight of the unmanned aerial vehicle in the target area according to the prior flight path, if a scene change area is detected, obtaining second sampling points and second probabilities of each of the second sampling points being located in the scene change area; Obtaining a second candidate view set associated with the target area, obtaining second importance degrees of each of the second candidate views included in the second candidate view set according to the second sampling points and the second probabilities, and obtaining a target flight view from the second candidate views according to the second importance degrees; Adjusting the prior flight path according to the target flight view to obtain a real-time flight path; the real-time flight path is used to control the real-time flight of the unmanned aerial vehicle in the target area to update the initial scene reconstruction model; The obtaining the first importance degrees of each of the first candidate views included in the first candidate view set according to the first sampling points and the first probabilities includes: Obtaining a current first candidate view and target first sampling points observable by the current first candidate view; the current first candidate view is any one of the first candidate views; Obtaining first sample change scores corresponding to each of the target first sampling points according to the first probabilities corresponding to the target first sampling points; Obtaining the first importance degree of the current first candidate view based on the first sample change scores; The obtaining the first sample change scores corresponding to each of the target first sampling points according to the first probabilities corresponding to the target first sampling points includes: Obtaining a current first sampling point and the number of first candidate views observable by the current first sampling point; the current first sampling point is any one of the first sampling points; Obtaining the first sample change score corresponding to the current first sampling point according to the number and the first probability corresponding to the current first sampling point.

2. The method according to claim 1, wherein The obtaining the first sampling points included in the initial scene reconstruction model and the first probabilities of each of the first sampling points being located in the scene change area of the target area includes: Obtaining the first sampling points included in the initial scene reconstruction model and the scene area type labels corresponding to each of the first sampling points; Obtain prior probabilities corresponding to each of the scene area type tags respectively; the prior probabilities are used to characterize the possibility of scene changes occurring in the areas corresponding to each of the scene area type tags; Use the prior probabilities corresponding to each of the first sampling points as the first probabilities corresponding to each of the first sampling points.

3. The method according to claim 1, wherein The obtaining the second importance degrees of each of the second candidate viewpoints included in the second candidate viewpoint set according to the second sampling points and the second probabilities includes: Obtain a current second candidate viewpoint; the current second candidate viewpoint is any one of each of the second candidate viewpoints; According to the second sampling points and the second probabilities, obtain second sample change scores of each of the second sampling points corresponding to the current second candidate viewpoint; According to the second sample change scores of each of the second sampling points corresponding to the current second candidate viewpoint, obtain the second importance degree of the current second candidate viewpoint.

4. The method according to claim 3, characterized in that, The obtaining the second sample change scores of each of the second sampling points corresponding to the current second candidate viewpoint according to the second sampling points and the second probabilities includes: Obtain a current second sampling point and the viewpoints that have been visited during the flight of the drone in the target area; the current second sampling point is any one of each of the second sampling points; According to the viewpoints that have been visited and the current second candidate viewpoint, obtain visibility information of the current second sampling point; the visibility information is used to characterize whether the current second sampling point can be observed by the current second candidate viewpoint and the viewpoints that have been visited; According to the second probability and the visibility information, obtain the second sample change score of the current second sampling point corresponding to the current second candidate viewpoint.

5. The method according to claim 3 or 4, characterized in that, The obtaining the second sampling points and the second probabilities of each of the second sampling points located in the scene change area includes: Obtain the sampling points included in the scene change area, and use the newly added sampling points included in the scene change area and the first sampling points other than the newly added sampling points as the second sampling points; Set the second probability corresponding to the newly added sampling point to 1, and use the first probability corresponding to the first sampling points other than the newly added sampling point as the second probability corresponding to the first sampling points other than the newly added sampling point.

6. The method according to claim 1, characterized in that The obtaining the first candidate viewpoint set associated with the target area includes: Perform Poisson disk sampling on the initial scene reconstruction model to obtain the first candidate viewpoint set; The obtaining the second candidate viewpoint set associated with the target area includes: Exclude the viewpoints that have been visited during the flight of the drone in the target area from the target viewpoints to obtain an unvisited viewpoint set; Perform Poisson disk sampling on the scene change area to obtain a candidate visual set for the change area; Use the union between the unvisited viewpoint set and the candidate visual set for the change area as the second candidate viewpoint set.

7. An online path planning device for an unmanned aerial vehicle that real-time detects scene update changes, characterized in that, The device includes: The first sampling acquisition module is used to acquire an initial scene reconstruction model of a target area, and acquire first sampling points included in the initial scene reconstruction model, and first probabilities of each of the first sampling points located in a scene change area in the target area; The target perspective screening module is used to acquire a first candidate perspective set associated with the target area, obtain first importance degrees of each first candidate perspective included in the first candidate perspective set according to the first sampling points and the first probabilities, and acquire a target perspective from the first candidate perspectives according to the first importance degrees; the perspective set composed of the target perspectives is a perspective set with the smallest number of perspectives and the largest coverage range for the target area; The prior path generation module is used to generate a prior flight path of the unmanned aerial vehicle according to each of the target perspectives; the prior flight path is used to control the estimated flight of the unmanned aerial vehicle in the target area; The second sampling acquisition module is used to acquire second sampling points and second probabilities of each of the second sampling points located in the scene change area if a scene change area is detected during the flight of the unmanned aerial vehicle in the target area according to the prior flight path; The flight perspective screening module is used to acquire a second candidate perspective set associated with the target area, obtain second importance degrees of each second candidate perspective included in the second candidate perspective set according to the second sampling points and the second probabilities, and acquire a target flight perspective from the second candidate perspectives according to the second importance degrees; The real-time path generation module is used to adjust the prior flight path according to the target flight perspective to obtain a real-time flight path; the real-time flight path is used to control the real-time flight of the unmanned aerial vehicle in the target area to update the initial scene reconstruction model; The target perspective screening module is further used to acquire a current first candidate perspective and target first sampling points observable by the current first candidate perspective; the current first candidate perspective is any one of the first candidate perspectives; obtain first sample change scores corresponding to each of the target first sampling points according to the first probabilities corresponding to the target first sampling points; and obtain the first importance degree of the current first candidate perspective based on the first sample change scores; The target perspective screening module is further used to acquire a current first sampling point and the number of first candidate perspectives observable by the current first sampling point; the current first sampling point is any one of the first sampling points; and obtain a first sample change score corresponding to the current first sampling point according to the number and the first probability corresponding to the current first sampling point.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images