An unmanned aerial vehicle emergency scene inspection method and system based on deep reinforcement learning

Abstract

The application provides a kind of unmanned plane emergency scene inspection method and system based on deep reinforcement learning, the method comprises the following steps: first, the position coordinates required to be inspected by disaster are obtained by inspection control center;Then the unmanned plane starts to execute the inspection task, in the process of unmanned plane executing the inspection task, the trajectory of unmanned plane is optimized to minimize the time of unmanned plane inspection through the position coordinates of known inspection points;When optimizing the trajectory of unmanned plane, the trajectory of unmanned plane is reasonably optimized using Dueling-DQN algorithm in deep reinforcement learning and collecting inspection data;When reaching the maximum training number, the optimal trajectory of unmanned plane and the collected inspection data are output, thereby reducing the total inspection time of the system.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and wireless communication, specifically to a method and system for emergency scene inspection using unmanned aerial vehicles (UAVs) based on deep reinforcement learning. Background Technology

[0002] In recent years, the demand for emergency communications has surged in disaster relief efforts following natural disasters such as earthquakes, tsunamis, floods, and forest fires. Disaster-stricken areas often face damage to communication infrastructure and difficulty in obtaining timely and detailed information about the disaster area, posing significant challenges to rapid rescue efforts. Therefore, quickly obtaining data from disaster areas is crucial for reducing casualties and minimizing economic losses. Current emergency inspection systems typically consume considerable time during inspections and suffer from drawbacks such as low transmission rates, insufficient bandwidth, and high latency during communication. Therefore, building a system capable of rapidly completing inspection tasks and transmitting inspection information to a data processing center in emergency scenarios is essential, as it can save rescue time, reduce casualties, and minimize economic losses.

[0003] With technological advancements and rapid economic development, drones have become increasingly affordable, leading to their widespread application not only in the military but also in civilian use. Drone inspection, in particular, is considered an effective way to improve inspection efficiency due to its advantages such as high maneuverability, rapid deployment, unrestricted flight by terrain, and high-probability Line-of-Sight (LoS) communication channels. However, the communication resources (such as bandwidth) of a drone communication system are limited. Proper allocation of these resources can significantly reduce data transmission time during drone inspections. Summary of the Invention

[0004] To address the aforementioned shortcomings of existing technologies, this invention proposes a method and system for UAV emergency scenario inspection based on deep reinforcement learning. After acquiring the location of inspection points in an emergency scenario, the UAV optimizes its trajectory and wireless communication resources, thereby reducing the total inspection time of the system.

[0005] A method for drone emergency scene inspection based on deep reinforcement learning includes the following steps:

[0006] Step S1. The inspection control center determines the coordinates of the locations that the drone needs to inspect and the location information of each ground rescue personnel based on the request sent by the emergency area;

[0007] Step S2. The UAV begins to perform the inspection task based on the obtained coordinates of the locations to be inspected and the location information of each ground rescuer. During the UAV's inspection task, the Dueling-DQN algorithm in deep reinforcement learning is used to optimize the UAV's trajectory by using the known coordinates of the inspection points to minimize the data collection time during the UAV inspection process.

[0008] Step S3. Output the optimized trajectory of the UAV inspection and the collected inspection data.

[0009] Furthermore, the specific process of step S2 is as follows:

[0010] Step S21. Parameter initialization: Initialize the various parameters of the UAV system, including the current position coordinates of the UAV and the parameters of the initialization model;

[0011] Step S22. The drone acquires the coordinates of the nearest inspection point to its current location and determines the location w of the inspection point it is currently inspecting. m =[x m ,y m ,z m ], where x m y m z m These represent the coordinates of the inspection points on the X, Y, and Z axes, respectively.

[0012] Step S23. Optimize the UAV trajectory: In the process of optimizing the inspection trajectory of the UAV using deep reinforcement learning, the established problem is first transformed into a Markov decision problem, which can be composed of four parts, namely...<S,A,P,R> Where S is the state space, A is the action space, P is the state transition probability when the UAV performs a task, and R is the reward function when the UAV performs an inspection task. Assume the current position of the UAV is q. n =[x n ,y n ,z n ], where x n y n z n Let v represent the coordinates of the UAV on the X, Y, and Z axes, respectively. Then, the UAV selects a current flight direction v from the action space A according to the strategy selection mechanism P. n The drone then moves to the next location q. n+1 =[x n+1 ,y n+1 ,z n+1 Assuming the distance the drone travels within one time slot is... The state transition of the UAV follows the formula:

[0013]

[0014] Step S24. The difference in distance between the inspection points selected by the drone based on the distance before and after movement. Receive a reward r n ,in This indicates the current distance between the drone and the selected inspection point. This indicates that the UAV selects a current flight direction v from the action space A according to the strategy selection mechanism P. n The distance from the selected inspection point is then used to design the reward as follows:

[0015]

[0016] Where r a A positive value indicates that the drone gets closer to the selected inspection point after the strategy selection movement; r b A negative value indicates that the drone is moving further and further away from the selected inspection point after the strategy selection process; furthermore, if the drone completes the inspection task, it will receive a large positive reward r. c ;

[0017] Step S25. Transfer the state transition result (q) n ,v n ,r n ,q n+1 Saved to the system's experience pool;

[0018] Step S26. Select N1 steps of samples from the experience pool and use gradient descent to reduce the loss function of the neural network, thereby optimizing the drone's trajectory and obtaining a larger reward. The loss function is defined as:

[0019]

[0020] Where r n+1 Let λ represent the reward the drone receives in time slot n+1, λ represent the discount factor, and θ represent the reward the drone receives in time slot n+1. * Let θ represent the factors affecting the parameters of the neural network model, and Q(q) n ,a n |θ) represents the current position q of the drone in the current network. n Take action a n Q value, This indicates the drone's current location q in the target network. n+1 Take action The Q value, the drone collects the current position q n+ 1. Data from inspection points and optimized allocation of communication resources to save time on data collection during inspections;

[0021] Step S27. Determine whether the drone has completed the inspection task. If not, the drone will proceed to step S22. If yes, proceed to step S28.

[0022] Step S28. Determine whether the drone has completed all inspection tasks. If not, proceed to step S22. If yes, the inspection task ends.

[0023] Step S29. Determine whether the maximum number of iterations has been reached. If not, repeat steps S22)-S28) until the algorithm reaches the maximum number of iterations.

[0024] An intelligent power line inspection system based on multiple drones includes an inspection task acquisition module, an inspection data collection module, a drone trajectory optimization module, and a result output module.

[0025] The inspection task acquisition module is used to determine the location coordinates of the location that the drone needs to inspect and the location information of each ground rescuer based on the request sent by the emergency area.

[0026] The UAV trajectory optimization module is used to optimize the UAV trajectory using the Dueling-DQN algorithm in deep reinforcement learning, based on the known location coordinates of the inspection points, during the UAV's inspection mission, thereby minimizing the UAV inspection time.

[0027] The inspection data collection module is used to collect inspection data during the inspection mission performed by the UAV.

[0028] The result output module step is used to output the optimized trajectory of the UAV inspection and the collected inspection data.

[0029] Furthermore, during the drone's inspection mission, the drone trajectory optimization module optimizes the drone's trajectory using the Dueling-DQN algorithm in deep reinforcement learning, based on the known coordinates of the inspection points, to minimize the drone's inspection time. Specifically, this includes:

[0030] Step S21. Parameter initialization: Initialize the various parameters of the UAV system, including the current position coordinates of the UAV and the parameters of the initialization model;

[0031] Step S22. The drone acquires the coordinates of the nearest inspection point to its current location and determines the location w of the inspection point it is currently inspecting. m =[x m ,y m ,z m ], where x m y m z mThese represent the coordinates of the inspection points on the X, Y, and Z axes, respectively.

[0032] Step S23. Optimize the UAV trajectory: In the process of optimizing the inspection trajectory of the UAV using deep reinforcement learning, the established problem is first transformed into a Markov decision problem, which can be composed of four parts, namely...<S,A,P,R> Where S is the state space, A is the action space, P is the state transition probability when the UAV performs a task, and R is the reward function when the UAV performs an inspection task. Assume the current position of the UAV is q. n =[x n ,y n ,z n ], where x n y n z n Let v represent the coordinates of the UAV on the X, Y, and Z axes, respectively. Then, the UAV selects a current flight direction v from the action space A according to the strategy selection mechanism P. n The drone then moves to the next location q. n+1 =[x n+1 ,y n+1 ,z n+1 Assuming the distance the drone travels within one time slot is... The state transition of the UAV follows the formula:

[0033]

[0034] Step S24. The difference in distance between the inspection points selected by the drone based on the distance before and after movement. Receive a reward r n ,in This indicates the current distance between the drone and the selected inspection point. This indicates that the UAV selects a current flight direction v from the action space A according to the strategy selection mechanism P. n The distance from the selected inspection point is then used to design the reward as follows:

[0035]

[0036] Where r a A positive value indicates that the drone gets closer to the selected inspection point after the strategy selection movement; r b A negative value indicates that the drone is moving further and further away from the selected inspection point after the strategy selection process; furthermore, if the drone completes the inspection task, it will receive a large positive reward r. c ;

[0037] Step S25. Transfer the state transition result (q) n ,vn ,r n ,q n+1 Saved to the system's experience pool;

[0038] Step S26. Select N1 steps of samples from the experience pool and use gradient descent to reduce the loss function of the neural network, thereby optimizing the drone's trajectory and obtaining a larger reward. The loss function is defined as:

[0039]

[0040] Where r n+1 Let λ represent the reward the drone receives in time slot n+1, λ represent the discount factor, and θ represent the reward the drone receives in time slot n+1. * Let θ represent the factors affecting the parameters of the neural network model, and Q(q) n ,a n |θ) represents the current position q of the drone in the current network. n Take action a n Q value, This indicates the drone's current location q in the target network. n+1 Take action The Q value, the drone collects the current position q n+1 The data from the inspection points is used to optimize the allocation of communication resources, thereby saving time for data collection during inspections.

[0041] Step S27. Determine whether the drone has completed the inspection task. If not, the drone will proceed to step S22. If yes, proceed to step S28.

[0042] Step S28. Determine whether the drone has completed all inspection tasks. If not, proceed to step S22. If yes, the inspection task ends.

[0043] Step S29. Determine whether the maximum number of iterations has been reached. If not, repeat steps S22)-S28) until the algorithm reaches the maximum number of iterations.

[0044] The present invention has the following advantages:

[0045] 1. This invention minimizes inspection time in appropriate scenarios by optimizing the trajectory of drones. It can complete the inspection task in a given emergency scenario in the shortest time, and has the advantages of low cost, low latency and high precision.

[0046] 2. Applying the Dueling-DQN algorithm to the emergency scenario time minimization inspection system makes the algorithm more stable and converges faster.

[0047] 3. The algorithm has strong scalability. It can not only be used for drone inspection to achieve the purpose of emergency inspection, but also be applied to more complex scenarios. The improved algorithm has strong robustness. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating the UAV emergency scenario inspection method based on deep reinforcement learning, according to an embodiment of the present invention.

[0049] Figure 2 This is a flowchart illustrating the process of acquiring inspection tasks in an embodiment of the present invention;

[0050] Figure 3 This is a flowchart of drone inspection data collection in an embodiment of the present invention;

[0051] Figure 4 This is a flowchart illustrating the optimization of UAV inspection trajectories based on deep reinforcement learning algorithms according to an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Figure 1 The diagram illustrates a flowchart of a UAV emergency scene inspection method based on deep reinforcement learning, according to an embodiment of the present invention. It assumes the system includes one UAV and M inspection points distributed within a K×Kkm area, and that the total time for the UAV to complete its inspection task is T. This invention aims to propose a scenario-based emergency rescue inspection method that leverages the high maneuverability and low cost of UAVs to quickly complete a given inspection task, enabling the inspection system to complete the task in the shortest possible time. During the UAV's inspection process, deep reinforcement learning is used to optimize the UAV's trajectory, thereby minimizing the inspection time T.

[0054] The method includes the following steps:

[0055] Step 101: The process begins.

[0056] Step 102: Obtaining inspection tasks.

[0057] Step 103: Inspection data collection.

[0058] Step 104: Drone trajectory optimization.

[0059] Step 105: Determine if the maximum number of training iterations has been reached.

[0060] Step 106: Output the optimal trajectory of the drone and the collected data.

[0061] Step 107: Process ends.

[0062] Further explaining step 102, the service center determines the inspection points based on the inspection cycle or other inspection factors, such as... Figure 2 As shown, the specific steps are as follows:

[0063] Step 201: Process begins.

[0064] Step 202: The service center determines whether an inspection is needed based on the needs of the emergency disaster area.

[0065] Step 203: Obtain the location coordinates of all task inspection points based on GPS, information collected about the disaster area, or inspection coordinates provided by rescue personnel.

[0066] Step 204: Process ends.

[0067] Next, data collection from drone inspections will be conducted in step 103. Specific steps are as follows: Figure 3 As shown:

[0068] Step 301: Process begins.

[0069] Step 302: The drone collects inspection point data based on the inspection point location and the inspection point location.

[0070] Step 303: Process ends.

[0071] Next, step 104 uses the Dueling-DQN algorithm from deep reinforcement learning to optimize the UAV's trajectory, focusing on minimizing the time required for the UAV to collect inspection data (such as taking photos and videos of the inspection area), thereby saving inspection time and giving ground emergency systems more time. Compared to the traditional DQN algorithm, it not only solves the overestimation problem of DQN, but also uses a multi-step propagation algorithm to accelerate the convergence speed, and a dueling network to make the algorithm more stable. The trajectory optimization minimizes the completion time of the UAV inspection task. Specific steps are as follows: Figure 4 As shown:

[0072] Step 401: Process begins.

[0073] Step 402: Parameter Initialization. Initialize various parameters of the UAV system, such as the UAV's current position coordinates and the parameters of the initialization model.

[0074] Step 403: The drone determines the current inspection point. Specifically, the drone acquires the coordinates of the nearest inspection point to its current location and determines the current inspection point location w. m =[x m ,y m ,z m ], where x m y m z m These represent the coordinates of the inspection points on the X, Y, and Z axes, respectively.

[0075] Step 404: The drone optimizes its trajectory for the current time slot.

[0076] In the process of optimizing inspection trajectories using deep reinforcement learning for drones, we first transform the established problem into a Markov decision problem, which consists of four parts, namely...<S,A,P,R> Let S be the state space, A be the action space, P be the state transition probability of the UAV when performing a task, and R be the reward function of the UAV when performing an inspection task. Assume the current position of the UAV is q. n =[x n ,y n ,z n ], where x n y n z n Let v represent the coordinates of the UAV on the X, Y, and Z axes, respectively. Then, the UAV selects a current flight direction v from the action space A according to the strategy selection mechanism P. n The drone then moves to the next location q. n+1 =[x n+1 ,y n+1 ,z n+1 Assume the distance the drone travels within one time slot is... The state transition of the UAV follows the formula

[0077]

[0078] 4) Then, the difference in distance between the inspection points selected by the drone based on the distance before and after movement. Receive a reward r n ,in This indicates the current distance between the drone and the selected inspection point. This indicates that the UAV selects a current flight direction v from the action space A according to the strategy selection mechanism P. n The distance to the selected inspection point. The reward design can be represented as...

[0079]

[0080] Where r a A positive value indicates that the drone gets closer to the selected inspection point after the strategy selection movement; r b A negative value indicates that the drone is moving further and further away from the selected inspection point after the strategy selection process; furthermore, if the drone completes the inspection task, it will receive a large positive reward r. c .

[0081] 5) Then the result of the state transition (q) n ,v n ,r n ,q n+1 Save it to the system's experience pool.

[0082] 6) Randomly select N1 steps of samples from the experience pool and use gradient descent to reduce the loss function of the neural network, thereby optimizing the drone's trajectory and obtaining a larger reward. The loss function is defined as follows:

[0083]

[0084] Where r n+1 Let λ represent the reward the drone receives in time slot n+1, λ represent the discount factor, and θ represent the reward the drone receives in time slot n+1. * Let θ represent the factors affecting the parameters of the neural network model, and Q(q) n ,a n |θ) represents the current position q of the drone in the current network. n Take action a n Q value, This indicates the drone's current location q in the target network. n+1 Take action The Q value.

[0085] Step 405: Determine whether the drone has completed the inspection task for this inspection point. If not, the drone proceeds to step 403; if yes, it proceeds to step 406.

[0086] Step 406: Determine whether the drone has completed the inspection tasks for all inspection points. If not, proceed to step 403; if yes, the inspection task ends.

[0087] Step 407: Determine if the algorithm has reached the maximum number of iterations. If not, repeat steps 402-406 until the algorithm reaches the maximum number of iterations.

[0088] Step 408: Process ends.

[0089] This invention addresses the need for emergency inspections in complex emergency scenarios, such as major natural disasters (mudslides, earthquakes, etc.). It proposes a UAV emergency scenario inspection method and system based on deep reinforcement learning. This method not only solves the problem of obtaining information about emergency areas in complex emergency scenarios, but also improves the emergency response capabilities of the entire system (e.g., it can quickly send inspection information of emergency areas to ground emergency rescue personnel, who can quickly formulate rescue plans). It features low cost, low latency, and high transmission rate.

[0090] This invention enables the completion of a given inspection task in the shortest possible time based on inspection coordinates provided by ground rescue personnel or obtained via GPS, achieving the goal of rapid inspection. It addresses the stability and convergence issues of the algorithm by improving its network architecture to optimize convergence speed and stability. Furthermore, it optimizes the drone's trajectory using deep reinforcement learning based on the location coordinates of all inspection points, thereby reducing the total inspection time and improving the overall system performance.

[0091] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for emergency scene inspection using unmanned aerial vehicles (UAVs) based on deep reinforcement learning, characterized in that, Includes the following steps: Step S1. The inspection control center determines the coordinates of the locations that the drone needs to inspect and the location information of each ground rescue personnel based on the request sent by the emergency area; Step S2. The UAV begins to perform the inspection task based on the obtained coordinates of the locations to be inspected and the location information of each ground rescuer. During the UAV's inspection task, the Dueling-DQN algorithm in deep reinforcement learning is used to optimize the UAV's trajectory by using the known coordinates of the inspection points to minimize the UAV's inspection time. Step S3. Output the optimized trajectory of the UAV inspection and the collected inspection data; The specific process of step S2 is as follows: Step S21. Parameter initialization: Initialize the various parameters of the UAV system, including the current position coordinates of the UAV and the parameters of the initialization model; Step S22. The drone obtains the coordinates of the nearest inspection point to its current location and determines the location of the inspection point it is currently inspecting. ,in , , These represent the coordinates of the inspection points on the X, Y, and Z axes, respectively. Step S23. Optimize the UAV trajectory: In the process of optimizing the inspection trajectory of the UAV using deep reinforcement learning, the established problem is first transformed into a Markov decision problem, which can be composed of four parts, namely... ,in For state space, For the action space, Let be the state transition probability when the drone performs a mission. Let the reward function be the drone's position when performing inspection tasks. ,in , , These represent the coordinates of the UAV on the X, Y, and Z axes, respectively. The UAV then selects a strategy based on the chosen mechanism. From the action space Select a current flight direction The drone then moves to the next location. Assuming the distance the drone travels within one time slot is The state transition of the UAV follows the formula: ； Step S24. The difference in distance between the inspection points selected by the drone based on the distance before and after movement. Receive a reward ,in , This indicates the current distance between the drone and the selected inspection point. This indicates that the drone selects a strategy mechanism. From the action space Select a current flight direction The distance from the selected inspection point is then used to design the reward as follows: ； in It is a positive value, which indicates that the drone gets closer and closer to the selected inspection point after the strategy selection movement; A negative value indicates that the drone is moving further and further away from the selected inspection point after the strategy selection process; furthermore, the drone receives a large positive reward if it completes the inspection task. ; Step S25. Result of the state transition Saved to the system's experience pool; Step S26. Select from the experience pool The algorithm iterates through samples and uses gradient descent to reduce the loss function of the neural network, thereby optimizing the drone's trajectory and obtaining a larger reward. The loss function is defined as follows: ； in Indicates that drones are in Rewards earned through time slots Indicates the discount factor. and These represent factors that influence the parameters of a neural network model. This indicates the current location of the drone in the current network. Take action of value, This indicates the drone's current location within the target network. Take action of value; Step S27. Determine whether the drone has completed the inspection task. If not, the drone executes step S22. If yes, execute step S28. Step S28. Determine whether the drone has completed all inspection tasks. If not, proceed to step S22. If yes, the inspection task ends. Step S29. Determine whether the maximum number of iterations has been reached. If not, repeat steps S22) to S28) until the algorithm reaches the maximum number of iterations.

2. A drone emergency scene inspection system based on deep reinforcement learning, characterized in that: It includes a patrol task acquisition module, a patrol data collection module, a drone trajectory optimization module, and a result output module; The inspection task acquisition module is used to determine the location coordinates of the location that the drone needs to inspect and the location information of each ground rescuer based on the request sent by the emergency area. The UAV trajectory optimization module is used to optimize the UAV trajectory using the Dueling-DQN algorithm in deep reinforcement learning, based on the known location coordinates of the inspection points, during the UAV's inspection mission, thereby minimizing the UAV inspection time. The inspection data collection module is used to collect inspection data during the inspection mission performed by the UAV. The result output module is used to output the optimized trajectory of the UAV inspection and the collected inspection data. The UAV trajectory optimization module optimizes the UAV's trajectory during inspection tasks by using the known coordinates of inspection points and the Dueling-DQN algorithm in deep reinforcement learning to minimize the UAV inspection time. Specifically, this includes: Step S21. Parameter initialization: Initialize the various parameters of the UAV system, including the current position coordinates of the UAV and the parameters of the initialization model; Step S22. The drone obtains the coordinates of the nearest inspection point to its current location and determines the location of the inspection point it is currently inspecting. ,in , , These represent the coordinates of the inspection points on the X, Y, and Z axes, respectively. Step S23. Optimize the UAV trajectory: In the process of optimizing the inspection trajectory of the UAV using deep reinforcement learning, the established problem is first transformed into a Markov decision problem, which can be composed of four parts, namely... ,in For state space, For the action space, Let be the state transition probability when the drone performs a mission. Let the reward function be the drone's position when performing inspection tasks. ,in , , These represent the coordinates of the UAV on the X, Y, and Z axes, respectively. The UAV then selects a strategy based on the chosen mechanism. From the action space Select a current flight direction The drone then moves to the next location. Assuming the distance the drone travels within one time slot is The state transition of the UAV follows the formula: ； Step S24. The difference in distance between the inspection points selected by the drone based on the distance before and after movement. Receive a reward ,in , This indicates the current distance between the drone and the selected inspection point. This indicates that the drone selects a strategy mechanism. From the action space Select a current flight direction The distance from the selected inspection point is then used to design the reward as follows: ； in It is a positive value, which indicates that the drone gets closer and closer to the selected inspection point after the strategy selection movement; A negative value indicates that the drone is moving further and further away from the selected inspection point after the strategy selection process; furthermore, the drone receives a large positive reward if it completes the inspection task. ; Step S25. Result of the state transition Saved to the system's experience pool; Step S26. Select from the experience pool The algorithm iterates through samples and uses gradient descent to reduce the loss function of the neural network, thereby optimizing the drone's trajectory and obtaining a larger reward. The loss function is defined as follows: ； in Indicates that drones are in Rewards earned through time slots Indicates the discount factor. and This represents the factors that influence the parameters of the neural network model, and represents the current position of the drone in the current network. Take action of value, This indicates the drone's current location within the target network. Take action of Value, the drone collects the current location The data from the inspection points is used to optimize the allocation of communication resources, thereby saving time for data collection during inspections. Step S27. Determine whether the drone has completed the inspection task. If not, the drone executes step S22. If yes, execute step S28. Step S28. Determine whether the drone has completed all inspection tasks. If not, proceed to step S22. If yes, the inspection task ends. Step S29. Determine whether the maximum number of iterations has been reached. If not, repeat steps S22) to S28) until the algorithm reaches the maximum number of iterations.

Images