A mobile robot social navigation method based on reinforcement learning

Abstract

The application relates to a mobile robot social navigation method based on reinforcement learning, and belongs to the technical field of reinforcement learning, in particular to a social navigation method based on reinforcement learning. First, a social navigation problem is modeled to obtain space-time state information such as an unmanned vehicle state and a pedestrian state; second, a reinforcement learning model based on DQN is constructed, and the model comprises four modules, namely, an interaction module, a historical feature extraction module, a pooling module and a planning module. The interaction module is used for processing interaction features of the mobile robot and the pedestrians, the historical feature extraction module is used for extracting time dimension features between the mobile robot and the crowd, the pooling module is used for converting an arbitrary number of pedestrian inputs into a fixed length feature vector output, and the planning module plans a feasible path according to the unmanned vehicle state and the extracted crowd features. Finally, the model is pre-trained through imitation learning of expert experience to accelerate the convergence speed of the model.

Description

TECHNICAL FIELD

[0001] The application relates to a mobile robot social navigation method based on reinforcement learning and belongs to the technical field of reinforcement learning. BACKGROUND

[0002] With the rapid development of science and technology, people's demand for mobile robots is increasing, and mobile robots are widely used in unmanned delivery and other fields, greatly facilitating people's life. Meanwhile, in the intelligent manufacturing scene, the production mode of man-machine coexistence requires mobile robots to complete raw material distribution and other related work in the presence of people. Mobile robot navigation technology is one of the key technologies for the application of mobile robots in various scenes. An excellent navigation algorithm can not only improve the work efficiency of unmanned vehicles, but also reduce energy consumption and cost. Nowadays, people's performance requirements for the navigation algorithm of mobile robots are gradually increasing. People not only hope that mobile robots can reach the target point without collision, but also hope that the robots have certain social attributes during the journey and can interact with pedestrians in a friendly manner. Obviously, the traditional navigation algorithm cannot meet people's needs, and it is urgent to explore an efficient navigation algorithm.

[0003] It is worth noting that in recent years, with the continuous development of artificial intelligence technology, reinforcement learning technology has made great progress, especially the development of deep neural networks has provided a new idea for traditional reinforcement learning algorithms. In recent years, deep reinforcement learning algorithms have shown excellent ability, especially in the field of electronic games, and the level of players trained by reinforcement learning algorithms has surpassed human experts. At the same time, the super-strong ability of AlphaGo based on deep reinforcement learning has prompted scholars to think about how to use reinforcement learning methods to solve practical problems. Under this background, using reinforcement learning algorithms to solve path planning problems has gradually become a research hotspot.

[0004] With the deepening of research, social navigation algorithms based on reinforcement learning are emerging. Chen et al. proposed a social nature of unmanned vehicle reinforcement learning path planning algorithm: CADRL algorithm, CADRL algorithm detects pedestrians in the surrounding environment through radar data, takes the state of the pedestrian and the state of the unmanned vehicle itself as the state input, selects the action through the greedy algorithm, and uses the value network to output the evaluation of the action. But this algorithm cannot learn any multi-pedestrian state input, limiting its scope of use. On this basis, Chen et al. proposed using LSTM network to extract features of pedestrian information, this method sorts the state of the pedestrian from far to near according to the distance between the pedestrian and the mobile robot, and inputs it into the recurrent neural network in turn, this method causes the features input by the decision module to be most relevant to the pedestrian closest to the mobile robot, and this algorithm only considers the interaction between the unmanned vehicle and the pedestrian, without modeling the crowd, making it difficult to completely describe the spatial features of the mobile robot and the crowd, reducing the reliability of the model prediction. Subsequently, Chen et al. proposed SARL algorithm, which sets up an attention module to weight the output of the interaction module between the unmanned vehicle and each pedestrian, to get the influence degree of each pedestrian on the unmanned vehicle, and constructs a coarse-grained grid map for each pedestrian to evaluate the influence of neighbors on the pedestrian itself. This method constructs a coarse-grained grid map that reflects social relationships, but for each pedestrian, the coarse-grained grid map only describes the speed information of the neighbors within a certain neighborhood range. When the crowd density increases, a smaller neighborhood range cannot effectively extract the crowd features, and a larger neighborhood range will increase the computational complexity of the algorithm and reduce the real-time performance of the algorithm. At the same time, the above algorithm only considers the spatial features between the mobile robot and the crowd, that is, the mobile robot can only respond to the current crowd state, and does not have the ability to predict based on the historical features of the crowd. Secondly, the above methods all assume that the crowd state is completely observable, but in actual application, due to factors such as line-of-sight obstruction, the mobile robot cannot obtain the state information of all pedestrians in the environment, and thus the above methods are prone to decision errors in actual scenarios. SUMMARY

[0005] To solve the above problems, the present application provides a mobile robot social navigation method based on reinforcement learning, which comprehensively considers the time and space features of the crowd, improving the efficiency of the social navigation algorithm. In the construction of the reinforcement learning model, the present application uses a graph attention network to construct the crowd interaction features, and introduces a spatiotemporal graph to describe the relationship between the mobile robot and the crowd. Through the recurrent neural network and the fully connected layer, the spatiotemporal graph features of the mobile robot and the crowd are extracted, realizing an efficient social navigation algorithm. The method of the present application can more accurately model the crowd, improve the perception of the unmanned vehicle to the environment, introduce the features of the mobile robot and the crowd in time and space, and consider the model construction method when the state of the pedestrian is not completely observable.

[0006] A mobile robot social navigation method based on reinforcement learning, the steps of the method comprising:

[0007] Step 1: Construct a model of the social navigation problem;

[0008] The social navigation problem is essentially a sequential decision problem, that is, at time t, a certain state s t , take a certain action a t , enter the next state s t+1 , and obtain a reward r t , get the four-tuple (s t , a t , r t , s t+1 ), and repeat the decision-making process until the termination state is reached.

[0009] In the social navigation problem, the state s t at time t is:

[0010]

[0011] where i = 1, 2, …, N represents the number of pedestrians, N is the number of pedestrians in the environment at time t, represents the joint state of the robot and each pedestrian, S t = [d gt , v pref , v xt , v yt , r] represents the state of the mobile robot at time t, where d gt represents the distance from the mobile robot to the target point, v pref represents the reference speed of the mobile robot, v xt , v yt represents the speed of the mobile robot in the x, y direction at time t, and r represents the radius of the mobile robot. represents the observable state of the i-th pedestrian at time t, where d t i represents the relative distance between the i-th pedestrian and the mobile robot at time t, represents the x, y coordinates of the i-th pedestrian relative to the mobile robot at time t, represents the speed of the i-th pedestrian relative to the mobile robot at time t, r i represents the assumed radius of the i-th pedestrian, and r + r i represents the sum of the radius of the mobile robot and the assumed radius of the i-th pedestrian.

[0012] The action a t at time t is defined as:

[0013] a t = [v xtv yt ]

[0014] action a at time t in state s t t reward r t is defined as:

[0015] r t = R(s t , a t )

[0016]

[0017] where d t represents the minimum distance between the vehicle and the pedestrian within the time interval [t-Δt,t], Δt represents the minimum time step for decision-making, P t c represents the position of the vehicle at time t, P g represents the target point of the vehicle.

[0018] Meanwhile, considering that the state of the crowd cannot be completely observed, a mask mask t is introduced in the modeling process, assuming that pedestrians can obtain the state of each other, while mobile robots cannot completely obtain the state information of all pedestrians in the crowd due to factors such as sensors. In order to describe this feature, the mask is defined as:

[0019]

[0020] where mask t is 1 when the state of the pedestrian is observable at time t, otherwise mask t is 0.

[0021] The optimal strategy π * : s t → a t of this sequential decision-making problem can be obtained by maximizing the following expected return.

[0022]

[0023]

[0024] where P(s t , a t , s t+Δt ) is the state transition probability from t to t+Δt, V * is the optimal value function, γ is the decay constant, and T is the termination time.

[0025] Step 2: Deep reinforcement learning network framework construction;

[0026] ​To solve the optimal strategy of the sequence decision problem constructed in step 1, the application builds a deep reinforcement learning decision network based on the traditional deep reinforcement learning algorithm DQN. The state information selected in step 1 includes the mobile robot state and the crowd state, and the dimensions of these state information are different and the quantities are indefinite. In order to effectively obtain the decision signal from these information, the application divides the deep network into four modules: interaction module, historical feature extraction module, pooling module and planning module. The functions of each module are as follows:

[0027] Interaction module: according to the constructed social navigation problem model, the interaction between the unmanned vehicle and the crowd is processed. Since the dimensions of the mobile robot state and the dimensions of the pedestrian state are different, the feature information needs to be extracted by using a fully connected layer first. The state features of the mobile robot are extracted by using a multi-layer fully connected layer:

[0028] r t =φ1(S t ;W1)

[0029] Where φ1 is a fully connected layer, and W1 is the weight corresponding to the fully connected layer.

[0030] Secondly, the features of each person-robot pair (mobile robot and single pedestrian) are extracted by using a multi-layer perceptron:

[0031] P t =φ2(J t ;W2)

[0032] The attention matrix of the crowd is calculated by using a graph attention mechanism, and the crowd features are extracted by using a graph convolutional neural network:

[0033] A t =Attention1(P t ;W3)

[0034] G t =GCN(A t ,P t ;W4)

[0035] Where Attention1 is an attention network.

[0036] The features extracted by the graph neural network and the person-robot pair (mobile robot and single pedestrian) features are combined as the input of the self-attention network to obtain the attention value of the mobile robot to each pedestrian:

[0037] scores t =Attention2(G t ,P t ;W5)

[0038] Where Attention2 is a self-attention network.

[0039] Considering that the robot cannot completely observe the state information of all pedestrians in the crowd, a mask is used t The attention value is weighted to represent the spatial characteristics of the robot and the crowd at the current time:

[0040] w t =softmax(scores x mask t ) x [G t ,P t ]

[0041] The historical feature extraction module is used to extract the time information of the mobile robot and the crowd spatiotemporal graph, and the spatial information of the mobile robot and the crowd at each time is taken as input to obtain the time feature:

[0042] h t =LSTM(w t ,h t-1 ,c t-1 ;W6)

[0043] The pooling module is used to process the input of any length into a fixed length feature vector, and the mean pooling is used in the pooling module:

[0044] c t =Avgpooling(w t ,h t ;W7)

[0045] The planning module takes the crowd features output by the pooling module and the current state features of the unmanned vehicle as the input of the decision network, and calculates the value of each action.

[0046] v t =φ4(r t ,c t ;W4)

[0047] Step 3: Train the deep reinforcement learning network built in step 2;

[0048] (1) Create a simulation environment with a size of 10*10, which contains a mobile robot and an arbitrary number of pedestrians, and the mobile robot and the pedestrians are represented by circles. To simulate the movement of pedestrians in a real scene, the invention randomly initializes the starting point and the ending point of the pedestrians in the simulation environment, and when the pedestrians reach the ending point, they will be stationary at the ending point. The ORCA algorithm is used to control the movement of the pedestrians from the starting point to the ending point. At the same time, the starting point and the ending point of the mobile robot are randomly initialized, and when the mobile robot reaches the ending point, the current simulation ends.

[0049] (2) Initialize network parameters and experience pool.

[0050] (3) Control the mobile robot to move from the starting point to the ending point in the simulation environment by using the ORCA algorithm, collect the quadruple (s t ,a t ,r t ,s t+1 ) data constructed in step one, and store them in the experience pool.

[0051] (4) When the experience pool is filled, update the network parameters by using the collected (s t ,a t ,r t ,s t+1 ) data and the imitation learning method, and complete the pre-training.

[0052] (5) Use the deep reinforcement learning network to replace ORCA as the decision model, control the mobile robot to continuously interact with the simulation environment, obtain (s t ,a t ,r t ,s t+1 ) data, update the experience pool, and adjust the network parameters by using the back propagation algorithm until the model converges.

[0053] Advantages

[0054] (1) The method of the present application models the spatiotemporal characteristics of the crowd, providing a more accurate model for the unmanned vehicle to perceive the actual crowd environment. Under the same experimental conditions, the algorithm of the present application can enable the unmanned vehicle to effectively avoid the crowd while reaching the target position faster.

[0055] (2) The method of the present application has great significance for improving the social navigation effect of the unmanned vehicle in the crowd environment.

[0056] (3) The method of the present application can meet the requirements of people for the performance of the unmanned vehicle navigation algorithm, enabling the unmanned vehicle to quickly reach the target point, and also enabling the unmanned vehicle to interact more friendly with people during the journey, especially in the intelligent manufacturing scene, the unmanned vehicle can realize efficient navigation in the presence of people.

[0057] (4) The method of the present invention first models the social navigation problem and obtains spatiotemporal state information such as the state of the unmanned vehicle and the state of pedestrians. Secondly, a reinforcement learning model based on DQN is constructed. The model includes four modules: an interaction module, a historical feature extraction module, a pooling module, and a planning module. The interaction module is used to process the interaction features between the mobile robot and the pedestrians, the historical feature extraction module is used to extract the time dimension features between the mobile robot and the crowd, the pooling module is used to convert any number of pedestrian inputs into a fixed-length feature vector output, and the planning module plans a feasible path based on the state of the unmanned vehicle and the extracted crowd features. Finally, the model is pre-trained by imitating the experience of experts to accelerate the model convergence speed.

[0058] (5) The present invention models crowd characteristics more accurately, which is of great significance to the social navigation of mobile robots in crowd environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 The present invention provides a method for constructing a spatiotemporal graph of mobile robots and people;

[0060] Figure 2 It is a DQN-based reinforcement learning model framework provided by the present invention. DETAILED DESCRIPTION

[0061] The present invention will be described in detail below in conjunction with the accompanying drawings. The following drawings are for illustration only and are not intended to limit the scope of the present invention.

[0062] This paper provides a reinforcement learning-based social navigation method for mobile robots, utilizing a deep reinforcement learning model based on DQN to address practical social navigation issues. By segmenting the deep reinforcement learning network framework into four modules: an interaction module, a historical feature extraction module, a pooling module, and a planning module, the method accurately extracts environmental features, achieving end-to-end control from multi-source features to decision output.

[0063] To achieve social navigation of mobile robots in a crowd, we first need to obtain the position, speed and other status information of pedestrians in the mobile robot's operating environment, and convert it into a polar coordinate system with the mobile robot as the origin and the mobile robot's target direction as the positive direction, and then construct a spatiotemporal graph of the relationship between the mobile robot and the crowd. The nodes of the spatiotemporal graph represent the mobile robot and each pedestrian, and the edges of the spatiotemporal graph represent the attention coefficient between two adjacent nodes. Figure 1 As shown in the figure, assume that there are four pedestrians in the crowd, represented by nodes h1, h2, h3, and h4 respectively, and the mobile robot is represented by node x. The edge starting from node x in the space-time graph is respectively represent the attention coefficient of the mobile robot to the four pedestrians at the first moment, the nodes between the mobile robot and the crowd are unidirectional communication, that is, the robot can obtain the information of the pedestrians, and the pedestrians cannot obtain the information of the robot. The edges starting from the pedestrian node in the space-time graph represent the attention coefficient of the pedestrian i to the pedestrian j at the first moment, and the nodes between the pedestrians are fully connected, that is, the pedestrians can obtain the state information of each other. At t moment, the network model interaction module can extract the feature w t representing the spatial information in the space-time graph t , the network model history feature extraction module can extract the feature vector h t containing time dimension information in the space-time graph t , [w t , h t ] represents all features of the current space-time graph.

[0064] Figure 2 is the network framework of the reinforcement learning model. As shown in Figure 2 , the reinforcement learning model is divided into four modules: interaction module, history feature processing module, pooling module and planning module.

[0065] Interaction module: according to the constructed social navigation problem model, the interaction between the unmanned vehicle and the crowd is processed. Since the state dimension of the mobile robot is different from that of the pedestrian, the feature information needs to be extracted by using a fully connected layer first. First, the state feature of the mobile robot is extracted by using a multi-layer fully connected layer:

[0066] r t =φ1(S t ;W1)

[0067] where φ1 is a fully connected layer, and W1 is the weight corresponding to the fully connected layer.

[0068] Secondly, the feature of each person-robot pair (mobile robot and single pedestrian) is also extracted by using a multi-layer perceptron:

[0069] P t =φ2(J t ;W2)

[0070] The attention matrix of the crowd is calculated by using a graph attention mechanism, and the crowd feature is extracted by using a graph convolutional neural network:

[0071] A t =Attention1(P t ;W3)

[0072] G t =GCN(A t ,P t ;W4)

[0073] wherein Attention1 is an attention network.

[0074] The features extracted by the graph neural network are combined with the human-robot pair (mobile robot and single pedestrian) features as the input of the self-attention network, and the attention value of the mobile robot to each pedestrian is obtained:

[0075] scores t =Attention2(G t ,P t ;W5)

[0076] wherein Attention2 is a self-attention network.

[0077] Considering that the robot cannot completely observe the state information of all pedestrians in the crowd, a mask mask t is used to weight the attention value, representing the spatial features of the robot and the crowd at the current time:

[0078] w t =softmax(scores×mask t )×[G t ,P t ]

[0079] History feature extraction module: The history feature extraction module is used to extract the temporal information of the mobile robot and the crowd spatiotemporal graph, and the spatial information of the mobile robot and the crowd at each time is taken as the input to obtain the time feature:

[0080] h t =LSTM(w t ,h t-1 ,c t-1 ;W6)

[0081] Pooling module: Since the number of pedestrians in the crowd is uncertain, it cannot be directly input into the planning module for training, and a pooling module is needed to process the input of any length into a fixed length feature vector. The pooling module uses mean pooling:

[0082] c t =Avgpooling(w t ,h t ;W7)

[0083] Planning module: The crowd features output by the pooling module and the current state features of the unmanned vehicle are taken as the input of the decision network, and the value of each action is calculated.

[0084] v t =φ4(r t ,c t ;W4)

[0085] On the basis of the model being completed, the model is pre-trained. Firstly, the ORCA algorithm is used to accumulate expert knowledge, obtain expert experience, fill the experience pool, and pre-train the model through imitation learning on the basis. The network parameters obtained by pre-training are used as the initialization parameters of the model, and reinforcement learning training is performed until the model converges.

[0086] In actual use, the acquired crowd state and mobile robot state are pre-processed and input into the trained deep reinforcement learning model, and the action corresponding to the maximum value in the model output is selected and executed, so that the mobile robot can quickly and efficiently socially navigate in the crowd environment.

[0087] Therefore, the application provides a mobile robot social navigation method based on reinforcement learning, which accurately extracts the space-time information of the crowd in the surrounding environment by constructing the space-time graph of the mobile robot and the crowd on the basis of detecting the position and speed information of the pedestrians. And the trained reinforcement learning model is used to select the action that maximizes the model output, ensuring that the mobile robot quickly and efficiently reaches the target position.

[0088] To sum up, the above is only a specific embodiment of the application and is not used to limit the protection scope of the application. Any modification or equivalent replacement, improvement, etc. made within the spirit and principles of the application shall be included in the protection scope of the application.

Claims

1. A method for social navigation of a mobile robot based on reinforcement learning, the method comprising: The steps of the method comprise: Step 1, constructing a model of a social navigation problem; Step 2, constructing a deep reinforcement learning network framework according to the model of the social navigation problem constructed in step 1; Step 3, training the deep reinforcement learning network framework constructed in step 2; Step 4, taking the deep reinforcement learning network framework trained in step 3 as an optimal strategy for actual navigation to complete social navigation of the mobile robot; In step 2, the deep reinforcement learning network framework comprises an interaction module, a historical feature extraction module, a pooling module and a planning module; The interaction module refers to processing the interaction between the unmanned vehicle and the crowd according to the constructed model of the social navigation problem, and extracting the state features of the mobile robot by using a multi-layer fully connected layer: r t = φ1(S t ; W1) Wherein, φ1 is a fully connected layer, and W1 is a weight corresponding to the fully connected layer; The features of each person-robot pair are extracted by using a multi-layer perception machine: P t = φ2(J t ; W2) The attention matrix of the crowd is calculated by using a graph attention mechanism, and the crowd features are extracted by using a graph convolutional neural network: A t = Attention1(P t ; W3) G t = GCN(A t , P t ; W4) Wherein, Attention1 is an attention network; The output features of the graph convolutional neural network and the person-robot pair features are jointly taken as the input of a self-attention network to obtain the attention values of the mobile robot to each pedestrian: scores t = Attention2(G t , P t ; W5) Wherein, Attention2 is a self-attention network; Utilizing a mask mask t The attention value is weighted to represent the spatial characteristics of the robot and the crowd at the current moment: w t = softmax(scores x mask t ) x [G t , P t ]; wherein mask t is 1 when the pedestrian state is observable at time t, otherwise mask t is 0; The historical feature extraction module is used to extract the time information in the space-time graph of the mobile robot and the crowd, and the spatial information of the mobile robot and the crowd at each time is taken as the input to obtain the time features: h t = LSTM(w t ,h t-1 ,c t-1 ; W6) The pooling module is used to process the input of any length into a fixed-length feature vector, and mean pooling is adopted: c t = Avgpooling(w t ,h t ; W7) The planning module is used to take the crowd features output by the pooling module and the current state features of the unmanned vehicle as the input of a decision network to calculate the values of each action: v t = φ4(r t ,c t ; W4).

2. The mobile robot social navigation method based on reinforcement learning according to claim 1, characterized in that: The step 1, the model of constructing social navigation problem is: t ,a t ,r t ,s t+1 ) four tuples; where s t is the state at time t, a t is the action at time t, s t+1 is the state at time t+1, r t is the reward at time t.

3. The mobile robot social navigation method based on reinforcement learning according to claim 2, characterized in that: The state s at time t is given by: t is given by: where the state includes the mobile robot state and the crowd state, i = 1, 2, …, N, denotes the pedestrian number, and N is the number of pedestrians in the environment at time t, denotes the joint state of the robot and each pedestrian, S t = [d gt , v pref , v xt , v yt , r] denotes the state of the mobile robot at time t, where d gt denotes the distance of the mobile robot to the target point, v pref denotes the reference speed of the mobile robot, v xt , v yt denotes the velocity of the mobile robot in x, y direction at time t, and r denotes the radius of the mobile robot, denotes the observable state of the i-th pedestrian at time t, where d t i denotes the relative distance of the i-th pedestrian to the mobile robot at time t, denotes the x, y coordinates of the i-th pedestrian relative to the mobile robot at time t, denotes the velocity of the i-th pedestrian relative to the mobile robot at time t, and r i denotes the assumed radius of the i-th pedestrian, and r + r i denotes the sum of the radius of the mobile robot and the assumed radius of the i-th pedestrian.

4. The mobile robot social navigation method based on reinforcement learning according to claim 3, characterized in that: The t moment action a t Is: a t = [v xt , v yt ].

5. The mobile robot social navigation method based on reinforcement learning according to claim 3 or 4, characterized in that: at time t in state s t take action a at time t t the reward r obtained t is: r t = R(s t ,a t ) wherein d t represents the minimum distance between the vehicle and the pedestrian in the time [t-Δt, t], Δt represents the minimum time step of decision, P t c represents the position of the vehicle at time t, P g represents the target point of the vehicle.

6. The mobile robot social navigation method based on reinforcement learning according to claim 3 or 4, characterized in that: The optimal strategy of the social navigation problem model in step 1 is π * :s t →a t The optimal strategy is obtained by maximizing the following expected return; where P(s t ,a t ,s t+Δt ) is the state transition probability from t to t+Δt, V * is the optimal value function, γ is a decay constant, and T is the termination time.

7. The mobile robot social navigation method based on reinforcement learning according to claim 1, characterized in that: In step 3, the method for training the deep reinforcement learning network framework is: (1) creating a simulation environment, wherein the mobile robot and any number of pedestrians are included, the mobile robot and the pedestrians are represented by circles, the starting point and the endpoint of the pedestrians are randomly initialized in the simulation environment, the pedestrians are kept stationary at the endpoint when reaching the endpoint, the ORCA algorithm is used to control the movement of the pedestrians from the starting point to the endpoint, and the starting point and the endpoint of the mobile robot are randomly initialized, and the current simulation ends when the mobile robot reaches the endpoint; (2) initializing network parameters and an experience pool; (3) Using ORCA algorithm to control the mobile robot to move from the starting point to the ending point in the simulation environment, collect the quadruple (s t ,a t ,r t ,s t+1 ) data constructed in step one, and store them into the experience pool; (4) When the experience pool is full, the collected (s t ,a t ,r t ,s t+1 ) data and the imitation learning method are used to update the network parameters, and pre-training is completed; (5) Using a deep reinforcement learning network instead of ORCA as a decision model, control the mobile robot to constantly interact with the simulation environment, get (s t ,a t ,r t ,s t+1 ) data, update the experience pool, and adjust the network parameters using the back propagation algorithm until the model converges.

Images