A parking management method

Abstract

The present application discloses a parking management method, which relates to the field of parking management technology, including: collecting parking environment data, the parking environment data including parking lot map and parking space status information; preprocessing the collected data set; establishing a parking environment model based on the preprocessed data set by using a reinforcement learning method; and obtaining the optimal parking path based on the parking environment model. In view of the problem of low parking path planning accuracy in the prior art, the present application establishes a parking environment model, and uses a reinforcement learning method to achieve autonomous learning and optimization of a complex and dynamically changing parking lot environment. The parking environment model is constructed using a Q learning algorithm. Through balanced exploration and utilization, the optimal path between every two parking spaces in the parking lot can be obtained to achieve global optimization. The A* algorithm is used for path search. Compared with a simple search algorithm, it can quickly and efficiently search for a driving route with the lowest driving cost, thereby improving the accuracy of parking path planning.

Description

Technical Field

[0001] The present application relates to the technical field of parking management, and in particular to a parking management method. Background Art

[0002] With the acceleration of urbanization and the continuous increase in the number of vehicles, parking has become a major problem in urban traffic management. In traditional parking lot management, parking path planning often relies on manual experience or simple rules, resulting in low parking efficiency, severe congestion, and difficulty in adapting to complex parking lot environments. In addition, traditional methods cannot make full use of data for real-time adjustment and optimization, so the accuracy of parking path planning is low and it is difficult to meet the growing parking demand.

[0003] In existing parking management systems, simple path search algorithms, such as those based on the shortest path or greedy algorithms, are often used, which often perform poorly in the face of complex parking environments. Due to the complex parking layout, high vehicle mobility, and uncertainty in congestion, simple path search algorithms can easily lead to the generation of non-optimal paths, resulting in low accuracy in parking path planning.

[0004] In the related art, for example, Chinese patent document CN116665481A provides a smart parking management method with linked video positioning, including: S1, a license plate recognition device recognizes the license plate number of a vehicle parked in a parking space and associates and binds it with the parking space number of the parking space, and generates binding information and pushes it to the smart parking management system; S2, the smart parking management system obtains the first channel information that the owner intends to enter after leaving the car and binds it to the owner's ID based on the parking space number carried in the binding information and the binding relationship between the parking space number and the corresponding channel established in advance; S3, when the car needs to be picked up, the owner arrives at any entrance to the underground garage and passes through the smart parking management system. The terminal can send a car search instruction to the smart parking management system; S4, the smart parking management system locates the entrance where the car owner is currently located according to the car search instruction, so as to obtain the second channel information of each second channel that may be passed after entering the underground garage from the entrance and bind it with the car owner ID; S5, the smart parking management system plans and updates the optimal path for the car owner to the target parking space in real time according to the second channel information and the first channel information associated with the car owner ID obtained in step S4 and step S2, and links the video positioning, and pushes the planned optimal path to the car owner in real time through the car search APP. However, the path planning of this scheme is based on the pre-established static binding relationship between the parking space number and the corresponding channel. This static planning method cannot reflect the changes in the congestion status of the channel in real time. When a channel is congested, it will still be planned to pass through this channel, and the congested area cannot be avoided, resulting in low path planning accuracy. Summary of the invention

[0005] 1. Technical issues to be solved

[0006] In response to the problem of low parking path planning accuracy in the prior art, the present application provides a parking management method, which constructs a parking environment model and uses a Q learning algorithm to obtain the optimal parking path, thereby achieving autonomous learning and optimization of complex parking environments and improving the parking path planning accuracy.

[0007] 2. Technical solution

[0008] The purpose of this application is achieved through the following technical solutions.

[0009] The embodiments of the present specification provide a parking management method, including: collecting parking environment data, the parking environment data including a parking lot map and parking space status information; preprocessing the collected data set; establishing a parking environment model based on the preprocessed data set using a reinforcement learning method; and obtaining an optimal parking path based on the parking environment model.

[0010] Specifically, a radio frequency RFID tag is set at each parking space, and an RFID reader is set at the entrance and main channel of the parking lot. A camera is installed in the parking lot to capture images of the parking area. When a vehicle enters the parking lot or parking space, the RFID reader identifies the tag and obtains the vehicle information. The parking space image captured by the camera is input into the image processing algorithm to identify the occupancy status of each parking space. The parking lot map is made of information such as the internal structure and entrance and exit locations of the parking lot. The vehicle information identified by the RFID is associated with the parking space status information obtained by image processing. The parking space status information is associated with the map location information to form a parking space status data set. The parking space status data set is regularly transmitted to the back-end server through the network.

[0011] Specifically, the raw data obtained from RFID tag recognition and camera image acquisition is subjected to denoising processing such as filtering and smoothing to improve data quality. Outliers and erroneous data in the data are detected and deleted from the data set. The data from RFID and image algorithms are fused to form a fused data set. For incomplete data, interpolation, nearest neighbor method and other methods are used to complete it. Feature data such as time, location, and status are extracted from the fused parking space status data. The data set of the parking lot map is annotated, such as the entrance and exit locations, main channels, etc. The processed data set is organized into a structured table form for easy use of the model. The data set is standardized to improve data consistency. According to the model input requirements, useful data is extracted to construct the model data set.

[0012] Among them, the reinforcement learning method is a machine learning method that can evaluate the user's behavior and guide the user to learn the optimal strategy through feedback information. In the technical solution of this application, the reinforcement learning method used is specifically the Q learning algorithm, and its specific application in the solution is as follows: the reinforcement learning subject is an intelligent user, which corresponds to the vehicle that needs to find the optimal parking path in the parking lot in this solution. The parking lot environment state is the environment in reinforcement learning, including the state of each parking space. The user's behavior space is the behavior space in reinforcement learning, which corresponds to the behavior of the vehicle moving from the current parking space to the target parking space in this solution. The reward function is a mechanism for evaluating the behavior in the behavior space. In this solution, a reward function that considers distance and congestion is designed. Through the value iteration of the Q learning algorithm, the optimal strategy from any parking space to other parking spaces in the parking environment can be obtained. Finally, a strategy model that can guide the vehicle to find the optimal path in the parking lot is obtained. Use the Q learning algorithm as a reinforcement learning method to establish a parking environment model, and learn through the evaluation and feedback of user behavior to obtain the optimal strategy for parking path planning.

[0013] Specifically, the position coordinates of the driver's current parking space and the target parking space are input into the parking environment model. Based on the map information stored in the parking environment model, the accessible parking spaces between the starting and ending positions are queried. The straight path between the starting and ending positions is used as the initial parking path. Based on the transition probability and reward function of the environment model, the expected driving cost of the initial path is calculated. Using the A* algorithm, with the goal of minimizing the expected cost, the optimal parking path with the minimum driving cost between the starting and ending positions is searched. The environment model determines the optimal parking path to be executed in the end based on the current parking space status. The optimal parking path output by the model is visualized and displayed to the driver. The driver or the autonomous driving system actually drives according to the optimal path.

[0014] Furthermore, a Q-learning algorithm is used to construct a parking environment model; the state space S of the Q-learning algorithm is set as the state of each parking space in the parking lot, the behavior space A is the action of driving from the current parking space to the target parking space, the reward function R is the negative cost of driving from the current parking space to the target parking space, and the strategy π is to select the action strategy with the smallest negative cost when executing the action; in the state space S, the traversal action strategy π is set to obtain the optimal strategy π in a way of maximizing the return in the long term; and the parking environment model is constructed based on the obtained optimal strategy π.

[0015] Among them, Q learning algorithm: a model-free reinforcement learning algorithm that obtains the optimal strategy by learning the action value function. State space S: the set of environmental states in reinforcement learning, which corresponds to the state of each parking space in the parking lot in this solution. Behavior space A: the set of behaviors that the agent can perform in reinforcement learning, which corresponds to the behavior of driving from the current parking space to the target parking space in this solution. Reward function R, a function that evaluates behavior. In this solution, a negative reward function that considers the driving distance and congestion to the target parking space is designed. Strategy π, a strategy that guides the agent to select behaviors in the behavior space, in this solution, is to select the driving behavior strategy with the smallest negative cost. In this application, the Q learning algorithm is used to obtain the optimal strategy π in the parking lot environment through trial and error learning. The state of each parking space in the parking lot is set to state space S, which contains environmental information. The driving behavior between different parking spaces is defined as behavior space A. The reward function R evaluates the behavior in behavior space A and calculates the negative cost. Use the optimal strategy π to guide how to select the behavior with the smallest negative cost in behavior space A. A strategy model that can obtain the optimal parking path in a complex parking lot environment is obtained.

[0016] Specifically, according to the parking lot layout and the number of parking spaces, each parking space is set to a state in the environmental state space S. For example, the driving behavior from the current parking space to the surrounding 8 adjacent parking spaces is defined as the behavior space A. R is the negative reward for driving from the current parking space i to the target parking space j, which includes the distance penalty and congestion penalty for driving to the target parking space. Construct a table to store the Q values ​​corresponding to the state s and behavior a, and initialize all Q(s,a) values. Use the ε greedy strategy to traverse all states in S, explore the behavior space in A, and record the immediate rewards corresponding to the behavior. According to the Q learning formula, Q(s,a) is continuously iterated and updated to obtain the final Q* value function. For each state s, select the behavior a with the largest Q(s,a) value to form the optimal strategy π. For different start and end states, the model outputs the optimal behavior in the behavior space according to π.

[0017] Specifically, the initial strategy π is set to a random exploration strategy, that is, a behavior a is randomly selected in each state s. For each execution of behavior a in each state s, the immediate reward r is calculated according to the reward function R, and Q(s,a) is iteratively updated. Traverse each state s and behavior a under strategy π, and evaluate the expected cumulative return of the Q(s,a) value corresponding to behavior a. For state-behavior pairs with small Q(s,a) values, adjust strategy π and select other behaviors to improve long-term returns. Continuously iterate strategy evaluation and improvement until strategy π converges and the return can no longer be improved. When strategy π is stable, output π at this time as the optimal strategy. Verify that the behavior selection of the optimal strategy π in the state space S can obtain the maximum long-term return. Save the optimal strategy π that maximizes the cumulative return and use it for the optimal path selection of the vehicle.

[0018] Specifically, the design includes modules such as state space, behavior space, reward function and policy function. Import the structural map of the parking lot into the model to construct the state space. Engineer the optimal strategy π learned into a policy function. Design a reward function based on the driving distance and congestion conditions, and calculate the return of behavior execution. Based on the imported map, simulate the operation of the vehicle in the state space. The policy function determines the optimal behavior space behavior to be executed based on the current state. Record the state transition path generated by the strategy during the simulation. Evaluate the duration, distance, comfort and other indicators of the simulation path to verify the model effect. Compare simulations under different parameters, adjust the strategy and reward function, and optimize the model. Obtain a parking environment reinforcement learning model with accuracy that meets the requirements.

[0019] Furthermore, the action path planning from the current parking space to the target parking space in the behavior space A adopts the A* search algorithm; the location layout of the parking spaces in the parking lot is obtained to construct a two-dimensional grid map; according to the coordinates of the current parking space and the target parking space in the constructed two-dimensional grid map, the heuristic function is set to the straight-line distance between the two points; an open list is set to store the extended grids; the grid where the current parking space is located is added to the open list as the starting point; the surrounding grids within the preset range of the current grid are traversed, and the driving cost from the current grid to the surrounding grids is calculated, and the driving cost is calculated by the driving distance and whether it passes through the congested area; the surrounding grids with the smallest driving cost are selected to be added to the open list; it is determined whether the target parking space is included in the open list, if not, repeat the previous step; according to the expansion order of each grid in the open list, the center points of the grids are connected in the expansion order until they are connected to the center point of the target grid, and the formed broken line path is used as the optimal driving path.

[0020] Among them, the A* search algorithm is an informed search algorithm, which uses the cost function f(n)=g(n)+h(n) to find the shortest path from the initial state to the target state in the solution space. In this application, the state space of the A* algorithm corresponds to each parking space in the parking lot. The initial state is the current parking space, and the target state is the target parking space to be reached. The cost function g(n) is the actual path cost from the initial state to the current state n. h(n) is the estimated cost from the current state n to the target state. f(n) is the total estimated cost to reach the current state n. By comparing the f(n) values ​​of adjacent states, the state with the smallest f value is selected for expansion to search for the minimum cost path from the initial state to the target state. Finally, the optimal driving path from the current parking space to the target parking space is obtained.

[0021] Among them, the two-dimensional grid map is a map that abstracts the two-dimensional space into a grid representation of rows and columns. In this application, the parking lot floor plan is abstracted into an M×N two-dimensional matrix. Each parking spot corresponds to a square in the matrix. The value in the square indicates whether the parking spot is free. The squares corresponding to the entrance and exit of the parking lot are marked with specific numbers. Aisles and roadways are represented by drivable squares. Construct a two-dimensional grid matrix containing all the squares and values ​​in the field. In the matrix, the drivable squares are connected to represent a drivable path. Find the optimal path in the two-dimensional grid matrix through a search algorithm. The optimal path corresponds to the vehicle driving route in the venue. Finally, the optimal path searched is visualized.

[0022] The heuristic function is a function used in the A* algorithm to estimate the distance from the current state to the target state. It provides an estimated distance to the target. In this application, the coordinates of the current parking space and the target parking space in the two-dimensional grid map are (x 1 ,y 1 ), (x 2 ,y 2 ). Mark the two points on the map and obtain their coordinate information. Calculate the straight-line distance between the two points This straight-line distance is the estimated distance from the current state to the target state. Set this straight-line distance as the heuristic function h(n) of the A* algorithm. During the A* search, h(n) will guide the search direction toward the target state. This changes the search method to an informed search, which can find the optimal path more quickly. Finally, the parking path with the shortest distance is obtained. That is, the optimal path from the current parking space to the target parking space. This solution uses the straight-line distance between two points as a heuristic function, which can effectively guide the A* algorithm to search for the optimal parking path.

[0023] Among them, open list: in the path search algorithm, it is used to store the list of nodes to be expanded. Extended grid: the new surrounding grid searched from the current grid. In this application, the open list is initialized, including the starting grid. Take out the grid A with the smallest cost f value from the open list. For example, check the adjacent grids in the four directions of up, down, left and right of A. Calculate the cost g of expanding from the current grid to each adjacent grid. Generate new grid data, including coordinates, g, parent node and other information. Add these newly expanded adjacent squares to the open list. Repeat the above steps until the target square is expanded. The open list stores all new squares expanded during the traversal. Finally, the minimum cost path can be obtained by backtracking through the parent node. Return the optimal path. The open list stores the information of the grids to be expanded, and the newly expanded grids will also be added to it for continuous search of the optimal path.

[0024] Among them, in this application, the surrounding squares refer to the eight squares in the up, down, left, right, and diagonal directions that are adjacent to the current square on the two-dimensional grid map. During initialization, the square where the current parking space is located is put into the open list as the starting square. Take out a current square from the open list and get its coordinate position. According to the coordinates, its eight surrounding squares in the up, down, left, right, and diagonal directions can be determined. Calculate the driving cost of expanding the current square to each surrounding square. The cost is determined by the driving distance and whether the surrounding squares are in a congested area. Generate the surrounding squares and the calculated cost information as new grid data. Add the new square to the open list. Repeat the above process to traverse all squares. Finally, search for the optimal path with the minimum cost.

[0025] Among them, the exercise cost: the driving cost of expanding from the current grid to the surrounding grids. Driving distance: the Manhattan distance from the current grid to the surrounding grids, or the actual distance of the roadway. Congested area: the area where vehicles are congested in the parking lot. In this application, the Manhattan distance from the current grid to the surrounding grid is calculated as the driving distance. Determine whether the surrounding grids are in the congested area. If the surrounding grids are in the congested area, add additional congestion penalty costs. Exercise cost = driving distance cost + congestion penalty cost. The lower the exercise cost, the higher the expansion priority. Expand the grids in order from low to high according to the exercise cost. Repeat the calculation of the exercise cost to search for the global minimum cost path. That is, the optimal path from the current parking space to the target parking space. Avoid congested areas and achieve fast parking in and out. This scheme evaluates the priority of expanding new grids through the exercise cost to obtain the optimal parking path with the shortest driving distance and avoiding congestion.

[0026] Specifically, calculate the exercise cost of expanding the current square to each surrounding square. Compare the exercise cost value of each surrounding square. Select a surrounding square A with the smallest exercise cost. Generate data for square A, including coordinates, exercise cost g, parent node and other information. Add square A to the open list. Determine whether the open list contains the square of the target parking space. If the open list does not contain the target square, repeat the above steps and continue to expand the new square: take out square B with the smallest cost f from the open list. Calculate the exercise cost of the surrounding squares of square B. Select the minimum cost square to add to the open list. Loop until the target square is included in the open list. Finally, the optimal path from the current parking space to the target parking space is obtained through the backtracking path. By iteratively expanding the minimum cost square, the optimal parking path in the parking lot can be quickly searched.

[0027] Specifically, take the square A with the smallest cost f from the open list and expand it. Record the coordinates (x, y) of square A. Repeat the above process for the surrounding squares of A, and record the coordinates of each square expanded. The final target square will also be expanded and its coordinates recorded. At this time, the open list stores the coordinate information of all the expanded squares. According to the expansion order of the squares, connect their coordinate points: (x1 ,y 1 )→(x 2 ,y 2 )→......→(x n ,y n ), the polyline path obtained by connecting the coordinate points is the optimal path. Because it expands in the order of minimum cost, the overall cost is the lowest. Draw the polyline path and visualize it on the parking map. The polyline path is the optimal driving route from the current parking space to the target parking space.

[0028] Furthermore, the exercise cost is calculated based on the driving distance and whether the vehicle passes through a congested area, including: in a constructed two-dimensional grid map, according to the Euclidean distance between the center points of each two grids, obtaining the driving distance data from the current grid to the surrounding grids; obtaining the entrance and exit locations and the main channel location of the parking lot, and using a regional growing algorithm to obtain a subset of congested grids as the congested area; establishing a corresponding table between driving distance and distance cost, inputting the calculated driving distance data, and obtaining the distance cost according to the corresponding table; constructing a congestion cost table, and presetting the congestion cost of the grids in the congested area; if the surrounding grids belong to the subset of congested grids, querying the constructed congestion cost table to obtain the congestion cost; and calculating the exercise cost based on the calculated distance cost and congestion cost.

[0029] The driving distance data refers to the actual driving distance from the current grid to the surrounding grids. In this application, the coordinates (x 1 ,y 1 ), (x 2 ,y 2 ). Calculate the Euclidean distance between two coordinate points: This distance is the driving distance from the current square to the surrounding squares. The driving distance is calculated for each of the 8 surrounding squares. The distance data corresponds to the square number to form a driving distance data set. The distance data will be used when calculating the exercise cost of expanding the current square to the surrounding squares. The exercise cost consists of two parts: driving distance and additional penalty. The algorithm will give priority to expanding squares with short distances and low costs. Repeatedly calculate the distance data set to provide a distance basis for the expansion of each square. Finally, find the optimal parking path with the shortest overall distance. This application uses Euclidean distance to calculate the driving distance data set, which is applied to the calculation of the square expansion exercise cost to search for the optimal parking path with the shortest distance.

[0030] Among them, the region growing algorithm is an image segmentation algorithm based on pixel clustering. It starts with a seed point and continuously aggregates adjacent pixels that meet the conditions into the same class, so that the region continues to expand and grow until different regions in the entire image are segmented. In this application, the entrance and exit and main channel grids are set as seed grids. Starting from the seed grid, check whether its surrounding grids are accessible and have a large traffic volume. Cluster the surrounding grids that meet the conditions into the current area. Repeat the above steps for the newly added grids to continuously expand the area. When there are no accessible surrounding grids with a large traffic volume, the current region growth ends. Finally, the high traffic congestion area near the entrance and exit and the main channel can be segmented. The congested area grid set is added as a penalty factor to the driving cost calculation. The path planning algorithm can avoid the congested area and obtain a smooth parking path. This solution uses the region growing algorithm to effectively extract the congested area of ​​the parking lot to guide the path optimization.

[0031] Among them, entrance and exit positions: the position coordinates of the vehicle entrance and exit of the parking lot. Main channel position: the position coordinates of the main carriageway in the parking lot. In this application, the positions of the entrances and exits and the main channel are marked on the CAD drawing of the parking lot. The grid coordinates corresponding to these positions are obtained as seed points. The entrance and exit and main channel grids are used as the initial seeds of the regional growing algorithm. Starting from them, the accessibility analysis and traffic flow evaluation are used to determine whether the surrounding grids are to be added. Finally, a set of high traffic congestion grids including the entrances and exits, the main channel and the surrounding areas is formed. When planning the path, an additional penalty cost is added to the path that passes through the congested grid set. The path planning algorithm is made to tend to bypass the congested grid set and obtain a parking path with less traffic. It provides users with a faster and more comfortable parking experience. This solution sets the positions of the entrances and exits and the main channel as the initial points for determining the congested area to guide the subsequent parking path optimization.

[0032] Specifically, a distance-cost mapping table is set, which contains: driving distance, corresponding distance cost, and the table is initialized, and a reasonable correspondence between distance and cost is set. For example: 0 to 5 meters, cost 10; 5 to 10 meters, cost 20; 10 to 15 meters, cost 30; obtain the driving distance data set from the current grid to the surrounding grids. Map each distance data to the distance-cost table. Look up the table to obtain the distance cost corresponding to each distance. The larger the distance, the longer the time required, and the larger the corresponding cost value. The obtained distance costs form a cost data set. When calculating the total cost of each grid expansion: total cost = distance cost + congestion penalty cost, repeatedly query the distance-cost table to obtain the distance cost in each direction. Expand the grid with the smallest total cost and search for the minimum cost path. Through the conversion of the distance-cost table, the distance information is mapped to the cost value required for algorithm optimization, and finally the optimal parking path with the minimum total cost is obtained.

[0033] Specifically, create a data table to store congestion costs. The table contains two columns: grid ID and congestion cost. Obtain a set of grids in the congested area through the region growing algorithm. Assign a unique ID to each grid in the congested area. Set the congestion cost of the grids in the congested area according to the actual situation, for example, 50. Write the ID of each grid in the congested area and the corresponding congestion cost into the table. For grids in non-congested areas, their congestion cost is set to 0. When calculating the total cost, the congestion cost can be found in the congestion cost table through the grid ID. Total cost = distance cost + congestion cost. The congestion cost will penalize the path passing through the congested area. The algorithm will automatically avoid congested grids and select non-congested grids. Finally, the optimal parking path that avoids the congested area is obtained. By setting the congestion cost table in advance, the algorithm can be guided to effectively plan the optimal parking path that avoids congestion.

[0034] Specifically, determine whether the surrounding squares of the current square are in the predetermined subset of congested squares. If it is a congested square, look up its congestion cost in the constructed congestion cost table. Otherwise, its congestion cost is 0. The distance and distance cost from the current square to the surrounding square have been calculated. Add the distance cost to the congestion cost: driving cost = distance cost + congestion cost, repeat the above steps for all surrounding squares of the current square, and calculate their driving costs. Select the surrounding square with the smallest driving cost for expansion. The congestion cost will increase the driving cost of the congested square. The algorithm tends to avoid congested squares and expand non-congested squares. Finally, the optimal parking path that avoids congestion and has the smallest driving cost is searched. By introducing the congestion cost, the algorithm can automatically plan a parking path that bypasses the congested area.

[0035] Furthermore, a regional growing algorithm is used to obtain a subset of congested squares, including: obtaining the entrance and exit locations and the main channel locations of the parking lot, and taking the corresponding squares as the initial congested area; taking each square in the initial congested area as the growth starting point, and traversing the surrounding squares within a preset range; determining whether the surrounding squares are marked as congested squares, and if not, executing the next step; calculating the exercise cost from the growth starting point to the unmarked surrounding squares; if the calculated exercise cost is greater than a preset cost threshold, determining the unmarked surrounding squares as congested squares, marking and adding them to the initial congested area; repeating the above steps until the number of squares in the initial congested area no longer changes, and then outputting the final congested area.

[0036] Specifically, mark the locations of the entrances and exits and the main channel on the CAD layout of the parking lot. Obtain the grid coordinates corresponding to the entrances and exits and the main channel. Initialize a congested grid set and add the entrances and exits and the main channel grids to the set. In the subsequent region growing algorithm, these grids are used as seed grids to start growing. Finally, the complete congested area grid set is expanded through the region growing algorithm. However, the initial seed set already includes the entrances and exits and the main channel. The entrances and exits are the areas with the largest traffic flow, and congestion is bound to occur. The main channel carries the main traffic flow in the field and is also a high-incidence area of ​​congestion. Setting these two places as initial seeds can enable the region growing algorithm to obtain the entire congested area more quickly and accurately. There is no need to grow from scratch, which improves the efficiency of the algorithm. It also makes the final congested area more accurate and reasonable. It is conducive to the subsequent planning of the optimal parking path to avoid congestion.

[0037] Specifically, initialize the congested grid set, including the entrances and exits and the main channel grids. Traverse the congested grid set, and take out each grid in turn as the current growth grid. For example, calculate and obtain the surrounding grids in the four directions of up, down, left, and right of the current grid. Determine whether each surrounding grid is already in the congested grid set. If the surrounding grid is not in the set, proceed to step 6, otherwise skip. According to the preset rules, determine whether the surrounding grid should be added to the congested area: check the accessibility of the grid, analyze the real-time traffic flow in the area where the grid is located, and if the conditions are met, add it to the congested grid set. Traverse the four surrounding grids of the current grid. Repeat the above steps, growing with each congested grid as the starting point. Until the area no longer expands, the final complete congested area is obtained. By traversing the surrounding area starting from the congested grid, the overall congested area of ​​the parking lot can be generated quickly and efficiently to guide path planning.

[0038] Specifically, for example, obtain the four surrounding squares of the current square. For each surrounding square, determine whether it exists in the marked congested square set. If the surrounding square already exists in the congested square set, it means that it has been marked as a congested square, then skip the square and do not execute the subsequent steps. Otherwise, the surrounding square is not marked as a congested square, and the next step of expansion judgment is executed. Obtain real-time traffic flow data for each surrounding square. According to the traffic flow data, refer to the preset congested traffic flow threshold. If the traffic flow is greater than the threshold, mark the surrounding square as a congested square. Update the congested square set and add the new congested square to the set. Otherwise, the traffic flow is less than the threshold and retains its non-congested state. Repeat steps 1-9 until all surrounding squares are processed. Finally, the expanded complete congested square set is obtained. This can avoid repeated processing of marked congested squares and improve the efficiency of regional growth.

[0039] Specifically, for each square i in the initial congested area, obtain its unmarked surrounding squares j. Calculate the driving cost from square i to j: cost = driving distance from square i to j * driving cost per unit distance, and compare the cost with the preset cost threshold. If the cost is greater than the threshold, it is considered that square j is affected by the congestion of square i. Mark square j as a congested square and add it to the initial congested area. Repeatedly grow and expand all initial area squares. Until the number of squares in the initial area no longer changes after one growth traversal. At this time, the initial area expansion is completed, and the final complete congested area is output. The driving cost is used as an evaluation criterion for the degree of congestion, which can effectively determine whether the surrounding squares are congested. By iteratively growing until the area is stable, the comprehensive congested area of ​​the entire parking lot can be obtained, providing support for subsequent path planning.

[0040] Furthermore, the reward function R is the negative reward from the current parking space to the target parking space; the negative reward includes the negative reward of the driving distance and the negative reward of the exercise cost.

[0041] Among them, negative reward for driving distance: the driving distance from the current parking space to the target parking space, the longer the distance, the greater the negative reward value. Negative reward for driving cost: the driving cost required from the current parking space to the target parking space, the higher the cost, the greater the negative reward value. In this application, the Euclidean distance from the current parking space to the target parking space is calculated as the driving distance. According to the preset distance-negative reward function, it is converted into a distance negative reward. The sum of the driving costs required to travel each segment of the path is calculated as the driving cost. According to the preset cost-negative reward function, it is converted into a cost negative reward. The total reward R is the sum of the distance negative reward and the cost negative reward. The agent learns the optimal path with short distance and minimum cost by maximizing the reward. The distance negative reward guides the agent to learn to avoid detours. The cost negative reward guides the agent to learn to avoid congestion. By designing dual negative rewards for distance and cost, the agent can learn the optimal parking path in the parking lot.

[0042] Furthermore, strategy π is the strategy with the smallest negative reward when selecting an action; strategy π adopts the ε-greedy strategy; the ε-greedy strategy selects a random action with probability ε and selects the action with the smallest negative reward with probability 1-ε.

[0043] Specifically, the goal of defining strategy π is to select the action with the smallest negative reward. The ε-greedy strategy is used to implement π. The ε-greedy strategy includes two action choices: exploration and utilization. Random actions are selected for exploration with probability ε. Random actions can obtain new states and reward feedback. The action with the smallest negative reward is selected for utilization with probability 1-ε. The action with the smallest negative reward is selected from the learned knowledge to obtain the maximum return. The initial value of ε is large and then gradually decreases. The exploration action gradually decreases and the utilization action gradually increases. Finally, the optimal strategy π that only uses knowledge to select the best action is obtained. π can minimize the negative reward and obtain the optimal parking path. Through the design of the ε-greedy strategy, the agent can learn the optimal parking path selection strategy through trial and error.

[0044] Further, the value of ε is adjusted according to the preset ε decay function; it is determined whether the current ε value is greater than the preset threshold; if so, an action is randomly generated and executed; otherwise, the action with the smallest current negative reward is selected for execution;

[0045] Among them, the decay function refers to a function that gradually reduces the function value as the iteration process proceeds. In this application, the ε in the ε-greedy strategy controls the ratio of exploration and utilization. Initially, ε takes a larger value, such as 0.9, for more exploration. Define the decay function of ε, for example, ε=ε*0.9. After each action is executed, use the decay function to update ε. In this way, ε will gradually decrease with iteration, such as 0.9, 0.81, 0.729, etc. When ε becomes smaller, the exploration behavior will decrease and the utilization behavior will increase. Eventually, ε will be lower than the set threshold, such as 0.05, and it will switch to pure utilization mode. No more exploration, only choose the action with the largest reward. The decay function realizes a natural and smooth transition from exploration to utilization. Help the intelligent agent train a strategy to select the optimal action.

[0046] Specifically, define the initial value of ε, such as 0.9. Set the decay function of ε, such as ε=ε*0.9. After each action is executed, update ε according to the decay function. Set the threshold of ε, such as 0.05. Determine whether the current ε is greater than the threshold. If it is greater than the threshold, enter the exploration mode. Use the random number generator to uniformly randomly select an action. Execute the random action and observe the changes in state and reward. If ε is less than or equal to the threshold, enter the exploitation mode. Among all currently available actions, select the action with the smallest negative reward. Execute the action with the smallest negative reward to obtain the maximum reward. Repeat this process, ε gradually shrinks, the exploration behavior gradually decreases, and the exploitation behavior gradually increases. Finally, the optimal strategy of only selecting the action with the least negative reward is obtained. Through the threshold judgment of the ε greedy strategy, the self-balance of exploration and exploitation is achieved, and the intelligent agent can learn the optimal action strategy.

[0047] Furthermore, the Monte Carlo method is used to traverse the strategy π in the state space S. Among them, the Monte Carlo method is a type of algorithm that obtains the results of a random process through random simulation. In the present application, the state space S is extremely large and it is impossible to traverse and obtain the optimal strategy for each state. The state space S is sampled using the Monte Carlo random simulation method. Randomly select the starting state and perform actions according to the strategy π. Observe the state transition and reward feedback. Record the state transition sequence and the cumulative reward. Repeat the simulation multiple times to obtain the state and reward data. Calculate the expected reward for each optional action in each state. Update the optimal strategy corresponding to each state according to the expected reward. After multiple iterations, the strategy converges and learns the approximate optimal strategy π. π can guide the agent to choose the optimal action in each state. Through the strategy iteration of Monte Carlo random sampling, the optimal strategy π is effectively trained in a large state space.

[0048] Furthermore, the Monte Carlo method obtains the optimal strategy π of expectation maximization through multiple random simulation samplings. Specifically, the strategy π(s) of all states s is initialized as a random strategy. Perform Monte Carlo simulation: Randomly sample and execute π from the starting state to obtain the path. Calculate the cumulative reward R on the path. Traverse all states s on the path. Calculate the expected reward for each optional action a under s: Q(s,a) = R(s,a) / number of times a is selected, π(s) = max_aQ(s,a), that is, select the action with the largest Q value. Get the updated strategy π. Repeat the Monte Carlo simulation sampling multiple times and iteratively update π. After multiple iterations, π converges and estimates the optimal strategy. π can guide the selection of the optimal action with the largest expected reward in each state. By iteratively calculating the expected reward of the action through Monte Carlo sampling, the intelligent agent can learn the overall optimal strategy π.

[0049] 3. Beneficial effects

[0050] Compared with the prior art, the advantages of this application are:

[0051] (1) Establishing a parking environment model and using reinforcement learning methods can achieve autonomous learning and optimization of complex and dynamically changing parking environments without the need for human intervention, making parking management more intelligent;

[0052] (2) The parking environment model is constructed using the Q-learning algorithm. By balancing exploration and utilization, the optimal path between every two parking spaces in the parking lot can be obtained to achieve global optimization.

[0053] (3) Using the A* algorithm for path search can quickly and efficiently search for the route with the lowest driving cost compared to simple search algorithms;

[0054] (4) The driving cost calculation takes into account the driving distance and congestion conditions, simulating the actual driving situation and making the path planning more accurate;

[0055] (5) Use the region growing algorithm to accurately obtain the congested area, avoid driving through the congested area, and improve driving comfort;

[0056] (6) The reward function includes the driving distance and cost penalty terms, which can achieve the best balance between speed and comfort;

[0057] (7) The ε-greedy strategy can balance exploration and utilization. It can explore new states and actions as well as utilize known optimal solutions. The ε-value decay mechanism enables the model to gradually pass the optimal strategy from random exploration and reliably obtain the optimal solution.

[0058] (8) The Monte Carlo sampling method can effectively obtain the optimal strategy, does not require an environmental model, and has high computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0060] Figure 1 is an exemplary flow chart of a parking management method according to some embodiments of this specification;

[0061] Figure 2 is an exemplary flow chart of building a parking environment model according to some embodiments of this specification;

[0062] Figure 3 is an exemplary flow chart of searching for an optimal driving path according to some embodiments of this specification;

[0063] Figure 4 An exemplary flowchart of building an open list according to some embodiments of the present specification. DETAILED DESCRIPTION

[0064] The method and system provided in the embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0065] Figure 1 This is an exemplary flow chart of a parking management method according to some embodiments of this specification, which collects parking environment data, the parking environment data includes a parking lot map and parking space status information; preprocesses the collected data set; uses a reinforcement learning method to establish a parking environment model based on the preprocessed data set; and obtains an optimal parking path based on the parking environment model.

[0066] Through various sensors (RFID readers, cameras, etc.), multi-source parking environment data including parking lot maps and real-time status of each parking space are collected and pre-processed to generate a structured data set. RFID readers are set up at the entrance and exit of the parking lot. When a vehicle enters, the RFID reader reads the electronic tag on the vehicle's windshield to obtain the vehicle's unique ID. When the vehicle leaves, the electronic tag is read again to confirm that the vehicle has left. Multiple groups of RFID readers are set up on the main road to detect the flow of traffic. A camera is installed above each parking space to capture images of the parking area. The image recognition algorithm is used to determine whether the parking space is occupied by a vehicle. Once a vehicle stops, the algorithm detects that the parking space status has changed to "occupied". The camera collects images at a frequency of 1Hz and continuously detects the status of the parking space. The data from RFID and cameras are aggregated to the central server to build a multi-source data set. The back-end algorithm can grasp the precise status of vehicles and parking spaces in real time.

[0067] The camera takes images of fixed parking spaces at a frequency of 1 Hz. The images are input into the designed convolutional neural network. The network determines the parking area in the image and outputs a binary result of whether there is a vehicle. The parking space status result is sent to the central database server. The relational database MySQL runs on the server. A vehicle information table and a parking space status table are created in the database. The vehicle information table contains information such as vehicle ID, detection time, and location. The parking space status table contains parking space ID, detection time, and status (idle or occupied). After receiving new data, SQL is used to perform an insert operation to update the information table and status table. The new status is reflected in the database in real time for the algorithm module to query and make decisions.

[0068] The data formats collected by RFID readers and cameras are not uniform. Design a custom data format that includes necessary fields: detection time, location, object ID, status, etc. After receiving new data, call the format conversion function to convert it to a custom format. Serialize the converted data with JSON or XML encoding. Start a new transaction in the database. According to the object ID and location, SQL performs an insert operation to insert the data into the corresponding data table. Execute the transaction commit and the data is persistently stored in the database. The algorithm module can obtain the required data through SQL query statements. For example, obtain the location and number of vehicles in a certain area within a certain period of time. Through the real-time insertion of the transaction mechanism, it can be ensured that the algorithm module obtains the latest status data in a timely manner. Normalization and transaction insertion can make massive multi-source data flow into the database reliably for real-time algorithm calls.

[0069] Figure 2This is an exemplary flow chart for building a parking environment model according to some embodiments of this specification. The preprocessed data set is read, and the state of each parking space in the parking lot is used as the state space, and the driving behavior from the current parking space to the target parking space is used as the behavior space to build the state space S and behavior space A in the Q learning algorithm. At the same time, the reward function R and the strategy π are defined. Create a parking space status table parking_spot_status in the MySQL database. The table contains the fields: id, spot_id, timestamp, status. id is the auto-increment primary key, spot_id is the unique number of the parking space. timestamp is the detection timestamp, and status is the parking space status. The status includes empty and occupied. The new status data detected is continuously inserted into the table through the SQL insert statement. In the Python program, use MySQL Connector to connect to the database. Query statement: SELECTspot_id, timestamp, status FROM parking_spot_status; fetchall from the database to obtain the status of all parking spaces with the most recent detection time. Organize the status data into a dict, with the key being spot_id and the value being the status. This constructs the state space S for the reinforcement learning algorithm.

[0070] Get the latest status data of all parking spaces from the database. Initialize a dictionary parity_spots_status. Iterate over all parking spaces, and store the parking space ID as the key and the status as the value in the dictionary. For example

[0071] parity_spots_status[spot1_id]=empty. For N parking spaces, get a dictionary of length N. In the Python class of the Q network, define the initialization function __init__(x). In __init__(x), create an array self.states=[] with a length of N. Add the dictionary key spot_id to the array states one by one. Define the number of states self.state_size=N. The constructed array self.states is the entire state space S. The array index is the state code s, and the value is the parking space ID. The algorithm can represent the state space through state coding operations. By constructing a discrete state space through an array, the storage and operation of the state by the reinforcement learning algorithm can be efficiently realized.

[0072] In the Python class of the Q network, define the behavior space A. A contains four basic actions: up, down, left, and right. Add four additional oblique actions: upper left, upper right, lower left, and lower right. Initialize an array self.actions = [] in the __init__(x) function. Define the code for each action, such as up = 0, down = 1, left = 2, and right = 3. Upper left = 4, upper right = 5, lower left = 6, and lower right = 7. Add the codes to the self.actions array in sequence. Define the number of actions self.action_size = 8. When selecting an action, generate a random integer in the range of [0, 7]. As the index of self.actions, get the corresponding action code. The action code can be used to calculate the state change after executing the action. Expanding the oblique actions can make the state space traversal faster and wider. Defining a discrete behavior space can easily encode each action for state transition.

[0073] Get the user's parking location from the mobile phone APP and determine that the parking space id is spot_1. Get user information from the identity recognition system and determine that the target parking space id is spot_105. Define self.start_state = 0, self.end_state = 104 in __init__(x) of the Q network. Find the serial number idx_1 of spot_1 in the self.states array. Assign idx_1 to self.start_state. Find the serial number idx_105 of spot_105 in the self.states array. Assign idx_105 to self.end_state. self.start_state represents the starting state s0 of reinforcement learning. self.end_state represents the target state sT that is expected to be achieved. The optimal strategy from s0 to sT is continuously optimized during training. By associating the parking space ID to the state code, the state of the user's current position and the target position can be initialized.

[0074] Manhattan distance is introduced to calculate the distance between the current state s and the target state s_goal. The Manhattan distance formula is: |s_xs_goal_x|+|s_ys_goal_y|, where (s_x, s_y) is the coordinate of state s, and (s_goal_x, s_goal_y) is the coordinate of s_goal. The x and y coordinates of the parking space corresponding to each state s are pre-recorded in the coordinate dictionary. Get the coordinates of the current state s and the target state s_goal. Calculate the Manhattan distance d between the two states. Use the distance as an immediate reward: R=-d, and the value of the reward function is a negative Manhattan distance. Users in reinforcement learning will maximize the cumulative reward. It is equivalent to minimizing the Manhattan distance between the state and the target. The Manhattan distance reward function can effectively guide users to learn the shortest path.

[0075] Define the initial policy function init_policy(x) in the Q network class. Create a two-dimensional array Q_table with a shape of [number of states, number of actions]. Use the random.rand(x) function in numpy to generate random numbers between [0, 1]. Fill the random numbers into the Q_table array. For each row / each state: sum the values ​​in the row to get sum_i, divide each value by sum_i to get the probability p_i, p_i is the probability of selecting each action, perform multiple iterative training, and gradually optimize the strategy π. Finally, the optimal strategy that maximizes the cumulative reward is obtained. The provided π can guide the agent to choose the optimal action. Release navigation resources and complete this service. By randomly initializing the strategy and iterative optimization, the optimal navigable strategy can be learned. Initializing to a random strategy, traversing all possibilities, and then approaching the optimal through training is a standard paradigm for reinforcement learning.

[0076] Inherit the nn.Module class to define the Q Network class, which represents the neural network of the Q function. In __init__(x), define a fully connected layer self.fc1, the input feature number is the number of states + the number of actions, and the output feature number is 128. Define the second fully connected layer self.fc2, the input feature number is 128, and the output feature number is 1. The forward propagation is defined as: the one-hot encoding of the input state s is concatenated with the one-hot encoding of the action a. Pass it to the fully connected layer fc1, and then pass it through the ReLU activation function. Then pass it to the fc2 layer to get a value, namely Q(s, a). Use MSE Loss(x) as the loss function to optimize the network parameters. Input state s and action a to predict the value of Q(s, a). Iterative training makes the value predicted by the Q network close to the true Q value. Finally, a Q network that can accurately predict the state-action value is obtained. Use PyTorch to build a network to represent the Q function, and you can learn the optimal state-action value function end-to-end.

[0077] Initialize a higher ε value, such as 0.9, to control the probability of random exploration. Define the decay function of ε, and reduce ε by a certain proportion in each iteration. In the Q network training loop: Generate a random number between [0, 1] based on ε. If the random number is less than ε, randomly select an action. Otherwise, select the greedy action with the largest Q value based on the Q network output. Execute the action, observe the reward and the next state. Calculate the Q target value and optimize the Q network. After multiple iterations of the loop, ε gradually decays, exploration decreases, and greedy actions increase. Finally, the Q network converges and can accurately predict the optimal action in each state. End training and save the final Q network. When used, directly output the optimal action selection recommended by the model. The ε-greedy strategy can effectively balance exploration and utilization and learn the optimal strategy.

[0078] Figure 3 This is an exemplary flowchart for searching for the optimal driving path according to some embodiments of this specification. Through a large number of random simulation sampling iterative learning, the intelligent agent can master the strategy of selecting the optimal action under different states. Initialize a dictionary Q = {}, the key is the (state, action) tuple, and the value is initialized to 0. Loop in an episode: reset the environment and initialize the starting state s. Select an action a according to the ε-greedy strategy. Execute action a, the environment transfers to the new state s', and obtains the immediate reward r. Record this transfer (s, a, s', r) to the experience pool replay memory. Randomly sample a transfer (s, a, s', r) from the experience pool. Add the reward r to the total reward corresponding to the state action in the dictionary: Q[s, a] + = r, s = s', and continue sampling actions until the end of the episode. Repeat multiple episodes to traverse all state-action spaces. Finally, Q[s, a] records the Monte Carlo reward for each state-action combination. Select the action with the largest reward in each state as the optimal strategy. Monte Carlo strategy iteration can be easily implemented using a dictionary to accumulate state-action returns.

[0079] After multi-episode Monte Carlo sampling, the dictionary Q records the cumulative rewards of each state-action combination. Traverse each state s and action a in the dictionary Q: Get the cumulative total reward Q[s, a] of the state action. Get the number of sampling times n[s] of state s. Calculate the average reward: Q[s, a] = Q[s, a] / n[s]. For each unique state s: Find the action a_max with the largest Q[s, a]. Take a_max as the action in the optimal strategy: π(s) = a_max. Get a dictionary π, which records the optimal action selection in each state. In the new problem, query π[s] to know the optimal action. By calculating the expected return to find the optimal action for each state, the overall optimal strategy can be constructed. Based on the learned optimal strategy π, the A* search algorithm is used in the behavior space A to plan the optimal path from the current parking space to the target parking space.

[0080] Policy evaluation function input_policy(x): Randomly initialize a state s. Repeatedly sample actions according to the policy π and record the cumulative return R. Repeat multiple rounds of simulation and calculate the average return V(π). Return V(π), perform Monte Carlo sampling, and get the new policy π_new. V_current=input_policy(π_new), V_last=input_policy(π_old), calculate the difference: delta=abs(V_current V_last), if delta<0.01: print "the policy has converged", return π_new as the final policy, otherwise: π_old=π_new, continue sampling iteration, and determine whether it has converged by calculating the change in policy value, which can effectively obtain a reliable optimal policy. Otherwise, perform a new round of sampling and repeat the above steps. After convergence, obtain the final policy π_final. Among them, π_final(s) gives the optimal action for each state s. In the new problem, the optimal action selection can be obtained by looking up the table π_final[s]. Complete this service and release navigation computing resources.

[0081] Figure 4According to the exemplary flowchart of constructing an open list shown in some embodiments of the present specification, an open list open_list is created, a starting state s0 is added, a closed list close_list is created and initialized to empty, while open_list is not empty: current = state s with the smallest f(s) value in open_list, if current is the target state: create an open list open_list and add the starting state s0. Create a closed list close_list and initialize it to empty. While open_list is not empty: current = state s with the smallest f(s) value in open_list, if current is the target state: add current to close_list, and traverse all neighbor states next_states of current. For state s'innext_states: if s' is not in open_list and close_list: calculate g(s') as the path cost from current to s', calculate h(s') as the estimated cost from s' to the target, f(s') = g(s') + h(s'), add s' to open_list, if s' is in open_list: if g(s') is smaller, update s' in open_list. By maintaining the open list and the closed list, the A* algorithm can be effectively implemented to search for the optimal path. While open_list is not empty: the current state current = the state with the smallest f(s) in open_list. Traverse all neighbor states next_states of current. for state s'innext_states: if s' is in open_list: calculate the path cost g(s') from current to s'. If g(s') is less than g(s') of s' in open_list: update g(s') of s' in open_list with g(s'), calculate the heuristic function h(s') of s', and update f(s') = g(s') + h(s'). Traverse all adjacent states in this way. When open_list is empty, it means that the path is not found, return failure, take the end point from open_list, backtrack the path, return the result, and complete this navigation service. By updating the smaller g(s') of the same state in open_list, the A* algorithm can select a better path and obtain the final minimum cost result.

[0082] The optimal path obtained by the search is visualized and presented to the user or passed to the autonomous driving system to complete the entire technical solution. According to the optimal strategy π finally learned, define the backtracking function get_optimal_path(s_start, s_goal). Backtracking from the target state s_goal, recursively call π(s) to obtain the optimal state of the previous step until s_start is reached. The obtained state sequence is restored to the corresponding parking space id sequence path = [s1, s2S, ..., sn]. Mark all parking space locations on the parking lot map. Use mat plot slib to draw the dynamic navigation trajectory of the parking space id sequence path. Save it as a navigation path animation and transmit it to the user terminal in video format. You can also convert the path into a control command of the autonomous driving system to navigate through the corresponding parking space step by step. When the path and trajectory actually driven by the autonomous driving vehicle match, this service is completed. Release computing resources. The visualized dynamic navigation path intuitively presents the optimal result given by the intelligent navigation system.

[0083] The invention of the present application and its implementation methods are described schematically above. The description is not restrictive. Without departing from the spirit or basic features of the present application, the present application can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention of the present application. The actual structure is not limited to this, and any figure mark in the claims should not limit the claims involved. Therefore, if ordinary technicians in this field are inspired by it, without departing from the purpose of the present invention, they can creatively design structural methods and embodiments similar to the technical solution, which should all belong to the scope of protection of the present application. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A parking management method, comprising: The collection includes parking environment data, which includes parking lot maps and parking space status information; Preprocess the collected data sets; Based on the preprocessed data set, a parking environment model is established using reinforcement learning method; Obtain the optimal parking path based on the parking environment model; The parking environment model is constructed using Q learning algorithm; Set the state space S of the Q learning algorithm to be the state of each parking space in the parking lot, the behavior space A to be the action of driving from the current parking space to the target parking space, the reward function R to be the negative cost of driving from the current parking space to the target parking space, and the strategy π to be the action strategy with the smallest negative cost when executing the action; In the state space S, set the traversal action strategy π to obtain the optimal strategy π in a way that maximizes the return in the long term; According to the obtained optimal strategy π, a parking environment model is constructed; Strategy π adopts the ε-greedy strategy, which selects a random action with probability ε and selects the action with the smallest negative reward with probability 1-ε; The reward function R is the negative reward from the current parking space to the target parking space; Negative rewards include negative rewards for driving distance and negative rewards for exercise cost; Adjust the value of ε according to the preset ε decay function; Determine whether the current ε value is greater than a preset threshold; If yes, then randomly generate an action and execute it; otherwise, select the action with the smallest negative reward to execute; Adjusting the value of ε according to a preset ε attenuation function includes the following steps: Set the initial value of ε to 0.9; set the preset threshold to 0.05; Set the decay function of ε to ε*0.9; After each action is executed, the ε value is updated using the decay function; The Monte Carlo method is used to traverse the strategy π in the state space S; The Monte Carlo method obtains the optimal strategy π of expectation maximization through multiple random simulation samplings, including the following steps: Initialize the strategy of all states s to a random strategy; Randomly sample the execution strategy π from the initial state to obtain the path; Calculate the cumulative reward R on the path; Traverse all states s on the path; Calculate the expected reward for each action a under s: Q(s,a) = R(s,a) / number of times a is chosen; Update strategy: π(s) = max_aQ(s, a); Repeat the Monte Carlo method multiple times and update iteratively; After multiple iterations, the optimal strategy is obtained; The action path planning from the current parking space to the target parking space in the behavior space A adopts the A* search algorithm; Get the location layout of parking spaces in the parking lot and construct a two-dimensional grid map; According to the coordinates of the current parking space and the target parking space in the constructed two-dimensional grid map, the heuristic function is set to the straight-line distance between the two points; Set up an open list to store the expansion grid; Add the square where the current parking space is located as the starting point to the open list; Traverse the surrounding squares within the preset range of the current square, and calculate the travel cost from the current square to the surrounding squares. The travel cost is calculated based on the driving distance and whether it passes through the congested area. Select the surrounding squares with the lowest exercise cost and add them to the open list; Determine whether the target parking space is included in the open list. If not, repeat the previous step; According to the expansion order of each square in the open list, the center points of the squares are connected in the expansion order until they are connected to the center point of the target square. The resulting broken line path is used as the optimal driving path; The driving cost is calculated based on the driving distance and whether the vehicle passes through a congested area, including: In the constructed two-dimensional grid map, the driving distance data from the current grid to the surrounding grids is obtained based on the Euclidean distance between the center points of each two grids; Obtain the entrance and exit locations and main channel locations of the parking lot, and use the region growing algorithm to obtain a subset of congested squares as the congested area; Establish a correspondence table between driving distance and distance cost, input the calculated driving distance data, and obtain the distance cost according to the correspondence table; Construct a congestion cost table and preset the congestion cost of the squares in the congested area; If the surrounding grid belongs to the congested grid subset, the constructed congestion cost table is queried to obtain the congestion cost; The exercise cost is calculated based on the calculated distance cost and congestion cost.

2. The parking management method according to claim 1, characterized in that: The congested grid subsets are obtained by using the region growing algorithm, including: Get the entrance and exit locations and main channel locations of the parking lot, and use the corresponding squares as the initial congestion area; Take each square in the initial congestion area as the growth starting point and traverse the surrounding squares within the preset range; Determine whether the surrounding squares are marked as congested squares. If not, proceed to the next step; Calculate the cost of growing from the starting point to the unmarked surrounding squares; If the calculated exercise cost is greater than the preset cost threshold, the unmarked surrounding squares are determined to be congested squares, marked and added to the initial congested area; Repeat the above steps until the number of squares in the initial congested area no longer changes, and then output the final congested area.

Images