Parking lot management system based on cloud computing

Abstract

The invention relates to the technical field of parking management, and discloses a parking lot management system based on cloud computing, and the system comprises a second judgment module which is used for calculating the Euclidean distance between a to-be-stored vehicle and a parking space when the number of parking spaces in an idle state is greater than 0 and the number of the to-be-stored vehicle is 1, the parking spaces are sorted from small to large, the first K parking spaces are obtained, the comprehensive scores of the K parking spaces are calculated, and the parking space with the highest comprehensive score is selected as the target parking space of the vehicle to be put into the garage; the path generation module is used for generating a driving path for the to-be-put-in-garage vehicle through reinforcement learning; according to the invention, the driving path of a single to-be-put-in-garage vehicle is rapidly generated through a white whale optimization algorithm, the driving paths of a plurality of to-be-put-in-garage vehicles are automatically generated through combination of reinforcement learning and a multi-layer sensor, and the situation that a plurality of vehicles fight for the same parking space at the same time is avoided through a queue and a state lock.

Description

Technical Field

[0001] The present invention relates to the technical field of parking management, and more particularly, to a parking lot management system based on cloud computing. Background Art

[0002] With the acceleration of urbanization and the continuous growth of car ownership, parking difficulties have gradually become a common problem. Existing parking management systems usually use the Manhattan distance between the vehicle's current position and the target parking space as the evaluation function, and implement path planning through the A* algorithm to assist vehicles in finding the target parking space. The advantage of this method is that it can quickly find the shortest path from the starting point to the end point, and is suitable for static path planning in a known environment.

[0003] However, this static path planning has the following defects:

[0004] 1. Parking environments are dynamic, and the target parking space may be occupied by other vehicles at any time. Especially during peak hours, parking resources are tight, and the availability of the target parking space is difficult to guarantee. In addition, vehicles may encounter congestion during driving, making the originally planned route no longer applicable.

[0005] 2. When multiple vehicles enter the parking lot at the same time, there may be a phenomenon in which multiple vehicles compete for the same parking space, resulting in conflicts in path planning. Summary of the Invention

[0006] The present invention provides a parking lot management system based on cloud computing to solve the technical problems in the above-mentioned background technology.

[0007] The present invention provides a parking lot management system based on cloud computing, comprising:

[0008] The parking lot map identification module is used to obtain the parking space occupancy status of the parking lot in real time through the camera combined with the target detection model, and mark it on the parking lot map;

[0009] Each parking space in the parking lot is assigned a unique number;

[0010] The status of the parking space is represented by 0 or 1, 0 means the parking space is vacant, and 1 means the parking space is occupied;

[0011] The waiting-for-entry vehicle acquisition module is used to obtain the number of waiting-for-entry vehicles at all parking entrances in real time through a camera combined with a target detection model;

[0012] A first judgment module is used to judge that when the number of parking spaces in the idle state is equal to 0, the waiting vehicle is not allowed to enter the parking lot;

[0013] The second judgment module is used to determine if the number of vacant parking spaces is greater than 0 and the number of vehicles waiting to enter is 1. Then, the Euclidean distance between the vehicles waiting to enter and the parking spaces is calculated, and the top K parking spaces are sorted in ascending order. The comprehensive scores of the K parking spaces are calculated, and the parking space with the highest comprehensive score is selected as the target parking space for the vehicle waiting to enter.

[0014] The third judgment module is used to judge that when the number of idle parking spaces is greater than or equal to the number of vehicles to be parked, and the number of vehicles to be parked is greater than 1, all vehicles to be parked are added to the waiting queue in ascending order of parking time;

[0015] The path generation module is used to calculate the comprehensive score between each waiting vehicle in the waiting queue and all available parking spaces in a first-in, first-out order, select the parking space with the highest comprehensive score as the target parking space for the waiting vehicle, and generate a driving path for the waiting vehicle through reinforcement learning.

[0016] Furthermore, the parking lot is divided into multiple parking areas by rectangular frames. A unique self-incrementing number is first assigned to each parking area, and a unique self-incrementing number is also assigned to each parking space in each parking area. The size of the matrix frame is a custom parameter.

[0017] Furthermore, the target detection model is a YOLO model, and the sample labels of the training samples used to train the target detection model are obtained through manual labeling.

[0018] Furthermore, K is a custom parameter, and K is greater than or equal to the number of parking spaces in an idle state.

[0019] Furthermore, K statistical ranges are generated with the center point between the parking space and the vehicle to be parked as the origin and the distance between the parking space and the vehicle to be parked as the diameter. The number of vehicles that deviate from the parking space within each K statistical range is obtained to calculate the comprehensive score of the K parking spaces.

[0020] The formula for calculating the comprehensive score is as follows:

[0021]

[0022] Where 1≤k≤K, Denotes the comprehensive score of the k-th parking space, Dis k represents the Euclidean distance between the kth parking space and the parking space to be entered, Car k represents the number of vehicles that deviate from the parking space within the kth statistical range. U1 and U2 represent the first weight coefficient and the second weight coefficient respectively. Both U1 and U2 are custom parameters, and the sum of U1 and U2 is 1.

[0023] Furthermore, when the number of idle parking spaces is greater than 0 and the number of vehicles waiting to enter the parking lot is 1, the position of the vehicle waiting to enter the parking lot is taken as the starting point, and the position of the target parking space is taken as the ending point. The path between the starting point and the ending point is divided into M sections at equal intervals, and the optimal path from the starting point to the ending point is generated by the White Whale optimization algorithm; the individuals of the initialized population of the White Whale optimization algorithm are represented by vectors, the number of dimensions of the vector is M, and each dimension value is represented by 0 or 1. The mth dimension value is 0, which means that the mth section is not passed, otherwise it means that the mth section is passed, where 1≤m≤M, M is a custom parameter, and the individual vector must meet the constraint condition, that is, each section from the starting point to the ending point must be connected; in each iteration of the White Whale optimization algorithm, the fitness value of the individual is calculated by the objective function until the preset maximum number of iterations is reached, and the vector of the individual with the largest fitness value is taken as the optimal path;

[0024] The objective function is calculated as follows:

[0025]

[0026] Among them, Fit represents the fitness value of the individual, Car m Indicates the number of vehicles appearing on the mth road segment.

[0027] Furthermore, a driving path is generated for the vehicle to be parked using reinforcement learning, including the following steps:

[0028] Step S201, defining a state space of vehicles to be entered into the warehouse, where the state space of the vehicles to be entered into the warehouse is represented by the positions of the vehicles to be entered into the warehouse;

[0029] Step S202: defining an action space of a vehicle to be entered into the warehouse, wherein the action space of the vehicle to be entered into the warehouse includes: moving forward, turning left, turning right, and stopping;

[0030] Step S203: Generate a random number between 0 and 1. If it is greater than or equal to a set threshold, randomly select an action in the action space as the action of the vehicle to be parked in the next time step. Otherwise, proceed to step S204, where the threshold is set as a custom parameter.

[0031] Step S204, using the Q-value update function to calculate the Q-value of each action of the vehicle to be entered in the action space, and selecting the action with the largest Q-value as the action of the vehicle to be entered in the next time step;

[0032] The calculation formula of the Q value update function is as follows:

[0033] Q(s t+1 ,a n )=Q(s t ,a t)+α[R(s t ,a t )+β×Q(s t+1 ,a n )-Q(s t ,a t )];

[0034] Where 1≤n≤4, a n represents the nth action of the vehicle waiting to enter the warehouse in the action space, a t Indicates the action of the vehicle waiting to enter the warehouse at the current time step, s t and s t+1 They represent the positions of the vehicle to be stored at the current time step and the next time step, respectively. Q(s t ,a t ) and R(s t ,a t ) represent the Q value and reward value of the vehicle to be stored at the current time step, Q(s t+1 ,a n ) represents the position of the vehicle to be stored in the next time step and the Q value of the n-th action, α represents the learning rate, and β represents the discount factor. Both α and β are custom parameters.

[0035] Step S205, repeating steps S203 to S204 until the vehicle to be parked arrives at the target parking space.

[0036] Furthermore, the calculation formula of the reward value includes:

[0037]

[0038] Where R represents the reward value, dis represents the Euclidean distance between the position of the vehicle to be parked and the target parking space, speed represents the speed of the vehicle to be parked, collision represents whether the vehicle to be parked has collided, 0 represents no, 1 represents yes, turn represents whether the vehicle to be parked has turned, 0 represents no, 1 represents yes, U3, U4, U5 and U6 represent the third weight coefficient, fourth weight coefficient, fifth weight coefficient and sixth weight coefficient respectively, and all are custom parameters.

[0039] Furthermore, the Q value of the position and action of the vehicle to be parked at the current time step is obtained through a multi-layer perceptron, where the input of the multi-layer perceptron is the state vector and the output is the Q value. The state vector is expressed as: A = {a1…a5}, where a1…a5 respectively represent the Euclidean distance between the position of the vehicle to be parked and the target parking space, the speed, whether a collision occurs, whether a turn occurs, and the number of vehicles in front. The sample labels of the training samples used to train the multi-layer perceptron are obtained through manual annotation, and the value range of the Q value is set to 1 to 4.

[0040] Furthermore, when the vehicle to be parked determines the target parking space, the status flag of the parking space is modified to 1.

[0041] The beneficial effects of the present invention are as follows: the present invention quickly generates the driving path of a single vehicle to be parked through the White Whale optimization algorithm, automatically generates the driving paths of multiple vehicles to be parked through reinforcement learning combined with a multi-layer perceptron, and avoids the situation where multiple vehicles compete for the same parking space at the same time through queues and state locks. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a schematic diagram of a parking lot management system based on cloud computing of the present invention;

[0043] Figure 2 This is a flow chart of the present invention for generating a driving path for a vehicle to be depotted through reinforcement learning.

[0044] In the figure: parking lot map identification module 101, waiting-for-parking vehicle acquisition module 102, first judgment module 103, second judgment module 104, third judgment module 105, path generation module 106. DETAILED DESCRIPTION

[0045] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed solely to enable those skilled in the art to better understand and implement the subject matter described herein, and that the functions and arrangements of the elements discussed may be varied without departing from the scope of this specification. Various examples may omit, substitute, or add various processes or components as needed. In addition, features described with respect to some examples may also be combined in other examples.

[0046] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of the present invention should have the usual meanings understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in one or more embodiments of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprising" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, but do not exclude other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0047] like Figures 1 and 2As shown, a parking lot management system based on cloud computing includes:

[0048] The parking lot map identification module 101 is used to obtain the parking space occupancy status of the parking lot in real time through a camera combined with a target detection model, and mark it on the parking lot map;

[0049] Each parking space in the parking lot is assigned a unique number;

[0050] The status of the parking space is represented by 0 or 1, 0 means the parking space is vacant, and 1 means the parking space is occupied;

[0051] The waiting-for-entry vehicle acquisition module 102 is used to obtain the number of waiting-for-entry vehicles at all parking entrances in real time through a camera combined with a target detection model;

[0052] A first judgment module 103 is used to judge that when the number of idle parking spaces is equal to 0, the waiting vehicle is not allowed to enter the parking lot;

[0053] The second judgment module 104 is configured to determine, when the number of vacant parking spaces is greater than 0 and the number of vehicles to be parked is 1, calculate the Euclidean distances between the vehicles to be parked and the parking spaces, sort the spaces in ascending order, obtain the top K parking spaces, calculate the comprehensive scores of the K parking spaces, and select the parking space with the highest comprehensive score as the target parking space for the vehicle to be parked;

[0054] The third judgment module 105 is used to judge that when the number of idle parking spaces is greater than or equal to the number of vehicles to be parked, and the number of vehicles to be parked is greater than 1, all vehicles to be parked are added to the waiting queue in ascending order of parking time;

[0055] The path generation module 106 is used to calculate the comprehensive score between each waiting vehicle in the waiting queue and all available parking spaces in a first-in, first-out order, select the parking space with the highest comprehensive score as the target parking space for the waiting vehicle, and generate a driving path for the waiting vehicle through reinforcement learning.

[0056] It should be noted that the cloud computing-based parking lot management system provided by the present invention can be integrated into mobile applications or WeChat applets, and can display the currently idle parking spaces in the parking lot to each driver of a vehicle waiting to enter the parking lot, so that the driver can find a parking space.

[0057] In one embodiment of the present invention, a parking lot is divided into multiple parking areas by rectangular frames. A unique self-incrementing number is first assigned to each parking area, and a unique self-incrementing number is also assigned to each parking space in each parking area. The size of the matrix frame is a custom parameter. For example, if the unique number of the parking spaces is 8-11, it means that the parking area numbered 8 has parking space numbered 11.

[0058] In one embodiment of the present invention, a unique number may be directly assigned to each parking space in the parking lot through a unique self-incrementing code, or a unique number may be generated through a snowflake algorithm.

[0059] In one embodiment of the present invention, the target detection model is a YOLO model, and the sample labels of the training samples used to train the target detection model are obtained by manual annotation. Specifically, the target detection model can be a YOLO-V8 model, or the target detection model can be pre-trained using an existing open source dataset, such as COCO or Open Images.

[0060] In one embodiment of the present invention, K is a custom parameter, and K is greater than or equal to the number of parking spaces in an idle state. Preferably, K is set to 3.

[0061] In one embodiment of the present invention, K statistical ranges are generated with the center point between the parking space and the vehicle to be parked as the origin and the distance between the parking space and the vehicle to be parked as the diameter. The number of vehicles that deviate from the parking space within each of the K statistical ranges is obtained to calculate the comprehensive score of the K parking spaces.

[0062] The formula for calculating the comprehensive score is as follows:

[0063]

[0064] Where 1≤k≤K, Denotes the comprehensive score of the k-th parking space, Dis k represents the Euclidean distance between the kth parking space and the parking space to be entered, Car k represents the number of vehicles that deviate from the parking space within the kth statistical range, U1 and U2 represent the first weight coefficient and the second weight coefficient respectively, U1 and U2 are both custom parameters, and the sum of U1 and U2 is 1. Preferably, U1 is set to 0.8 and U2 is set to 0.2.

[0065] It should be noted that vehicles that deviate from parking spaces are used to indicate the congestion level of the lane. Similarly, the number of vehicles that deviate from parking spaces can also be obtained through the target detection model, which will not be elaborated here.

[0066] In one embodiment of the present invention, when the number of idle parking spaces is greater than 0 and the number of vehicles to be parked is 1, the position of the vehicle to be parked is taken as the starting point, and the position of the target parking space is taken as the ending point. The path between the starting point and the ending point is divided into M sections at equal intervals, and the optimal path from the starting point to the ending point is generated by the White Whale optimization algorithm; the individuals of the initialized population of the White Whale optimization algorithm are represented by vectors, the number of dimensions of the vector is M, and each dimension value is represented by 0 or 1. The mth dimension value is 0, which means that the mth section is not passed, otherwise it means that the mth section is passed, where 1≤m≤M, M is a custom parameter, preferably, M is set to 20, and the vector of the individual must meet the constraint condition, that is, each section from the starting point to the ending point must be connected; in each iteration of the White Whale optimization algorithm, the fitness value of the individual is calculated by the objective function until the preset maximum number of iterations is reached, and the vector of the individual with the largest fitness value is taken as the optimal path;

[0067] The objective function is calculated as follows:

[0068]

[0069] Among them, Fit represents the fitness value of the individual, Car m Indicates the number of vehicles appearing on the mth road segment.

[0070] It should be noted that each road segment from the starting point to the end point must be connected, indicating that the generated vector must represent a continuous path. In addition, the optimal path can also be generated through other heuristic algorithms, such as the particle swarm optimization algorithm, the firefly optimization algorithm, etc., or directly through the A* algorithm. The path planning for a single vehicle to be stored is mainly based on computational efficiency.

[0071] In one embodiment of the present invention, Figure 2 As shown in the figure, generating a driving path for a vehicle to be parked by reinforcement learning includes the following steps:

[0072] Step S201, defining a state space of vehicles to be entered into the warehouse, where the state space of the vehicles to be entered into the warehouse is represented by the positions of the vehicles to be entered into the warehouse;

[0073] Step S202: defining an action space of a vehicle to be entered into the warehouse, wherein the action space of the vehicle to be entered into the warehouse includes: moving forward, turning left, turning right, and stopping;

[0074] Step S203: Generate a random number between 0 and 1. If the random number is greater than or equal to a set threshold, randomly select an action in the action space as the action of the vehicle to be parked in the next time step. Otherwise, proceed to step S204. The set threshold is a custom parameter. Preferably, the set threshold is set to 0.9.

[0075] Step S204, using the Q-value update function to calculate the Q-value of each action of the vehicle to be entered in the action space, and selecting the action with the largest Q-value as the action of the vehicle to be entered in the next time step;

[0076] The calculation formula of the Q value update function is as follows:

[0077] Q(s t+1 ,a n )=Q(s t ,a t )+α[R(s t ,a t )+β×Q(s t+1 ,a n )-Q(s t ,a t )];

[0078] Where 1≤n≤4, a n represents the nth action of the vehicle waiting to enter the warehouse in the action space, a t Indicates the action of the vehicle waiting to enter the warehouse at the current time step, s t and s t+1 They represent the positions of the vehicle to be stored at the current time step and the next time step, respectively. Q(s t ,a t ) and R(s t ,a t ) represent the Q value and reward value of the vehicle to be stored at the current time step, Q(s t+1 ,a n ) represents the position of the vehicle to be stored in the next time step and the Q value of the n-th action, α represents the learning rate, and β represents the discount factor. Both α and β are custom parameters. Preferably, α is set to 0.1 and β is set to 0.9;

[0079] Step S205, repeating steps S203 to S204 until the vehicle to be parked arrives at the target parking space.

[0080] In one embodiment of the present invention, the calculation formula of the reward value includes:

[0081]

[0082] Where R represents the reward value, dis represents the Euclidean distance between the position of the vehicle to be parked and the target parking space, speed represents the speed of the vehicle to be parked, collision represents whether the vehicle to be parked has collided, 0 represents no, 1 represents yes, turn represents whether the vehicle to be parked has turned, 0 represents no, 1 represents yes, U3, U4, U5 and U6 represent the third weight coefficient, fourth weight coefficient, fifth weight coefficient and sixth weight coefficient respectively, and all are custom parameters.

[0083] It should be noted that U3, U4, U5 and U6 are used to indicate the intensity of rewards and punishments. The larger the value, the greater the intensity of rewards and punishments. Preferably, U3, U4, U5 and U6 are set to 4, 2, 3 and 1 respectively.

[0084] In one embodiment of the present invention, the Q value of the position and action of the vehicle to be parked at the current time step is obtained through a multi-layer perceptron, wherein the input of the multi-layer perceptron is a state vector, and the output is a Q value. The state vector is expressed as: A = {a1…a5}, wherein a1…a5 respectively represent the Euclidean distance between the position of the vehicle to be parked and the target parking space, the speed, whether a collision occurs, whether a turn occurs, and the number of vehicles in front. The sample labels of the training samples used to train the multi-layer perceptron are obtained by manual labeling, and the value range of the Q value is set to 1 to 4.

[0085] In one embodiment of the present invention, when the vehicle to be parked determines the target parking space, the status indicator of the parking space is changed to 1 to prevent multiple vehicles from competing for the same parking space at the same time. In addition, the parking space can also be locked by a distributed lock, which will not be elaborated here.

[0086] It should be noted that when the target parking space of the vehicle to be parked is occupied by other vehicles, the comprehensive score between the vehicle to be parked and all idle parking spaces will be recalculated to avoid the situation where multiple vehicles compete for the same parking space at the same time.

[0087] Furthermore, the present invention also includes an automatic vehicle search module 107, which automatically saves the unique number of the target parking space when the vehicle to be parked arrives at the target parking space, and determines the location of the target parking space according to the unique number of the target parking space when searching for the vehicle.

[0088] The above describes the embodiments of this embodiment, but this embodiment is not limited to the above specific implementation methods. The above specific implementation methods are merely illustrative and not restrictive. Ordinary technicians in this field can also make many forms based on the inspiration of this embodiment, all of which are protected by this embodiment.

Claims

1. A parking lot management system based on cloud computing, characterized in that: include: The parking lot map identification module is used to obtain the parking space occupancy status of the parking lot in real time through the camera combined with the target detection model, and mark it on the parking lot map; Each parking space in the parking lot is assigned a unique number; The status of the parking space is represented by 0 or 1, 0 means the parking space is vacant, and 1 means the parking space is occupied; The waiting-for-entry vehicle acquisition module is used to obtain the number of waiting-for-entry vehicles at all parking entrances in real time through a camera combined with a target detection model; A first judgment module is used to judge that when the number of parking spaces in the idle state is equal to 0, the waiting vehicle is not allowed to enter the parking lot; The second judgment module is used to determine if the number of vacant parking spaces is greater than 0 and the number of vehicles waiting to enter is 1. Then, the Euclidean distance between the vehicles waiting to enter and the parking spaces is calculated, and the top K parking spaces are sorted in ascending order. The comprehensive scores of the K parking spaces are calculated, and the parking space with the highest comprehensive score is selected as the target parking space for the vehicle waiting to enter. The third judgment module is used to judge that when the number of idle parking spaces is greater than or equal to the number of vehicles to be parked, and the number of vehicles to be parked is greater than 1, all vehicles to be parked are added to the waiting queue in ascending order of parking time; The path generation module is used to calculate the comprehensive score between each waiting vehicle in the waiting queue and all available parking spaces in a first-in, first-out order, select the parking space with the highest comprehensive score as the target parking space for the waiting vehicle, and generate a driving path for the waiting vehicle through reinforcement learning.

2. The cloud computing-based parking lot management system according to claim 1, characterized in that: The parking lot is divided into multiple parking areas using rectangular frames. Each parking area is first assigned a unique self-incrementing number, and each parking space in each parking area is also assigned a unique self-incrementing number. The size of the matrix frame is a custom parameter.

3. The parking lot management system based on cloud computing according to claim 1, characterized in that: The target detection model is the YOLO model, and the sample labels of the training samples used to train the target detection model are obtained through manual annotation.

4. The parking lot management system based on cloud computing according to claim 1, characterized in that: K is a custom parameter, and K is greater than or equal to the number of idle parking spaces.

5. The cloud computing-based parking lot management system according to claim 1, characterized in that: Generate K statistical ranges with the center point between the parking space and the vehicle to be parked as the origin and the distance between the parking space and the vehicle to be parked as the diameter. Then obtain the number of vehicles that deviate from the parking space within the K statistical ranges to calculate the comprehensive score of the K parking spaces. The formula for calculating the comprehensive score is as follows: Where 1≤k≤K, Denotes the comprehensive score of the k-th parking space, Dis k represents the Euclidean distance between the kth parking space and the parking space to be entered, Car k represents the number of vehicles that deviate from the parking space within the kth statistical range. U1 and U2 represent the first weight coefficient and the second weight coefficient respectively. Both U1 and U2 are custom parameters, and the sum of U1 and U2 is 1.

6. The cloud computing-based parking lot management system according to claim 1, characterized in that: When the number of idle parking spaces is greater than 0 and the number of vehicles waiting to enter the parking lot is 1, the position of the vehicle waiting to enter the parking lot is taken as the starting point, and the position of the target parking space is taken as the end point. The path between the starting point and the end point is divided into M sections at equal intervals, and the optimal path from the starting point to the end point is generated by the White Whale optimization algorithm; the individuals of the initialized population of the White Whale optimization algorithm are represented by vectors, the number of dimensions of the vector is M, and each dimension value is represented by 0 or 1. The mth dimension value is 0, which means that the mth section is not passed, otherwise it means that the mth section is passed, where 1≤m≤M, M is a custom parameter, and the individual vector must meet the constraint condition, that is, each section from the starting point to the end point must be connected; in each iteration of the White Whale optimization algorithm, the individual fitness value is calculated by the objective function until the preset maximum number of iterations is reached, and the vector of the individual with the largest fitness value is taken as the optimal path; The objective function is calculated as follows: Among them, Fit represents the fitness value of the individual, Car m Indicates the number of vehicles appearing on the mth road segment.

7. The cloud computing-based parking lot management system according to claim 1, characterized in that: Generating a driving path for a vehicle waiting to enter the warehouse through reinforcement learning includes the following steps: Step S201, defining a state space of vehicles to be entered into the warehouse, where the state space of the vehicles to be entered into the warehouse is represented by the positions of the vehicles to be entered into the warehouse; Step S202: defining an action space of a vehicle to be entered into the warehouse, wherein the action space of the vehicle to be entered into the warehouse includes: moving forward, turning left, turning right, and stopping; Step S203: Generate a random number between 0 and 1. If it is greater than or equal to a set threshold, randomly select an action in the action space as the action of the vehicle to be parked in the next time step. Otherwise, proceed to step S204, where the threshold is set as a custom parameter. Step S204, using the Q-value update function to calculate the Q-value of each action of the vehicle to be entered in the action space, and selecting the action with the largest Q-value as the action of the vehicle to be entered in the next time step; The calculation formula of the Q value update function is as follows: Q(s t+1 ,a n )=Q(s t ,a t )+α[R(s t ,a t )+β×Q(s t+1 ,a n )-Q(s t ,a t )]; Where 1≤n≤4, a n represents the nth action of the vehicle waiting to enter the warehouse in the action space, a t Indicates the action of the vehicle waiting to enter the warehouse at the current time step, s t and s t+1 They represent the positions of the vehicle to be stored at the current time step and the next time step, respectively. Q(s t ,a t ) and R(s t ,a t ) represent the Q value and reward value of the vehicle to be stored at the current time step, Q(s t+1 ,a n ) represents the position of the vehicle to be stored in the next time step and the Q value of the n-th action, α represents the learning rate, and β represents the discount factor. Both α and β are custom parameters. Step S205, repeating steps S203 to S204 until the vehicle to be parked arrives at the target parking space.

8. The cloud computing-based parking lot management system according to claim 1, characterized in that: The reward value calculation formula includes: Where R represents the reward value, dis represents the Euclidean distance between the position of the vehicle to be parked and the target parking space, speed represents the speed of the vehicle to be parked, collision represents whether the vehicle to be parked has collided, 0 represents no, 1 represents yes, turn represents whether the vehicle to be parked has turned, 0 represents no, 1 represents yes, U3, U4, U5 and U6 represent the third weight coefficient, fourth weight coefficient, fifth weight coefficient and sixth weight coefficient respectively, and all are custom parameters.

9. The cloud computing-based parking lot management system according to claim 1, characterized in that: The Q value of the position and action of the vehicle to be parked at the current time step is obtained through a multi-layer perceptron, where the input of the multi-layer perceptron is the state vector and the output is the Q value. The state vector is expressed as: A = a1…a5}, where a1…a5 represent the Euclidean distance between the position of the vehicle to be parked and the target parking space, the speed, whether a collision occurs, whether a turn occurs, and the number of vehicles in front, respectively. The sample labels of the training samples used to train the multi-layer perceptron are obtained through manual annotation, and the value range of the Q value is set to 1 to 4.

10. The parking lot management system based on cloud computing according to claim 1, characterized in that: When the vehicle to be parked has determined the target parking space, the status flag of the parking space is changed to 1.

Images