Parking space intelligent dynamic allocation and guidance method, device, equipment and medium

Abstract

The present invention discloses a method, device, equipment and medium for intelligent dynamic allocation and guidance of parking spaces, the method comprising: collecting environmental information of the parking lot to be optimized, analyzing parking behavior, and establishing a numerical simulation system of the parking lot; establishing an intelligent allocation model for parking spaces by combining dynamic allocation and delayed matching under the framework of the DQN deep reinforcement learning algorithm; substituting the intelligent allocation model for parking spaces into the numerical simulation system for training, as well as example analysis; after the training is completed, outputting a parking space allocation strategy, providing parking space allocation information and parking navigation, and motivating users to obey the allocation through a guidance mechanism. The present invention has stronger intelligence and adaptability, can improve the efficiency of vehicle parking, the charging demand satisfaction rate and the turnover rate of the parking system, and provide users with high-quality parking services. At the same time, it conforms to the development trend of technologies such as autonomous driving, and provides a solution to the parking space matching problem in the process of intelligent valet parking.

Description

Technical Field

[0001] The present invention relates to a method, device, equipment and medium for intelligent dynamic allocation and guidance of parking spaces, and belongs to the field of smart parking. Background Art

[0002] At present, the current intelligent allocation and guidance system of parking resources in my country only targets "parking lots", while the management of "parking spaces" is still the traditional random parking mode, which does not substantially solve the problems of low turnover efficiency of parking lots, unsatisfied charging demand and inefficient parking search of vehicles. It also seriously limits the improvement of parking system efficiency by intelligent valet parking and autonomous driving technology. In order to solve the parking problem, in addition to continuing to strengthen the construction of rigid demand parking facilities, we must also continuously improve the level of intelligence in parking resource management. Summary of the invention

[0003] In view of this, the present invention provides a method, device, system, computer equipment and storage medium for intelligent dynamic allocation and guidance of parking spaces, which fully considers the complex and dynamically changing electrical, temporal and spatial environment of the parking lot, and can improve the user's parking efficiency, the charging demand satisfaction rate and the parking lot turnover rate; at the same time, it conforms to the development trend of artificial intelligence and intelligent valet parking, and provides a solution to the parking space matching problem in the process of intelligent valet parking.

[0004] The first object of the present invention is to provide a method for intelligent dynamic allocation and guidance of parking spaces.

[0005] The second object of the present invention is to provide a parking space intelligent dynamic allocation and guidance device.

[0006] The third object of the present invention is to provide an intelligent parking system.

[0007] A fourth object of the present invention is to provide a computer device.

[0008] A fifth object of the present invention is to provide a storage medium.

[0009] The first object of the present invention can be achieved by adopting the following technical solutions:

[0010] A method for intelligent dynamic allocation and guidance of parking spaces, the method comprising:

[0011] Collect environmental information of the parking lot to be optimized, analyze parking behavior, and establish a numerical simulation system for the parking lot;

[0012] Under the framework of DQN deep reinforcement learning algorithm, a parking space intelligent allocation model is established by combining dynamic allocation and delayed matching;

[0013] Substituting the parking space intelligent allocation model into the numerical simulation system for training and example analysis;

[0014] After training is completed, the parking space allocation strategy is output, parking space allocation information and parking navigation are provided, and users are encouraged to obey the allocation through a guidance mechanism.

[0015] Preferably, the setting process of the Markov decision process of the parking space intelligent allocation model includes:

[0016] Set the parking system manager as an intelligent agent;

[0017] Define the state space as follows:

[0018]

[0019] Among them, S t represents the state space, t represents time, X occ Indicates the parking space occupancy status, X cha Indicates the distribution information of charging piles. Indicates the actual time. Indicates the vehicle energy type, Indicates the vehicle size type, Indicates the accumulated delay time of the vehicle;

[0020] Define the action space as follows:

[0021] A t ={n}, n∈[0, 2*N-1]

[0022] Among them, A t represents the action space, n is the action value selected by the agent, and N is the number of parking spaces in the parking lot;

[0023] When A t <N, then the corresponding vehicle is delayed matched;

[0024] When A t > N, then the corresponding vehicle is directly matched, and the matching parking space number is (A t -N);

[0025] Define the reward function as follows:

[0026]

[0027] Among them, R t represents the reward function, r park Indicates the reward value corresponding to the parking space distance, r charge Indicates the reward value corresponding to the charging demand, r delay Indicates the reward value corresponding to delayed matching;

[0028]

[0029] Where L is the reference distance, l drive is the driving distance from the entrance to the parking space, l walk is the walking distance from the parking space to the destination, is the ratio of vehicle speed to human walking speed, α is the user's walking willingness coefficient;

[0030]

[0031] in, Indicates the vehicle energy type, Indicates that vehicle i is a tram, Indicates that vehicle i is a fuel truck, a t represents the action value taken by the agent on the vehicle, [a t ]=1 means that the parking space assigned by the agent to vehicle i is equipped with a charging station, [A t ] = 0 means that the parking space assigned by the agent to vehicle i does not have a charging station;

[0032]

[0033] Set up the state transfer process: When a vehicle enters the parking lot, the agent outputs action A as a parking space allocation decision based on the current state S, obtains the corresponding reward R, and updates the next state S′.

[0034] Preferably, the setting process of the DQN deep reinforcement learning algorithm framework includes:

[0035] Set the Q value as follows:

[0036] Q(s,a)=E[R+γ*maxQ(s′,a′)]

[0037] Among them, Q(s, a) is the Q value corresponding to state s and action a, E represents the expected calculation of future cumulative returns, R is the corresponding reward value, γ is the discount factor, and maxQ(s′a′) is the maximum Q value of the next state and action;

[0038] A fully connected neural network is used as the neural network model to estimate the Q function, and the action corresponding to the maximum Q value output in each step is the decision;

[0039] Define the loss function as follows:

[0040]

[0041] Among them, θ is the model parameter, N is the number of samples, and s i is the input state, a iThe action selected by the model, Q(s i , a i ; θ) represents the model for state s i The selected action a i The predicted value, y i is the target value.

[0042] Preferably, the process of setting up the DQN deep reinforcement learning algorithm framework further includes:

[0043] Based on the experience experienced by the agent, a first preset number of experiences are randomly selected and stored in an experience replay pool, and a second preset number of experiences are randomly selected from the experience replay pool for training during training;

[0044] Introducing the target Q network;

[0045] The agent is required to select random actions according to the preset probability, and the optimal parking space allocation strategy is obtained through training, as shown in the following formula:

[0046]

[0047] Among them, a t is the action selected at time t, Q(s t , a) is state s t and the Q value of action a, random.randint(0, N-1) represents a random integer between [0, N-1], which is the number of the corresponding parking space, p represents the preset probability, and ε is the exploration rate.

[0048] Preferably, the parking navigation includes shortest parking path guidance based on Dijkstra algorithm.

[0049] Preferably, the step of motivating users to comply with allocation through a guiding mechanism comprises:

[0050] When the user complies with the parking space allocation, a positive reward is provided to the user, and the positive reward includes at least one of: parking fee discount and shopping points in the mall;

[0051] When the user does not comply with the parking space allocation, the user is guided by means, wherein the means include at least one of: a program pop-up window prompt, questionnaire feedback, and feedback on the walking time that the user will lose.

[0052] The second object of the present invention can be achieved by adopting the following technical solutions:

[0053] A parking space intelligent dynamic allocation and guidance device, the device comprising:

[0054] The first establishment module is used to collect environmental information of the parking lot to be optimized, analyze parking behavior, and establish a numerical simulation system for the parking lot;

[0055] The second building module is used to build a parking space intelligent allocation model by combining dynamic allocation and delayed matching under the DQN deep reinforcement learning algorithm framework;

[0056] A training module, used to substitute the parking space intelligent allocation model into the numerical simulation system for training and example analysis;

[0057] The allocation module is used to output the parking space allocation strategy after training, provide parking space allocation information and parking navigation, and motivate users to obey the allocation through a guidance mechanism.

[0058] The third object of the present invention can be achieved by adopting the following technical solutions:

[0059] An intelligent parking system, comprising:

[0060] The processing end is used to collect environmental information of the parking lot to be optimized, analyze parking behavior, and establish a numerical simulation system for the parking lot; under the framework of the DQN deep reinforcement learning algorithm, a parking space intelligent allocation model is established by combining dynamic allocation and delay matching; the parking space intelligent allocation model is substituted into the numerical simulation system for training and case analysis; after the training is completed, the parking space allocation strategy is output;

[0061] The parking lot management server is used to obtain the parking space allocation strategy;

[0062] A detection device, used to collect environmental information and vehicle information of the parking lot to be optimized;

[0063] Mini-programs / APPs are used to provide parking space allocation information and parking navigation, and motivate users to comply with allocations through guidance mechanisms.

[0064] The fourth object of the present invention can be achieved by adopting the following technical solutions:

[0065] A computer device includes a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the above-mentioned intelligent dynamic allocation and guidance method of parking spaces is implemented.

[0066] The fifth object of the present invention can be achieved by adopting the following technical solutions:

[0067] A storage medium stores a program, and when the program is executed by a processor, the above-mentioned parking space intelligent dynamic allocation and guidance method is implemented.

[0068] The present invention has the following beneficial effects compared with the prior art:

[0069] (1) This embodiment provides a design for intelligent parking space allocation and guidance, which combines innovative functions such as dynamic parking space allocation, parking path navigation, and user compliance guidance mechanism, and uses WeChat mini-programs as interactive tools. It can substantially improve parking lot turnover rate and guide users to quickly reach the optimal parking space, significantly improving user parking efficiency and satisfaction.

[0070] (2) This embodiment combines dynamic parking space allocation with intelligent delayed matching strategy, and realizes the combination of the two in the DQN algorithm framework for the first time, enriching existing research and providing a solution to the parking space allocation problem in the intelligent parking process;

[0071] (3) Compared with the heuristic algorithm adopted in the existing research, the deep reinforcement learning algorithm in this embodiment has stronger intelligence, adaptability, and foresight in the parking space allocation problem, as well as faster decision-making speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0073] Figure 1 This is a flow chart of the method for intelligent dynamic allocation and guidance of parking spaces according to the first embodiment of the present invention.

[0074] Figure 2 This is an operation flow chart of the parking lot numerical simulation system according to the first embodiment of the present invention.

[0075] Figure 3 Schematic diagram of the neural network structure in the DQN algorithm of the first embodiment of the present invention.

[0076] Figure 4 Schematic diagram of the application of the DQN algorithm strategy of the first embodiment of the present invention.

[0077] Figure 5 This is a training graph of the average reward of the model according to the first embodiment of the present invention.

[0078] Figure 6 This is a training diagram of the model loss function of the first embodiment of the present invention.

[0079] Figure 7 4 is a comparison chart of the parking turnover rate of the first embodiment of the present invention.

[0080] Figure 8 It is a comparison chart of the average parking time of the first embodiment of the present invention.

[0081] Fig. 9 Schematic diagram of application of the system at various stages according to the first embodiment of the present invention.

[0082] Fig.10 This is a diagram of the parking path navigation implementation process in the early application stage of the first embodiment of the present invention.

[0083] Fig.11 This is a diagram of the user compliance allocation guidance mechanism in the early application stage of the first embodiment of the present invention.

[0084] Fig.12 It is a structural block diagram of a parking space intelligent dynamic allocation and guidance device according to a second embodiment of the present invention.

[0085] Fig.13 FIG. 4 is a structural block diagram of a computer device according to a third embodiment of the present invention. DETAILED DESCRIPTION

[0086] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0087] First embodiment:

[0088] like Figure 1 As shown, this embodiment provides a method for intelligent dynamic allocation and guidance of parking spaces, the method comprising the following steps:

[0089] (1) Collect environmental information of the parking lot to be optimized, analyze parking behavior, and establish a numerical simulation system for the parking lot.

[0090] In step (1), the environmental information of the parking lot includes the number information, distribution information, occupancy status, charging pile configuration information, vehicle energy type, vehicle size type and time information of parking spaces. The parking behavior includes vehicle arrival distribution and parking duration distribution. These statistical features are applied to the parameter settings in the simulation system, so that the simulation system can simulate the parking behavior in the actual parking lot. At the same time, the numerical simulation system can also conveniently adjust parameters and variables, such as the number of parking spaces, vehicle arrival and parking duration distribution, so that the parking space allocation model can be trained accordingly according to different actual parking lot environments.

[0091] like Figure 2 As shown in FIG. 1 , the operation process of the parking lot numerical simulation system includes the following steps:

[0092] a. System initialization;

[0093] b. Generate parking demand, where vehicle arrival follows Poisson distribution and parking duration follows normal distribution;

[0094] c. Agent interaction: the system allocates the best parking space for the vehicle through reasonable delay waiting or directly according to the real-time environmental information and intelligent delay matching strategy;

[0095] d. System update, repeat the above steps until the simulation ends.

[0096] By analyzing the actual parking lot data and referring to relevant literature, the statistical characteristics of vehicle arrival and parking time are obtained. This includes the statistical distribution of the time intervals between vehicle arrivals in the actual parking lot, the distribution of parking time, and the flow changes during peak and trough periods. These statistical characteristics are applied to the parameter settings in the simulation system, so that the simulation system can simulate the parking behavior in the actual parking lot.

[0097] The numerical simulation system can also conveniently adjust parameters and variables, such as the number of parking spaces, vehicle arrival and parking duration distribution, so that the parking space allocation model can be trained accordingly according to different actual parking lot environments.

[0098] The parameter settings of the parking lot numerical simulation system of this embodiment are shown in Table 1:

[0099] Table 1 Parking lot numerical simulation parameters

[0100]

[0101]

[0102] (2) Under the DQN deep reinforcement learning algorithm framework, a parking space intelligent allocation model is established by combining dynamic allocation and intelligent delay matching.

[0103] In step (2), the Markov decision process of the parking space intelligent allocation model is as follows:

[0104] 1) Agent: The parking system manager is set as an agent to allocate the best parking space for each arriving parking demand.

[0105] 2) State: The state is the parking environment information and vehicle information that the agent can observe at every moment. The environment information includes the parking space occupancy state X occ , charging pile distribution information X cha , both of which use One-hot encoding, and the actual time Vehicle information including the vehicle's energy type Size Type and the accumulated delay time t represents time. The above together constitute the state space:

[0106]

[0107] 3) Including the allocation of parking space numbers and whether to adopt delayed matching. The intelligent delayed matching strategy refers to obtaining better parking resources through appropriate delayed waiting. In order to simplify the action space, the two dimensions are innovatively merged into one dimension:

[0108] A t ={n}, n∈[0, 2*N-1]

[0109] Among them, A t represents the action space, n is the action value selected by the agent, and N is the number of parking spaces in the parking lot;

[0110] When A t <N, then the corresponding vehicle is delayed matched;

[0111] When A t > N, then the corresponding vehicle is directly matched, and the matching parking space number is (A t -N).

[0112] 4) Reward function, as follows:

[0113]

[0114] Among them, R t represents the reward function, r park Indicates the reward value corresponding to the parking space distance, r charge Indicates the reward value corresponding to the charging demand, r delay Indicates the reward value corresponding to delayed matching;

[0115]

[0116] Among them, L is the reference distance, the smaller the distance, the greater the reward value, l drive is the driving distance from the entrance to the parking space, l walk is the walking distance from the parking space to the destination, is the ratio of vehicle speed to human walking speed, α is the user's walking willingness coefficient;

[0117]

[0118] in, Indicates the vehicle energy type, Indicates that vehicle i is a tram, Indicates that vehicle i is a fuel truck, a t represents the action value taken by the agent on the vehicle, [at ]=1 means that the parking space assigned by the agent to vehicle i is equipped with a charging station, [a t ] = 0 means that the parking space assigned by the agent to vehicle i does not have a charging station;

[0119]

[0120] 5) State transfer: When a vehicle enters the parking lot, the agent outputs action A as a parking space allocation decision based on the current state S, obtains the corresponding reward R, and finally updates the parking lot environment to the next state S'.

[0121] like Figure 3 As shown in FIG. 1 , the neural network structure design in the DQN algorithm of this embodiment is shown. This embodiment uses a fully connected neural network (FCNN) to estimate the Q function to better handle problems with large state space and action space. The action corresponding to the maximum Q value output in each step is a decision, and the parking space allocation strategy π obtained after the model training is completed θ (s, a) will be stored in the neural network in the form of weight parameters and can be directly put into actual parking lot applications.

[0122] In step (2), if Figure 4 As shown, the framework design of the DQN deep reinforcement learning algorithm (the strategy application of this embodiment in the DQN algorithm) is as follows:

[0123] a. Q value: refers to the expected value of the cumulative reward that can be obtained by taking a certain action in a certain state, calculated by the Bellman equation:

[0124] Q(s,a)=E[R+γ*maxQ(s',a')]

[0125] Among them, W(s, a) is the Q value corresponding to state s and action a, E represents the expected calculation of future cumulative returns, R is the corresponding reward value, γ is the discount factor, and maxQ(s', a') is the maximum Q value of the next state and action.

[0126] b. Neural network structure design: A fully connected neural network (FCNN) is used to estimate the Q function to better handle high-dimensional state space and action space. The action corresponding to the maximum Q value output in each step is the decision, and the parking space allocation strategy π is obtained after the model training is completed. θ (s, a) will be stored in the neural network in the form of weight parameters and can be directly put into actual parking lot applications.

[0127] c. Loss function: It is the optimization goal of the model and is used to guide the parameter update of the model during the training process. By optimizing the loss function, the model will adjust the parameters to maximize the cumulative return, thereby achieving a better decision-making strategy in the dynamic allocation of parking spaces. At the same time, the model will also improve the generalization performance of the model by reducing the difference between the predicted value and the target value. The model uses Mean Squared Error (MSE) as the loss function, and its formula is as follows:

[0128]

[0129] Among them, θ is the model parameter, N is the number of samples, and s i is the input state, a i The action selected by the model, Q(s i , a i ; θ) represents the model for state s i The selected action a i The predicted value, y i is the target value.

[0130] d. Experience Replay: In the problem of dynamic parking allocation, the dimensions of the state space and action space are large, and simply taking an action on the current state to update the weight parameters of the neural network is unstable. Therefore, the experience replay strategy is adopted to randomly extract a certain number of experiences experienced by the agent and store them in the experience replay pool, and randomly extract a certain number of experiences from it for training during training, thereby improving the utilization of data and the stability of training, and helping the agent better learn the optimal actions in different parking environments.

[0131] e. Fixed Q-Target Strategy: In order to avoid the estimation error of the Q function causing training instability and low convergence efficiency, a fixed Q-Target strategy is adopted. The target Q network is introduced and updated from the current Q value function at a certain frequency to reduce the estimation error of the Q value function and improve the stability and convergence efficiency of training.

[0132] f. ε-greedy exploration strategy: In the process of training the dynamic parking space allocation model, the ε-greedy exploration strategy is added, that is, the agent is required to select random actions according to a certain probability, rather than the optimal action corresponding to the maximum Q value. At the same time, the exploration probability will gradually decrease with the number of iterations. This helps the agent avoid falling into the local optimal solution, better explore the environment, and thus train a better parking space allocation strategy. The formula is as follows:

[0133]

[0134] Among them, a t is the action selected at time t, Q(s t , a) is state s t and the Q value of action a; N is the number of parking spaces in the parking lot, random·randint(0, N-1) represents a random integer between [0, N-1], i.e., the number of the corresponding parking space; ε is the exploration rate, which starts at 1.0 and decreases linearly to 0.05 over time.

[0135] The algorithm hyperparameter design of this embodiment is shown in Table 2:

[0136] Table 2 DQN algorithm hyperparameter setting table

[0137]

[0138]

[0139] This embodiment uses the Adam (Adaptive Moment Estimation) trainer to train the DQN model. In the scenario of dynamic parking space allocation, the Adam trainer can adaptively adjust the learning rate according to the historical gradient information of each parameter, and uses the exponential weighted average method to estimate the first and second moments of the gradient. It is efficient in calculation and does not require the storage of a large amount of historical information. It can also better control the direction and amplitude of parameter updates, and improve the stability and generalization ability of the model.

[0140] It should be noted that intelligent delayed matching refers to allocating better parking spaces to vehicles through appropriate delayed waiting. The intelligent delayed matching strategy dynamically selects a reasonable delayed waiting time to match a better parking space based on the real-time parking lot status and vehicle arrival situation, thereby improving the utilization efficiency of the parking lot and user satisfaction.

[0141] (3) The model is substituted into the numerical simulation system for training and case analysis.

[0142] This embodiment uses the PyTorch framework to construct a neural network, and substitutes the model into the numerical simulation system for training and runs it on the cloud server. In each iteration, the simulation step length is 1s, the simulation time is 12 hours of parking operation from 10:00 to 22:00, and the model is iterated and trained 1000 times.

[0143] like Figure 5 As shown in FIG. 1 , the average reward training graph obtained after the training of this embodiment is completed, it can be seen that the reward value curve first oscillates and rises, and finally tends to be flat and stabilized within a certain range. Figure 6As shown in FIG. 1 , the loss function training diagram obtained after the training of this embodiment is completed, it can be seen that the loss function value oscillates and decreases, and finally stabilizes in a low value range. The above all indicate that the model training effect is good.

[0144] After the model training of this embodiment is completed, the intelligent dynamic allocation strategy of parking spaces obtained through training is analyzed by examples and tested and compared with other methods, as follows:

[0145] (3.1) Performance analysis of the intelligent delay matching strategy: This method only adopts the intelligent delay matching strategy during non-peak hours to avoid congestion during peak hours. The average delay waiting time obtained after model training is 4.4s, which is 2.9% lower than the average total parking time of the DQN algorithm that does not adopt the delay matching strategy. Therefore, it can be considered that the application of the intelligent delay matching strategy is effective and achieves the purpose of optimizing operating efficiency and energy saving and emission reduction.

[0146] (3.2) Analysis of average parking reward value: Whether the system efficiency of the parking lot is improved is the focus of close attention of parking lot managers. The current parking space management mode is mostly a random parking mode, which does not interfere with the user's parking space selection and allows users to freely choose parking spaces. The greedy algorithm is a heuristic algorithm based on the principles of "first come first served" and "user optimal", and is the allocation method that best fits human logic. In this embodiment, the three methods of DQN algorithm, greedy algorithm and random parking mode are input into the numerical simulation system for synchronous testing. By comparing the average reward value, it can be seen that although the initial reward value of the DQN algorithm is slightly lower than that of the greedy algorithm, due to the DQN algorithm's grasp of the occupation and change rules of spatiotemporal resources, it is more far-sighted in the allocation strategy, so the overall reward average is 11.6% higher than the greedy algorithm, and significantly higher than the random state of 35.2%. It can be seen that the parking space intelligent dynamic allocation method proposed in the present invention can provide a more profitable parking space selection, and its allocation strategy is more globally optimal and far-sighted.

[0147] (3.3) Parking turnover rate analysis: Parking turnover rate is the average number of times each parking space in a parking lot is used every day, and is an important reference indicator for measuring parking efficiency. In this embodiment, the DQN algorithm, the greedy algorithm, and the random state algorithm are input into the numerical simulation system, and the parking lot turnover rate is tested and compared 100 times synchronously, and a parking turnover rate test comparison chart is obtained, as shown in the figure below. Figure 7 As shown in the figure, the parking lot turnover rate under the parking space allocation mode based on the DQN algorithm is 5.28 times / day / parking space, which is significantly improved by 10.5% and 6.9% compared with the greedy algorithm and random state, respectively. It can be seen that the application of the intelligent dynamic allocation strategy for parking spaces has significantly improved the operating efficiency of the parking lot and the utilization rate of parking spaces from the system level.

[0148] (3.4) Analysis of average parking time: For parking users, personal parking efficiency is the most important concern for users and the most significant focus for improving user satisfaction. The average parking time, including the driving time to the target parking space and the walking time to the mall entrance and exit, is the most intuitive and obvious evaluation indicator of the user's personal efficiency. This embodiment uses four models, namely the DQN algorithm with intelligent delay matching strategy, the DQN algorithm without intelligent delay matching strategy, the greedy algorithm, and the random state. The average parking time is tested and compared 100 times in the numerical simulation system, and the test comparison chart of the average parking time is obtained, as shown in the figure below. Figure 8 As shown. It can be seen that the average total parking time under the DQN algorithm with intelligent delay matching strategy is the shortest, which is 67.59s, 2.9% lower than the DQN algorithm without delay matching strategy, 6.3% lower than the greedy algorithm, and 15.9% lower than the random state. Therefore, the combination of dynamic parking space allocation and intelligent delay matching strategy under the DQN algorithm framework significantly improves the algorithm performance and user parking efficiency.

[0149] (4) After training is completed, the parking space allocation strategy is output and loaded to the parking lot management server.

[0150] The parking space allocation strategy π obtained after the training in this embodiment θ (s, a) is a combination of dynamic allocation of parking spaces and intelligent delayed matching strategy. After model training is completed, it is stored in the neural network in the form of weight parameters θ and output in PyTorch format. The allocation strategy file can be loaded into the parking lot management server to implement intelligent dynamic allocation of parking spaces based on the parking lot environment information collected by sensors and other equipment.

[0151] (5) In the application stage, the WeChat applet is used as the interactive interface to provide parking space allocation information and parking navigation, and a guidance mechanism is added to encourage users to comply with the allocation.

[0152] In step (5), the WeChat applet is an interactive tool for users, which combines functions such as dynamic parking space allocation model, intelligent delayed matching strategy, parking path navigation and compliance allocation guidance mechanism to serve the application and promotion of the scientific management model of intelligent parking space allocation.

[0153] After allocating the best parking space for the arriving vehicle, the present invention will also provide parking path navigation for the vehicle to guide the vehicle to quickly reach the target parking space. Parking navigation requires the pre-import of the parking lot plan and rasterization. For each matching result between the arriving vehicle and the allocated parking space, the Dijkstra algorithm is used to solve its shortest path, and finally the parking path guidance is implemented in the WeChat applet interface.

[0154] Incentivize users to obey the allocation through guidance mechanisms, including:

[0155] When the user complies with the parking space allocation, a positive reward is provided to the user, and the positive reward includes at least one of: parking fee discount and shopping points in the mall;

[0156] When the user does not comply with the parking space allocation, the user is guided by means, wherein the means include at least one of: a program pop-up window prompt, questionnaire feedback, and feedback on the walking time that the user will lose.

[0157] In order to solve the pain points of users in the parking industry such as "difficulty in finding parking spaces", inefficient parking search, low parking lot turnover rate and backward parking resource management mode, this embodiment also provides an intelligent parking system, including:

[0158] The processing end is used to collect environmental information of the parking lot to be optimized, analyze parking behavior, and establish a numerical simulation system for the parking lot; under the framework of the DQN deep reinforcement learning algorithm, a parking space intelligent allocation model is established by combining dynamic allocation and delay matching; the parking space intelligent allocation model is substituted into the numerical simulation system for training and case analysis; after the training is completed, the parking space allocation strategy is output;

[0159] The parking lot management server is used to obtain the parking space allocation strategy;

[0160] A detection device, used to collect environmental information and vehicle information of the parking lot to be optimized;

[0161] Mini-programs / APPs are used to provide parking space allocation information and parking navigation, and motivate users to comply with allocations through guidance mechanisms.

[0162] like Fig. 9 As shown, the parking system includes the following three stages:

[0163] (5.1) In the application preparation stage, the parking lot to be optimized is taken as the target, and a parking space allocation model based on the DQN algorithm is constructed. The intelligent allocation strategy is obtained through numerical simulation system training, and finally loaded to the parking system server for use, so as to dynamically allocate the optimal parking space to the user based on the real-time environment.

[0164] (5.2) In the early application stage, the mobile app is used as the interactive interface to provide users with parking space allocation information and parking route navigation, and the guidance mechanism is used to improve users' compliance with the allocation results.

[0165] (5.3) In the later application stage, it can be directly connected to the autonomous driving vehicle, and the vehicle will automatically drive to the parking space assigned by the system. The autonomous driving will complete the "last mile" of the user's travel, greatly improving the efficiency of the user's parking and the parking system.

[0166] like Fig.10The figure shows the parking path navigation implementation process of this embodiment in the early application stage. The system needs to import the floor plan of the parking lot to be optimized in advance and rasterize it. After allocating the optimal parking space for each arriving vehicle, the Dijkstra algorithm is used to solve the shortest path from the entrance to the target parking space, and finally the parking path guidance is implemented in the WeChat applet interface.

[0167] like Fig.11 The user compliance allocation guidance mechanism in the early application stage of this embodiment is shown. In the early application stage of the parking space allocation system, it is inevitable that there are users who do not obey the parking space allocated by the system, resulting in limited improvement in the turnover efficiency of the parking system. Therefore, a user compliance guidance mechanism is specially designed in the early application stage to improve the user's compliance. When the user arrives at the parking lot, the system will provide the user with three optimal parking space allocation options, and provide the walking time and parking time corresponding to different target parking spaces. When the user complies with the parking space allocation, after the vehicle arrives at the designated allocated parking space, the user will receive rewards such as parking fee discounts and shopping mall shopping points, thereby positively motivating the user to continue to obey the allocation in the next parking. When the user does not comply with the parking space allocation, the system will prompt the user through a pop-up window of the WeChat applet to feedback the walking time and parking fee discount that the user will lose. At the same time, rewards will be used to guide users to fill in questionnaire feedback, and on this basis, algorithm improvement, trial management regulations and increased publicity and promotion will be adopted to improve user compliance.

[0168] Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above embodiments may be completed by instructing related hardware through a program, and the corresponding program may be stored in a computer-readable storage medium.

[0169] It should be noted that although the method operations of the above embodiments are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0170] Second embodiment:

[0171] like Fig.12 As shown, this embodiment provides a parking space intelligent dynamic allocation and guidance device, which includes a first establishment module 1201, a second establishment module 1202, a training module 1203 and an allocation module 1204. The specific functions of each module are as follows:

[0172] The first establishment module 1201 is used to collect environmental information of the parking lot to be optimized, analyze parking behavior, and establish a numerical simulation system for the parking lot;

[0173] The second building module 1202 is used to build a parking space intelligent allocation model by combining dynamic allocation and delayed matching under the DQN deep reinforcement learning algorithm framework;

[0174] A training module 1203 is used to substitute the parking space intelligent allocation model into the numerical simulation system for training and example analysis;

[0175] The allocation module 1204 is used to output the parking space allocation strategy after training is completed, provide parking space allocation information and parking navigation, and motivate users to obey the allocation through a guidance mechanism.

[0176] Third embodiment:

[0177] like Fig.13 As shown, this embodiment provides a computer device, which includes a processor 1302, a memory, an input device 1303, a display device 1304 and a network interface 1305 connected through a system bus 1301. Among them, the processor 1302 is used to provide computing and control capabilities, and the memory includes a non-volatile storage medium 1306 and an internal memory 1307. The non-volatile storage medium 1306 stores an operating system, a computer program and a database. The internal memory 1307 provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium 1306. When the computer program is executed by the processor 1302, the parking space intelligent dynamic allocation and guidance method of the above-mentioned embodiment 1 is implemented as follows:

[0178] Collect environmental information of the parking lot to be optimized, analyze parking behavior, and establish a numerical simulation system for the parking lot;

[0179] Under the framework of DQN deep reinforcement learning algorithm, a parking space intelligent allocation model is established by combining dynamic allocation and delayed matching;

[0180] Substituting the parking space intelligent allocation model into the numerical simulation system for training and example analysis;

[0181] After training is completed, the parking space allocation strategy is output, parking space allocation information and parking navigation are provided, and users are encouraged to obey the allocation through a guidance mechanism.

[0182] Fourth embodiment:

[0183] This embodiment provides a storage medium, which is a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the parking space intelligent dynamic allocation and guidance method of the above embodiment 1 is implemented as follows:

[0184] Collect environmental information of the parking lot to be optimized, analyze parking behavior, and establish a numerical simulation system for the parking lot;

[0185] Under the framework of DQN deep reinforcement learning algorithm, a parking space intelligent allocation model is established by combining dynamic allocation and delayed matching;

[0186] Substituting the parking space intelligent allocation model into the numerical simulation system for training and example analysis;

[0187] After training is completed, the parking space allocation strategy is output, parking space allocation information and parking navigation are provided, and users are encouraged to obey the allocation through a guidance mechanism.

[0188] It should be noted that the computer-readable storage medium of the present embodiment may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0189] In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device, or device. In this embodiment, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program used by or in combination with an instruction execution system, device, or device. The computer program contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0190] The computer readable storage medium can be written in one or more programming languages ​​or a combination thereof to execute the computer program of the present embodiment, and the programming language includes an object-oriented programming language, such as Java, Python, C++, and also includes a conventional procedural programming language, such as C language or a similar programming language. The program can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0191] In summary, the present invention fully considers the complex and dynamically changing electrical, temporal and spatial environment of the parking lot, and can improve the user's parking efficiency, the charging demand satisfaction rate and the parking lot turnover rate; at the same time, it conforms to the development trend of artificial intelligence and intelligent valet parking, and provides a solution to the parking space matching problem in the process of intelligent valet parking.

[0192] The above is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solution and inventive concept of the present invention within the scope disclosed by the present invention, which shall fall within the protection scope of the present invention.

Claims

1. A parking space intelligent dynamic allocation and guidance method, characterized in that: include: Collect environmental information of the parking lot to be optimized, analyze parking behavior, and establish a numerical simulation system for the parking lot; Under the DQN deep reinforcement learning algorithm framework, a parking space intelligent allocation model is established by combining dynamic allocation and delayed matching: The setting process of the Markov decision process of the parking space intelligent allocation model includes: Set the parking system manager as an intelligent agent; Define the state space as follows: Among them, S t represents the state space, t represents the time, X occ Indicates the parking space occupancy status, X cha Indicates the distribution information of charging piles. Indicates the actual time. Indicates the vehicle energy type, Indicates the vehicle size type, Indicates the accumulated delay time of the vehicle; Define the action space as follows: A t ={n},n∈[0,2*N-1] Among them, A t represents the action space, n is the action value selected by the agent, and N is the number of parking spaces in the parking lot; When A t <N, then the corresponding vehicle will be delayed matched; When A t > N, then the corresponding vehicle is directly matched, and the matching parking space number is (A t -N); Define the reward function as follows: Among them, R t represents the reward function, r park Indicates the reward value corresponding to the parking space distance, r charge Indicates the reward value corresponding to the charging demand, r delay Indicates the reward value corresponding to delayed matching; Where L is the reference distance, l drive is the driving distance from the entrance to the parking space, l walk is the walking distance from the parking space to the destination, is the ratio of vehicle speed to human walking speed, α is the user's walking willingness coefficient; in, Indicates the vehicle energy type, Indicates that vehicle i is a tram, Indicates that vehicle i is a fuel truck, a t represents the action value taken by the agent on the vehicle, [a t ]=1 means that the parking space assigned by the agent to vehicle i is equipped with a charging station, [a t ] = 0 means that the parking space assigned by the agent to vehicle i does not have a charging station; Set up the state transfer process: When a vehicle enters the parking lot, the agent outputs action A as a parking space allocation decision based on the current state S, obtains the corresponding reward R, and updates the next state S'; Substituting the parking space intelligent allocation model into the numerical simulation system for training and example analysis; After training is completed, the parking space allocation strategy is output, parking space allocation information and parking navigation are provided, and users are encouraged to obey the allocation through a guidance mechanism.

2. The method for intelligent dynamic allocation and guidance of parking spaces according to claim 1, characterized in that: The setup process of the DQN deep reinforcement learning algorithm framework includes: Set the Q value as follows: Q(s,a)=E[R+γ*maxQ(s',a')] Among them, Q(s, a) is the Q value corresponding to state s and action a, E represents the expected calculation of future cumulative returns, R is the corresponding reward value, γ is the discount factor, and maxQ(s', a') is the maximum Q value of the next state and action; A fully connected neural network is used as the neural network model to estimate the Q function, and the action corresponding to the maximum Q value output in each step is the decision; Define the loss function as follows: Among them, θ is the model parameter, N is the number of samples, and s i is the input state, a i The action selected by the model, Q(s i , a i ; θ) represents the model for state s i The selected action a i The predicted value, y i is the target value.

3. The method for intelligent dynamic allocation and guidance of parking spaces according to claim 2, characterized in that: The setting process of the DQN deep reinforcement learning algorithm framework also includes: Based on the experience experienced by the agent, a first preset number of experiences are randomly selected and stored in an experience replay pool, and a second preset number of experiences are randomly selected from the experience replay pool for training during training; Introducing the target Q network; The agent is required to select random actions according to the preset probability, and the optimal parking space allocation strategy is obtained through training, as shown in the following formula: Among them, a t is the action selected at time t, Q(s t ,a) is state s t and the Q value of action a, random.randint(0,N-1) represents a random integer between [0,N-1], which is the number of the corresponding parking space, p represents the preset probability, and ε is the exploration rate.

4. The method for intelligent dynamic allocation and guidance of parking spaces according to claim 1, characterized in that: The parking navigation includes the shortest parking path guidance based on the Dijkstra algorithm.

5. The method for intelligent dynamic allocation and guidance of parking spaces according to claim 1, characterized in that: The method of motivating users to comply with the allocation through the guidance mechanism includes: When the user complies with the parking space allocation, a positive reward is provided to the user, and the positive reward includes at least one of: parking fee discount and shopping points in the mall; When the user does not comply with the parking space allocation, the user is guided by means, wherein the means include at least one of: a program pop-up window prompt, questionnaire feedback, and feedback on the walking time that the user will lose.

6. A parking space intelligent dynamic allocation and guidance device, characterized in that: The method for realizing the intelligent dynamic allocation and guidance of parking spaces according to any one of claims 1 to 5 comprises: The first establishment module is used to collect environmental information of the parking lot to be optimized, analyze parking behavior, and establish a numerical simulation system for the parking lot; The second building module is used to build a parking space intelligent allocation model by combining dynamic allocation and delayed matching under the DQN deep reinforcement learning algorithm framework; A training module, used to substitute the parking space intelligent allocation model into the numerical simulation system for training and example analysis; The allocation module is used to output the parking space allocation strategy after training, provide parking space allocation information and parking navigation, and motivate users to obey the allocation through a guidance mechanism.

7. An intelligent parking system, characterized in that: include: The processing end is provided with the parking space intelligent dynamic allocation and guidance device as claimed in claim 6, which is used to collect environmental information of the parking lot to be optimized, analyze parking behavior, and establish a numerical simulation system for the parking lot; under the framework of the DQN deep reinforcement learning algorithm, a parking space intelligent allocation model is established by combining dynamic allocation and delay matching; the parking space intelligent allocation model is substituted into the numerical simulation system for training, and example analysis is performed; After training is completed, the parking space allocation strategy is output; The parking lot management server is used to obtain the parking space allocation strategy; A detection device, used to collect environmental information and vehicle information of the parking lot to be optimized; Mini-programs / APPs are used to provide parking space allocation information and parking navigation, and motivate users to comply with allocations through guidance mechanisms.

8. A computer device comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, the parking space intelligent dynamic allocation and guidance method according to any one of claims 1 to 5 is implemented.

9. A storage medium storing a program, characterized in that: When the program is executed by a processor, the intelligent dynamic allocation and guidance method for parking spaces described in any one of claims 1 to 5 is implemented.

Images