Parking decision-making method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of vehicles, and discloses a parking decision-making method and device, electronic equipment and a storage medium. The method comprises the steps of at least collecting user state data and vehicle state data to determine whether a user has a parking intention or not; monitoring traffic environment data in real time so as to determine whether the parking of the user is safe or not based on the current traffic environment data at least after determining that the user has the parking intention; and at least after determining that parking is safe, calling the multi-dimensional knowledge graph, the user state data and the third-party API interface by using the parking decision model to generate a personalized user parking decision. The parking experience of the user is improved, the parking violation risk of the user is reduced, the vehicle use safety of the user is improved, and the user experience is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to a parking decision method and device, electronic equipment and storage medium. BACKGROUND

[0002] At present, intelligent parking technology has developed from single parking space detection to multi-source information fusion stage. According to IEEE ITS-2023 literature, 85% of parking solutions are limited to parking lot scenarios, which is difficult to adapt to complex urban roads. At the same time, the IDC 2024 report shows that the user's personalized demand for parking assistance system increases by 37% per year. However, the existing intelligent parking technology generally lacks multi-source information integration and dynamic learning ability, and cannot meet the user's personalized demand for parking assistance system. SUMMARY

[0003] The present application aims to provide a parking decision method, device, electronic equipment and storage medium, to at least solve the problem of poor adaptability of existing intelligent parking technology in complex road scenarios and insufficient personalized services, so as to improve the overall, intelligent and personalized degree of parking decision method, improve the user's driving safety and protect the user's experience.

[0004] In order to solve the above technical problems, in a first aspect, the present application provides a parking decision method, at least comprising:

[0005] At least collecting user state data and vehicle state data to determine whether the user has a parking intention;

[0006] Real-time monitoring of traffic environment data, to at least determine whether the user's parking is safe based on the current traffic environment data after determining that the user has the parking intention;

[0007] At least after determining that the parking is safe, using a parking decision model to call a multi-dimensional knowledge graph, the user state data and a third-party API interface to generate a personalized user parking decision.

[0008] Optionally, the parking decision model is obtained at least by pre-training processing;

[0009] Wherein, the pre-training processing at least includes:

[0010] Establishing an initial decision model;

[0011] At least based on driving behavior data, the initial parking decision model is trained to perform initial training;

[0012] At least based on user portrait data and vehicle image database, the initial parking decision model after the initial training is adjusted to obtain the parking decision model.

[0013] Optionally, before the personalized user parking decision is generated by calling the multi-dimensional knowledge graph, the user state data and the third-party API interface using the parking decision model after at least determining that the parking is safe, the method further comprises:

[0014] collecting a user parking related knowledge base;

[0015] classifying the user parking related knowledge base from at least a spatial dimension, a time dimension, a user dimension and an environment dimension to generate at least the multi-dimensional knowledge graph.

[0016] Optionally, the personalized user parking decision is generated by calling the multi-dimensional knowledge graph, the user state data and the third-party API interface using the parking decision model after at least determining that the parking is safe, specifically comprising:

[0017] generating an initial parking strategy based on the multi-dimensional knowledge base, the user state data and the third-party API interface after at least determining that the parking is safe;

[0018] calculating a reward value of each available sub-strategy in the initial parking strategy based on a user satisfaction, a strategy overall efficiency, a strategy compliance index value and a strategy safety index value;

[0019] generating the personalized user parking decision based on the reward values of all the available sub-strategies.

[0020] Optionally, the reward value is confirmed at least by the following way:

[0021] ;

[0022] In the formula, R represents the reward value, U represents the user satisfaction, E represents the strategy working efficiency, C represents the strategy compliance index value, S represents the strategy safety index value, and α1, α2, α3 and α4 represent the weights of the user satisfaction, the strategy working efficiency, the strategy compliance index value and the strategy safety index value in sequence.

[0023] Optionally, the user state data at least comprises one of user eye data, user voice data, user age data, user gender data and user gesture data;

[0024] The vehicle state data at least comprises one of steering wheel rotation angle data, acceleration data and gear data;

[0025] The road traffic data at least comprises one of traffic sign data and road environment data.

[0026] In a second aspect, the present application further provides a parking decision device, at least comprising:

[0027] a data collection module configured to collect at least user state data and vehicle state data to determine whether the user has a parking intention;

[0028] a safety determination module configured to monitor traffic environment data in real time to determine whether it is safe for the user to park based on the current traffic environment data at least after determining that the user has the parking intention;

[0029] a parking prompt module configured to call a multi-dimensional knowledge graph, the user state data and a third-party API interface by using a parking decision model to generate a personalized user parking decision at least after determining that it is safe to park.

[0030] Optionally, the parking decision model is obtained at least by pre-training processing.

[0031] The pre-training processing at least includes:

[0032] establishing an initial decision model;

[0033] performing initial training on the initial parking decision model based at least on driving behavior data;

[0034] adjusting the initial decision model after the initial adjustment training to obtain the parking decision model based at least on user portrait data and a vehicle-mounted image database.

[0035] In a third aspect, the present application further provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and wherein the processor implements the steps of the parking decision method according to any one of the first aspect when executing the program.

[0036] In a fourth aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the parking decision method according to any one of the first aspect when executed by a processor.

[0037] The technical solution provided by the embodiments of the present application first collects at least user state data and vehicle state data to determine whether the user has a parking intention, then monitors traffic environment data in real time to determine whether it is safe for the user to park based on the current traffic environment data at least after determining that the user has the parking intention, and finally calls a multi-dimensional knowledge graph, the user state data and a third-party API interface by using a parking decision model to generate a personalized user parking decision at least after determining that it is safe to park.

[0038] It can be seen that the embodiment of the application proposes a personalized recommendation mechanism to improve the dynamic learning ability of parking decision, and after confirming that the user has a parking intention and the user is safe in parking, the multi-dimensional knowledge base, user state data and third party API interface are called by using the parking decision model to improve the dynamic learning ability of parking decision, and personalized parking decision is provided for the user. The embodiment of the application at least solves the problems of poor adaptability, insufficient personalized service and lack of multi-source information integration of the existing intelligent parking technology in a complex road scene, is beneficial to improving the user parking experience, reducing the user parking violation risk, improving the user vehicle safety and protecting the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is a flowchart of a parking decision method provided by the embodiment of the application;

[0040] Figure 2 is a flowchart of another parking decision method provided by the embodiment of the application;

[0041] Figure 3 is a parking decision system architecture diagram provided by the embodiment of the application;

[0042] Figure 4 is a structural schematic diagram of a parking decision device provided by the embodiment of the application;

[0043] Figure 5 is a structural schematic diagram of an electronic device provided by the embodiment of the application. DETAILED DESCRIPTION

[0044] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0045] The terms used in the embodiments of the present application are only for the purpose of describing the specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. "Multiple" generally includes at least two.

[0046] It should be understood that the term "and / or" used herein is only to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects.

[0047] It should be understood that, although the terms first, second, third, etc. can be employed in this application, these are merely used to differentiate one element from another, and do not connote any order or priority. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of the present application.

[0048] Depending on the context, the word "if" as used herein can be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting." Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]."

[0049] It is also to be noted that the terms "comprising", "including", and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0050] It is particularly noted that symbols and / or numbers present in the specification, if not marked in the description of the drawings, are not drawing reference numbers.

[0051] As mentioned in the background, the existing intelligent parking technology generally lacks dynamic learning ability and cannot provide personalized services for users. The applicant analyzes the intelligent parking technology related patents applied by the current well-known companies, and the specific analysis is as follows:

[0052] (1) The technical solution disclosed in patent number "CN118824043A" is to perform time series analysis and pattern recognition processing on target traffic data sets to obtain traffic volume prediction data and parking demand prediction data. Further multi-objective optimization processing is performed on the parking demand prediction data to obtain target decision data, and multi-channel information format conversion and priority sorting processing is performed to generate real-time information data suitable for different publishing platforms. The user parking behavior data, real-time traffic condition data, and parking lot occupancy rate data are analyzed and processed to obtain candidate parking recommendation path data. Through multi-dimensional cross analysis of user feedback data and occupancy rate data, evaluation index data is obtained to correct the candidate parking recommendation path data, and finally the target parking recommendation path data is obtained. However, the above-mentioned scheme has the following problems: on the one hand, the environmental perception is limited, mainly relying on traffic data sets, and the identification of road key information such as no-parking signs is insufficient, which cannot provide comprehensive parking environment information. On the other hand, the personalized recommendation is insufficient, and the user historical data cannot be fully utilized, making it difficult to accurately grasp the user's diverse needs and respond to the user's real-time parking needs in a timely manner, lacking real-time performance.

[0053] (2) The technical solution disclosed in patent number "CN119296361A" is to obtain environmental data collected by the current vehicle, update the real-time parking space status information of the current parking lot using these data, and share the updated parking space status information. However, this scheme has the following problems: on the one hand, the parking assistance function is single, only updating the parking space status, lacking key functions such as personalized parking recommendation and road parking judgment. On the other hand, this scheme cannot fully integrate multi-source information, and cannot provide comprehensive parking information services for users.

[0054] (3) The technical solution disclosed in patent number "CN119495207A" includes a database module, an image capture module, a vehicle detection and tracking module, a vehicle cutting module, and a feature matching module. Through the cooperative action of these modules, vehicle perception data, parking space information, camera images, and other key information are captured and analyzed to provide data support for parking lot managers to realize real-time updating and intelligent management of vehicle and parking space information in the parking lot. However, this scheme lacks external road parking legality judgment, mainly focusing on the internal management of underground parking lots, and cannot provide effective help for user parking decisions when on external roads. Moreover, the parking recommendation is not comprehensive, only providing parking lot guidance, and cannot provide comprehensive parking recommendations for users based on their travel needs and external traffic conditions.

[0055] In summary, the existing intelligent parking technology related patents have obvious deficiencies in environmental perception, personalized service, multi-source data integration, etc. The inventors propose the following solutions to address the above technical deficiencies:

[0056] Figure 1 is a flowchart of a parking decision method provided by an embodiment of the present application. The parking decision method can be executed by a parking decision device as an execution subject in the embodiment of the present application, but is not limited thereto. The execution subject can be implemented in the form of software and / or hardware. As shown in the figure, the parking decision method at least includes the following steps: Figure 1

[0057] S1, collecting at least user state data and vehicle state data to determine whether the user has a parking intention.

[0058] The parking intention can refer to the user looking for a parking space or preparing to perform a parking operation. For example, when the user slows down and approaches the roadside, the steering wheel is correspondingly turned, and the voice mentions parking-related content, it can be determined that the user has a parking intention. It can be understood that the parking intention can be determined by a driver monitoring system (DMS).

[0059] In a specific embodiment, optionally, the user state data at least includes one of user eye data, user voice data, user age data, user gender data, and user gesture data; and the vehicle state data at least includes one of steering wheel angle data, acceleration data, and gear data.

[0060] It can be understood that after obtaining the user state data and the vehicle state data, the data of different sources can be time-aligned, and the time sequence of the user state data and the vehicle state data can be processed by using a sliding window mechanism. Further, the user state data and the vehicle state data can be unified into the same coordinate system by using a Kalman filtering algorithm, so as to improve the accuracy and reliability of the data.

[0061] S2, monitoring traffic environment data in real time, so as to determine whether the user parking is safe based on the current traffic environment data at least after it is determined that the user has a parking intention.

[0062] The traffic environment data can refer to traffic information that can represent the environmental characteristics of the vehicle. In another specific embodiment, optionally, the road environment data at least includes one of traffic sign data and road environment data.

[0063] It can be understood that when the user has a parking intention, but there is a no-parking sign on the roadside, pedestrians nearby, or heavy traffic affecting parking, it is considered that the current environment is not suitable for parking (i.e., the user parking is not safe), and the car central control can feed back the specific reason for the unsafe parking to the user, such as the no-parking sign.

[0064] ​S3, after determining the parking safety, a multi-dimensional knowledge graph, user state data and third-party API interfaces are called by the parking decision model to generate personalized user parking decisions.

[0065] The parking decision model can be a large model with strong learning ability and adaptability, such as a Pegasus Visual Language Model (VLM), a GPT series model, etc. Through daily training and optimization of the parking decision model, the function of learning user behavior and recommending parking strategies can be realized. In this embodiment, Pegasus VLM is preferred as the parking decision model. The third-party Application Programming Interface (API) can be a navigation API, a search API, a weather API, etc.

[0066] Illustratively, in the process of generating user parking strategies, first, real-time traffic information and path planning information are obtained through the navigation API. Further, weather conditions are obtained through the weather API to determine the parking decision. Under normal weather, the parking decision model directly searches for surrounding parking lots and related information through the search API to generate corresponding parking strategies in combination with the navigation API information. Under abnormal weather (such as thunderstorm, hail weather), the parking decision model automatically adjusts the parking recommendation strategy, and in combination with the navigation API information, preferentially selects indoor parking lots near the navigation route or parking spaces with canopies to ensure the safety and comfort of the user's parking. It can be understood that in actual application, after the user inputs the destination, the parking decision model can obtain real-time traffic congestion through the navigation API, and in combination with the weather information provided by the weather API, the user's historical parking preferences, travel purposes (i.e., user state data) and local parking laws and regulations (i.e., multi-dimensional knowledge graph) to plan the best travel route for the user and recommend suitable parking lots along the way to improve the user's driving comfort.

[0067] In a possible implementation, an intention-aware agent, a risk control agent, an intelligent recommendation agent, and a user interaction agent can also be constructed. The intention-aware agent can be used to call the DMS to determine whether the user has a parking intention. The risk control agent calls the parking decision model to continuously monitor the static and dynamic environment of the traffic, and outputs whether the current environment is suitable for parking and the specific reasons why it is not suitable for parking through the central control, thereby effectively avoiding safety problems caused by the user parking in illegal or unsafe areas, and helping to protect the legal rights and personal safety of the user, while also helping to maintain the traffic order. The intelligent recommendation agent is used to generate a user parking strategy using the parking decision model, and calls the corresponding API to implement the corresponding functions, such as calling the navigation API to obtain route information, calling the search API to find parking lots near the surrounding or destination, and calling the weather API to obtain weather conditions. These APIs and tools can provide multi-dimensional support for real-time recommendation services. The multi-dimensional knowledge base at least includes a traffic regulation knowledge base and a road sign knowledge base, and a historical illegal parking penalty knowledge base. The parking strategy can be a recommended parking location. The user interaction agent is used to receive the output of other agents, complete the rendering and prompting of the user interface, and receive the user's input, and pass the user's feedback information to other agents to achieve good interaction between the system and the user. For example, when the risk control agent determines that the current environment is not suitable for parking, the user interaction agent will remind the user through voice and the car central control screen, and display other recommended suitable parking locations; when the user inputs special requirements for parking through voice or touch screen, the user interaction agent will pass these information to the intelligent recommendation agent to generate a parking strategy that better meets the user's needs.

[0068] It can be understood that the user interaction agent, as a bridge between the system and the user, can ensure accurate transmission and timely feedback of information, improve user experience, and enhance the user's trust and dependence on the system. The arrangement as in this embodiment can accurately determine the user's potential and real parking needs, generate personalized parking strategies for the user, fully consider the user's individual needs and actual situation, provide accurate and thoughtful parking recommendation services, greatly improve the user's parking satisfaction and parking convenience, and save the user's time and cost of searching for parking spaces.

[0069] In another specific implementation, optionally, the parking decision model is obtained at least by pre-training processing;

[0070] The pre-training processing at least includes:

[0071] (301) establishing an initial decision model.

[0072] (302) performing initial training on the initial decision model for parking based at least on driving behavior data.

[0073] In the initial decision model, the first stage training (i.e., step 302) can feed 100,000 sets of driving behavior data into the Pegasus VLM as training data, or divide 100,000 sets of driving behavior data into a training set and a test set in a ratio of 8:2. The training set is used to train the Pegasus VLM. The test set is used to test the trained Pegasus VLM, and the test result can be corrected by manual operation to ensure that the Pegasus VLM adapts to the general logic of the driving scene and the accuracy of recognition. After the above training, the Pegasus VLM can be put into use and has a preliminary parking decision ability. For example, the Pegasus VLM originally only recognizes "the parking line in the image", and the model further associates "the self-vehicle width is 1.9 meters, and the parking width is 2.3 meters" after the first stage training. However, the Pegasus VLM is a general model at this time, and lacks the adaptation ability to specific users and vehicle types.

[0074] (303) Adjusting the initial decision model after the initial training based on at least user portrait data and an in-vehicle image database to obtain a parking decision model.

[0075] The user portrait data can refer to the characteristic information of the user, such as user preferences, gender, driving style, driving experience, etc. The in-vehicle image database can refer to a specific vehicle type, such as camera angle, vehicle body size, etc. The second stage training (i.e., step 303) is mainly aimed at the model after the first stage training. At this time, the Pegasus VLM cannot distinguish the difference between "user A" and "user B". The core of the second stage training is to adapt the model to the individualized characteristics of each user on the premise of protecting privacy. For example, for a user with a more aggressive driving style, the model can adjust the parking strategy according to his driving data and recommend a wider and more convenient parking space.

[0076] Specifically, the training of the model can be divided into a general basic layer (corresponding to the first training stage) and a local fine-tuning layer (corresponding to the second training stage). The user portrait data is only trained on the local device or the vehicle machine, and only updates the parameters of the "individualized fine-tuning layer" (such as the fine-tuning layer of user A remembers that "he likes parking spaces wider than 3 meters"). In one possible implementation, the Pegasus VLM can also be continuously adjusted in a federated aggregation manner. The specific adjustment method can be to upload the parameters of the user's local fine-tuning layer, and send the updated direction of the parameters (such as the need for a wider parking space) to the Pegasus cloud server after encryption, and the server aggregates the updates of all user parameters to optimize the general basic layer. The large model of each user after the above training includes a general basic layer (shared) + a local fine-tuning layer (dedicated), which adapts to both general rules and individual characteristics.

[0077] The technical solution provided by the embodiment firstly collects at least user state data and vehicle state data to determine whether the user has a parking intention, secondly, monitors traffic environment data in real time to determine whether the user can park safely based on the current traffic environment data at least after determining that the user has a parking intention, and finally, at least after determining that the user can park safely, calls a multi-dimensional knowledge graph, user state data and a third-party API interface by using a parking decision model to generate a personalized user parking decision.

[0078] It can be seen that the embodiment proposes a personalized recommendation mechanism to improve the dynamic learning ability of the parking decision, and after confirming that the user has a parking intention and that the user can park safely, the multi-dimensional knowledge base, user state data and third-party API interface are called by using the parking decision model to improve the dynamic learning ability of the parking decision, and a personalized parking decision is provided for the user. The embodiment of the application at least solves the problems of poor adaptability, insufficient personalized service and lack of multi-source information integration of the existing intelligent parking technology in a complex road scene, is beneficial to improving the user parking experience, reducing the user parking violation risk, improving the user vehicle safety and guaranteeing the user experience.

[0079] On the basis of the above embodiment or embodiment, Figure 2 is a flowchart of another parking decision method provided by the embodiment of the application, and the embodiment is added on the basis of the above embodiment. As shown in Figure 2 , the parking decision method at least includes the following steps:

[0080] S1, at least collect user state data and vehicle state data to determine whether the user has a parking intention.

[0081] S2, monitor traffic environment data in real time to determine whether the user can park safely based on the current traffic environment data at least after determining that the user has a parking intention.

[0082] S41, collect a user parking related knowledge base.

[0083] The parking related knowledge base includes but is not limited to traffic regulations in a specific area, fixed traffic signs and historical parking violation cases.

[0084] S42, at least classify the user parking related knowledge base from a spatial dimension, a time dimension, a user dimension and an environment dimension to at least generate a multi-dimensional knowledge graph.

[0085] The space dimension can be GPS, coordinates, etc. The time dimension can be holidays, morning / evening rush hours, whether or not there is a traffic restriction, etc. The user dimension can be the user's driver's license type, driving age, vehicle type of the vehicle driven, etc. The environment dimension can be weather / visibility, etc. In the specific use of the multi-dimensional knowledge graph, the user dimension, the time dimension, the space dimension, and the environment dimension can be retrieved in the order of priority according to the graph neural network matching algorithm, or can be retrieved simultaneously. It can be understood that, as in this embodiment, the local regulations and the actual situation can be fully considered when making a parking decision, and more practical parking suggestions can be provided for the user. For example, in some areas, parking is prohibited on some roads during a certain period of time, at this time, the driver can be reminded to avoid parking on the road during the period of time according to the multi-dimensional knowledge graph information of the time dimension, and the driver can also be recommended a nearby legal parking location in combination with the multi-dimensional knowledge graph information of the space dimension.

[0086] S31, at least after determining that parking is safe, generating an initial parking strategy based on the multi-dimensional knowledge base, the user state data, and the third-party API interface.

[0087] The initial parking strategy can be a plurality of parking location or parking lot information.

[0088] S32, calculating a reward value of each available sub-strategy in the initial parking strategy based on a user satisfaction degree, a strategy work efficiency, a strategy compliance index value, and a strategy safety index value.

[0089] Among them, the available sub-strategy can be a parking location or parking lot information that meets the user's needs. User satisfaction mainly reflects the user's subjective feeling of the parking service, including the satisfaction of the recommended parking lot, the interactive experience satisfaction, etc. In the model training stage, it is mainly calculated by user feedback survey, behavior analysis in the interaction process (such as user's click, confirmation and other operations on the recommended results), and long-term user evaluation data. In the training process, a scoring system is adopted, for example, the user's satisfaction with each parking service is scored from 1 to 5, and then the weighted average of multiple scores is obtained. The standard value of user satisfaction is set to 1. In actual use, if the user satisfaction is not less than the standard value, the user satisfaction is set to 1. If it is less than the standard value, the output user satisfaction can be appropriately reduced. The value range of user satisfaction is always between [0, 1]. The user satisfaction output by the model can also be manually corrected in the model test until the user satisfaction output by the model meets the user's needs. The strategy work efficiency mainly reflects the speed of completing the parking assistance task, covering the total time from intent perception to parking strategy generation. The total time from starting to perceive the user's parking intention to finally completing the parking recommendation and interacting with the user is recorded, and then the overall work efficiency is obtained by normalizing according to the set efficiency standard. It can be understood that if the actual time is less than or equal to the efficiency standard, the overall work efficiency can be set to 1. If the actual time exceeds the efficiency standard, the score is deducted according to the proportion of the excess. The value range of the strategy work efficiency is always between [0, 1]. The strategy compliance index value mainly considers whether the traffic regulations, parking lot regulations, and whether there is a situation of illegal guidance in the parking recommendation process. In specific implementation, based on the multi-dimensional knowledge graph related to traffic regulations and parking lot regulations, the recommended parking lot is checked one by one. If the recommended parking lot completely meets the relevant regulations, the strategy compliance index value can be 1. If there is a violation, the score is deducted according to the severity of the violation, for example, a minor violation can deduct 0.2 points, and a serious violation can deduct 0.5 points. The value range of the strategy compliance index value is always between [0, 1]. The strategy safety index value mainly focuses on the safety factors in the parking process, such as whether there are safety hazards around the recommended parking lot, whether the parking location will pose a safety threat to traffic and pedestrians, etc. In specific implementation, the environment around the recommended parking lot can be evaluated by combining user state data, traffic environment data, and spatial dimension data in the multi-dimensional knowledge graph. For example, check whether there is good lighting around the parking lot, whether there are traffic accident-prone points, etc. If the environment around the parking lot is safe, the strategy safety index value is 1. If there are safety hazards, the score is deducted according to the severity of the hazard. The value range of the strategy safety index value is always between [0, 1].

[0090] It can be known that, in another specific embodiment, the reward value is at least confirmed by the following means:

[0091] ;

[0092] In the formula, R represents the reward value, U represents the user satisfaction, E represents the overall work efficiency, C represents the compliance index value, S represents the safety index value, and a1, a2, a3, and a4 represent the weights of the user satisfaction, the policy work efficiency, the policy compliance index value, and the policy safety index value, respectively.

[0093] The sum of a1, a2, a3, and a4 is equal to 1. The user can also make partial adjustments according to actual needs and scene characteristics. For example, in the urban center area, the policy compliance index value and the policy safety index value may be more important, and the values of a3 and a4 can be appropriately increased.

[0094] S33, based on the reward value of all available sub-policies, generates a personalized user parking decision.

[0095] The calculation formula of the reward value can be understood as a kind of collaborative reward function, which can be divided into the following five stages in specific use:

[0096] The data collection stage is used to collect user satisfaction data, system working time data, compliance data of recommended parking lots, and safety data in real time during system operation.

[0097] The index calculation stage is used to process and calculate the collected data periodically (such as every day, every week) to obtain the specific values of U, E, C, and S of each parking space or parking lot.

[0098] The weight adjustment stage is used to reasonably adjust the values of the weight coefficients (i.e., a1, a2, a3, and a4) according to different application scenarios and needs. For example, in the business district, due to the shortage of parking resources, users may pay more attention to the timeliness and accuracy of the recommendations, and the values of a1 and a2 can be appropriately increased. In areas such as schools and hospitals where safety requirements are high, the value of a4 can be increased.

[0099] The reward calculation stage is used to substitute the calculated index values and the adjusted weight coefficients into the collaborative reward function, and calculate the reward value R of each parking decision.

[0100] The model optimization stage is used to optimize and adjust each agent in the system according to the size of the reward value R. For example, if the R value of a parking decision is low, it means that the parking decision has some deficiencies in some aspects, and the algorithm optimization or parameter adjustment can be performed on the corresponding agent (such as the intention perception agent, the intelligent recommendation agent, etc.) to improve the overall performance of the system. Through continuous cyclic optimization, the system can better meet the user needs and provide better parking assistance services.

[0101] Figure 3 is a parking decision system architecture provided by an embodiment of the present application, as shown in Figure 3

[0102] In another possible implementation, when the intention perception agent detects that the vehicle is slowing down and turning to the roadside, the steering wheel is turning to the roadside, and the user has relevant parking voice instructions, etc. (i.e. there is a parking intention), the risk control agent starts to work. Further, the risk control agent analyzes the image data collected by the outside camera and radar data, etc. using the Pegasus VLM, and if it identifies that the current road is a no-parking road section (such as a no-parking sign), the user interaction agent will alert the user through voice and screen. Further, the intelligent recommendation agent starts to work, and according to the user's feedback data on the timeliness of the no-parking warning and the satisfaction of the recommended parking lot, the time from the perception of the parking intention to the completion of the recommendation, the compliance of the recommended parking lot, and the safety of the surrounding area of the recommended parking lot, the parking lot A reward value is calculated as 0.8, the parking lot B reward value is 0.7, and the parking lot C reward value is 0.6. Further, the user interaction agent alerts the user that the front road section prohibits parking, please go to the nearby A parking lot to park, and at the same time displays the location and navigation route of the A parking lot on the screen.

[0103] In another possible implementation, when the vehicle drives to the vicinity of a parking lot, the intention perception agent determines that the user has the intention to enter the parking lot. However, the intention perception agent detects that the driver is not skilled enough in parking operations (poor parking ability) through DMS at this time, and determines that the parking space is complex or it is difficult to get out of the car after parking through the outside camera and radar data. Further, the intelligent recommendation agent recommends that the user turn on the automatic parking function and calculates the optimal parking space based on the cooperative reward function. Further, the central control screen prompts "It is detected that the current parking environment is complex, it is recommended to turn on the automatic parking function". Further, the user clicks to confirm, and the system starts the automatic parking program.

[0104] In another possible implementation, if the user's navigation destination is a shopping mall, and through analysis of the user's historical parking data, it is found that the user prefers parking spaces closer to the shopping mall, the intelligent recommendation agent will preferentially recommend parking spaces closer to the shopping mall. Further, if there is a parking discount activity in a certain parking lot during the current period (obtained through a third-party API), and the traffic of the parking lot is relatively small and safer, the intelligent recommendation agent will also provide it as a recommended option to the user.

[0105] ​It can be seen that, on the one hand, the embodiment provides a multi-modal perception system, which confirms the user's parking intention and the user's parking safety, and uses a parking decision model to call a multi-dimensional knowledge base, user state data and a third-party API interface to improve the dynamic learning ability of the parking decision, and provides personalized parking decisions for the user. The embodiment at least solves the problems of poor adaptability, insufficient personalized service and lack of multi-source information integration of existing intelligent parking technology in complex road scenes, which is beneficial to improving the user's parking experience, reducing the user's parking violation risk and improving the user's parking satisfaction. On the other hand, the embodiment integrates vehicle and user input data (such as eye contact, voice, steering wheel angle, etc.) and environmental input data (images, sounds, etc.) to provide comprehensive and real-time information for the upper intelligent agent and model. On the other hand, the embodiment trains the general basic ability of the parking decision model through a large amount of driving behavior data, so that the model adapts to the general logic of the driving scene, and then realizes personalized fine-tuning through federated learning, so that the model can provide personalized services according to the user portrait, driving data, etc. and protect the user's privacy. On the other hand, the embodiment integrates vehicle image data, external knowledge base data, user data, etc. to improve the model's understanding and decision-making ability for the parking scene, and provides API tools such as navigation, search and weather, providing multi-dimensional support for intelligent recommendation and other functions. On the other hand, the embodiment designs a cooperative reward function including user satisfaction, overall efficiency of strategy, strategy compliance index value and strategy safety index value, adjusts the weight coefficient to adapt to different scene requirements, uses the reward value R to guide system optimization, improves the overall performance, realizes intelligent parking assistance in multiple scenes such as illegal parking risk warning, automatic parking recommendation, parking lot / parking space recommendation and parking money saving assistant, and covers the whole process service before, during and after parking.

[0106] The following scheme is an alternative scheme provided by the embodiment, which can be used to replace one or more steps in the embodiment to complete the technical scheme provided by the embodiment.

[0107] (1) In addition to federated learning, transfer learning can also be used instead of federated learning. Specifically, a basic model is first trained on a large-scale general driving data, and then the user's small amount of personalized data is used to fine-tune the parking decision model, so that the parking decision model quickly adapts to the needs of a specific user. For example, a pre-trained parking decision model is used to perform a small amount of iterative training on the driving style data of a certain user, so that the parking decision model learns the user's preferences.

[0108] (2) A powerful central intelligent agent is used instead of multiple intelligent agents with clear division of labor. The central intelligent agent integrates the functions of all intelligent agents and uniformly processes data and makes decisions. For example, the central intelligent agent is responsible for intention perception, risk control, intelligent recommendation and user interaction at the same time, and completes the parking assistance function through the cooperative work of internal modules.

[0109] (3) Design reward functions with user satisfaction, overall work efficiency, compliance index value, and safety index value as single objectives respectively, and realize overall optimization of the system through alternating optimization or multi-objective optimization algorithms. For example, first optimize user satisfaction, and then optimize work efficiency and other indicators on the basis of meeting user satisfaction.

[0110] (4) Introduce reinforcement learning algorithms, model the interaction process between the system and the environment as a Markov decision process, and through the definition of state, action, and reward function, let the system continuously learn and optimize the strategy in the interaction with the environment. For example, use user feedback and system performance indicators as reward signals to let the system autonomously learn the optimal parking assistance strategy.

[0111] (5) For scenarios such as illegal parking risk warning, a large number of rules can be defined in advance. For example, according to the characteristics of traffic regulations and road signs, rules are written to determine whether the current section is an illegal parking section and trigger the corresponding warning and recommendation functions.

[0112] (6) Build an expert system based on expert knowledge, convert expert experience and knowledge in the parking field into rules and reasoning mechanisms, and realize intelligent parking assistance functions. For example, the expert system can recommend appropriate parking lots and parking spaces based on the driving behavior of the driver and the parking environment, combined with expert knowledge.

[0113] Figure 4 is a structural schematic diagram of a parking decision device provided by an embodiment of the present application. The parking decision device can be realized in software and / or hardware. As shown in the figure, the parking decision device 100 at least includes: Figure 4

[0114] A data acquisition module 110 is configured to acquire at least user state data and vehicle state data to determine whether the user has a parking intention.

[0115] A safety determination module 120 is configured to monitor traffic environment data in real time, and determine whether the user's parking is safe based on the current traffic environment data at least after determining that the user has a parking intention.

[0116] A parking prompt module 130 is configured to utilize a parking decision model to call a multi-dimensional knowledge graph, user state data, and a third-party API interface to generate personalized user parking decisions at least after determining that the parking is safe.

[0117] Optionally, the parking decision model is obtained at least through pre-training processing.

[0118] The pre-training processing at least includes:

[0119] ​establishing an initial decision model, and performing initial training on the initial decision model based on at least driving behavior data, and adjusting the initial decision model based on at least user profile data and a vehicle image database to obtain a parking decision model.

[0120] Optionally, the method further comprises:

[0121] a graph generation module configured to collect a user parking-related knowledge base, and classify the user parking-related knowledge base from at least a spatial dimension, a time dimension, a user dimension, and an environment dimension to generate at least a multi-dimensional knowledge graph.

[0122] Optionally, the parking prompt module 130 is specifically configured to:

[0123] generate an initial parking strategy based on the multi-dimensional knowledge base, user state data, and a third-party API interface after determining that the parking is safe, and calculate a reward value of each available sub-strategy in the initial parking strategy based on user satisfaction, strategy work efficiency, strategy compliance indicator values, and strategy safety indicator values, and generate a personalized user parking decision based on the reward values of all available sub-strategies.

[0124] Optionally, the reward value is determined at least by:

[0125] ;

[0126] wherein R represents the reward value, U represents user satisfaction, E represents strategy work efficiency, C represents strategy compliance indicator values, S represents strategy safety indicator values, and α1, α2, α3, and α4 represent the weights of user satisfaction, strategy work efficiency, strategy compliance indicator values, and strategy safety indicator values, respectively.

[0127] Optionally, the user state data at least includes one of user eye data, user voice data, user age data, user gender data, and user gesture data.

[0128] The vehicle state data at least includes one of steering wheel angle data, acceleration data, and gear data.

[0129] The road traffic data at least includes one of traffic sign data and road environment data.

[0130] The technical scheme provided by the embodiment firstly collects user state data and vehicle state data through the data collection module to determine whether the user has a parking intention, secondly, monitors traffic environment data in real time, and determines whether the user parking is safe based on the current traffic environment data through the safety determination module at least after it is determined that the user has a parking intention, and finally, after it is determined that the parking is safe, the personalized user parking decision is generated by calling the multi-dimensional knowledge graph, the user state data and the third-party API interface through the parking prompt module based on the parking decision model.

[0131] Therefore, the embodiment provides a personalized recommendation mechanism to improve the dynamic learning ability of the parking decision, and after it is determined that the user has a parking intention and the user parking is safe, the multi-dimensional knowledge base, the user state data and the third-party API interface are called through the parking decision model to improve the dynamic learning ability of the parking decision, and the personalized parking decision is provided for the user. The embodiment at least solves the problems of poor adaptability, insufficient personalized service and lack of multi-source information integration of the existing intelligent parking technology in a complex road scene, is beneficial to improving the user parking experience, reducing the user parking violation risk, improving the user vehicle safety and guaranteeing the user experience.

[0132] The embodiment provides an electronic device, Figure 5 is a structural schematic diagram of an electronic device provided by the embodiment of the present application, referring to Figure 5 The electronic device 1000 includes a processor 1001 and a memory 1002, and the memory 1002 stores computer readable instructions, when the computer readable instructions are executed by the processor 1001, the steps in any one of the parking decision methods described above are run. Through the above technical scheme, the processor 1001 and the memory 1002 are interconnected and communicate with each other through a communication bus and / or other forms of connection mechanism (not marked), the memory 1002 stores the computer program executable by the processor, when the electronic device 1000 is running, the processor 1001 executes the computer program to execute the parking decision method in any optional implementation manner of the above embodiment, to at least realize the following functions: at least collect user state data and vehicle state data to determine whether the user has a parking intention; monitor traffic environment data in real time, and determine whether the user parking is safe based on the current traffic environment data at least after it is determined that the user has a parking intention; and at least after it is determined that the parking is safe, the personalized user parking decision is generated by calling the multi-dimensional knowledge graph, the user state data and the third-party API interface through the parking decision model.

[0133] The embodiment provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement a parking decision method provided by all the embodiments of the application: at least collecting user state data and vehicle state data to determine whether a user has a parking intention; monitoring traffic environment data in real time to determine whether the user can park safely based on the current traffic environment data at least after it is determined that the user has the parking intention; and at least after it is determined that the parking is safe, calling a multi-dimensional knowledge graph, the user state data and a third-party API interface by using a parking decision model to generate an individualized user parking decision.

[0134] Any combination of one or more computer readable medium can be employed. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, the computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0135] A computer readable signal medium can include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium can be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0136] Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0137] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0138] Finally, it should be noted that the above-described embodiments are merely intended to illustrate the technical solutions of the present application, and are not intended to limit the present application; even though the present application has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the above-described embodiments, or make equivalent replacements for some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A parking decision method, characterized in that, At least comprising: At least collecting user state data and vehicle state data to determine whether the user has a parking intention; Real-time monitoring of traffic environment data to determine whether the user can park safely based on the current traffic environment data at least after determining that the user has the parking intention; At least after determining that the parking is safe, a parking decision model is used to call a multi-dimensional knowledge graph, the user state data and a third-party API interface to generate a personalized user parking decision.

2. The parking decision method according to claim 1, characterized in that, The parking decision model is obtained at least by pre-training processing; The pre-training processing at least includes: Establishing an initial decision model; At least based on driving behavior data, performing initial training on the initial decision model; At least based on user portrait data and vehicle image database, adjusting the initial decision model after the initial training to obtain the parking decision model.

3. The parking decision method according to claim 1, characterized in that, Before at least after determining that the parking is safe, a parking decision model is used to call a multi-dimensional knowledge graph, the user state data and a third-party API interface to generate a personalized user parking decision, it further includes: Collecting user parking related knowledge base; At least from the spatial dimension, the time dimension, the user dimension and the environment dimension, the user parking related knowledge base is classified to at least generate the multi-dimensional knowledge graph.

4. The parking decision method according to claim 1, characterized in that, The at least after determining that the parking is safe, a parking decision model is used to call a multi-dimensional knowledge graph, the user state data and a third-party API interface to generate a personalized user parking decision, specifically includes: At least after determining that the parking is safe, based on the multi-dimensional knowledge base, the user state data and the third-party API interface, an initial parking strategy is generated; Based on user satisfaction, strategy work efficiency, strategy compliance index value and strategy safety index value, the reward value of each available sub-strategy in the initial parking strategy is calculated; Based on the reward value of all the available sub-strategies, a personalized user parking decision is generated.

5. The parking decision method according to claim 4, characterized in that, The reward value is at least confirmed by the following way: ； In the formula, R represents the reward value, U represents the user satisfaction, E represents the strategy work efficiency, C represents the strategy compliance index value, S represents the strategy safety index value, and α1, α2, α3 and α4 represent the weights of the user satisfaction, the strategy work efficiency, the strategy compliance index value and the strategy safety index value in turn.

6. The parking decision method of claim 1, wherein, The user state data at least includes one of user eye data, user voice data, user age data, user gender data and user gesture data; The vehicle state data at least includes one of steering wheel angle data, acceleration data and gear data; The traffic environment data at least includes one of traffic sign data and road environment data.

7. A parking decision device, characterized by comprising: At least comprising: A data acquisition module for at least collecting user state data and vehicle state data to determine whether the user has a parking intention; A safety determination module for real-time monitoring of traffic environment data to determine whether the user can park safely based on the current traffic environment data at least after determining that the user has the parking intention; The parking suggestion module is configured to utilize a parking decision model to call a multi-dimensional knowledge graph, the user state data, and a third-party API interface to generate a personalized user parking decision after determining that the parking is safe.

8. The parking decision method of claim 1, wherein, The parking decision model is obtained at least by pre-training processing; The pre-training processing at least includes: establishing an initial decision model, performing initial training on the initial decision model based on driving behavior data, and adjusting the initial decision model after the initial training based on user portrait data and a vehicle-mounted image database to obtain the parking decision model.

9. An electronic device comprising a memory and a processor, said memory storing a computer program operable on said processor, characterized in that, The processor implements the steps in the parking decision method of any one of claims 1 to 6 when executing the program.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the steps in the parking decision method of any one of claims 1 to 6 when executed by the processor.

Images