Parking navigation method, system and equipment based on multiple modes

Abstract

The invention discloses a multi-mode-based parking navigation method, system and equipment, and the method comprises the steps: obtaining a vehicle image of a target vehicle in response to the detection of the target vehicle, and initiating a communication request to the target vehicle; in response to establishment of connection with the target vehicle based on the communication request, obtaining a parking demand of the target vehicle from the target vehicle and / or other equipment connected with the target vehicle; and according to the parking demand of the target vehicle, planning a target parking space and / or a target parking route for the target vehicle, and guiding the target vehicle to park according to the target parking space and / or the target parking route. The parking lot management system at least solves the problems that although an existing parking lot management system can carry out refined management on parking spaces and can distribute the parking spaces for users, the distribution logic of the parking spaces can only be planned based on vehicle paths in the parking lot and the sizes of the parking spaces, and the parking spaces facilitating follow-up activities of the users cannot be distributed for the users according to the requirements of the users.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a multimodal parking navigation method, system and device. Background Art

[0002] With the continuous advancement of urbanization, the number of motor vehicles continues to grow, the pressure on urban traffic is increasing, and the problem of parking difficulties is becoming increasingly prominent. In this context, parking lot management systems have been widely used as an important technical means to solve parking problems. Traditional parking lot management mainly relies on manual management, and there are problems such as manual charging, low parking space utilization, and cumbersome parking process, which affect the operational efficiency and user experience of parking lots. With the rapid development of information technology and Internet of Things technology, intelligent parking management systems have gradually become an effective way to solve this problem. Modern parking lot management systems usually include functional modules such as parking space monitoring, license plate recognition, automatic charging, and vacancy indication. Parking space monitoring technology monitors the status of parking spaces in the parking lot in real time through sensors or cameras, which can effectively grasp the occupancy of parking spaces and improve the utilization efficiency of parking spaces. License plate recognition technology automatically recognizes the license plate information of vehicles entering and leaving, realizes fast entry and exit and charging management, reduces manual intervention, and improves traffic efficiency. The automatic charging system can automatically calculate the fee according to the parking time or parking duration of the owner, and settle the fee through a variety of payment methods, which is convenient and fast. In addition, with the development of wireless communication technology and cloud computing technology, parking lot management systems are gradually developing in the direction of intelligence and integration. The parking system can not only monitor the vehicle information inside and outside the parking lot in real time, but also communicate with other traffic management systems to provide more comprehensive services. For example, car owners can remotely view the information of available parking spaces, make reservations in advance, and pay parking fees through the mobile phone APP, which greatly improves the convenience of parking and user experience.

[0003] Although the existing parking lot management system has made some progress, there are still some problems that need to be solved. Although the existing parking lot management system can manage parking spaces in a refined manner and allocate parking spaces to users, since the existing parking lot management system generally can only rely on image recognition technology to identify vehicle-related information, the parking space allocation logic can only be planned based on the vehicle path and parking space size in the field, and it is impossible to allocate parking spaces to users based on their needs to facilitate their subsequent activities. Summary of the invention

[0004] The present invention provides a parking navigation method, system and device based on multimodality, which at least solves the problem that although the existing parking lot management system can manage parking spaces in a refined manner and allocate parking spaces to users, since the existing parking lot management system generally can only rely on image recognition technology to identify vehicle-related information, the parking space allocation logic can only be planned based on the vehicle path and parking space size in the parking lot, and it is impossible to allocate parking spaces to users according to user needs to facilitate the user's subsequent activities.

[0005] The present invention provides a parking navigation method based on multimodality, comprising:

[0006] In response to detecting a target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle;

[0007] In response to establishing a connection with the target vehicle based on the communication request, obtaining a parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle;

[0008] According to the parking demand of the target vehicle, a target parking space and / or a target parking route is planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route.

[0009] Optionally, in response to detecting a target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle includes:

[0010] In response to detecting the target vehicle, obtaining a vehicle image of the target vehicle;

[0011] Obtaining the model of the target vehicle according to the vehicle image of the target vehicle;

[0012] According to the model of the target vehicle, a corresponding communication protocol is obtained, and a communication request is initiated to the target vehicle according to the corresponding communication protocol.

[0013] Optionally, in response to detecting a target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle includes:

[0014] In response to detecting the target vehicle, obtaining a vehicle image of the target vehicle;

[0015] Obtaining a unique identifier of the target vehicle according to the vehicle image of the target vehicle;

[0016] A communication request is initiated to the target vehicle and / or other devices connected to the target vehicle according to the unique identifier of the target vehicle.

[0017] Optionally, in response to establishing a connection with the target vehicle based on the communication request, obtaining a parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle includes:

[0018] In response to establishing a connection with the target vehicle based on the communication request, requesting target navigation data from the target vehicle and / or other devices connected to the target vehicle, the target navigation data being used to indicate a destination of the target vehicle;

[0019] When the target navigation data is received within a specified time, parsing the target navigation data to obtain a navigation destination pointed to by the target navigation data;

[0020] According to the navigation destination, determining whether the navigation destination is a fuzzy destination or a precise destination, wherein the fuzzy destination is configured as a destination with more than one entity corresponding to the address of the destination, and the precise destination is configured as a destination with only one entity corresponding to the address of the destination;

[0021] When the navigation destination is an accurate destination, the accurate destination is used as the parking requirement of the target vehicle.

[0022] Optionally, in response to establishing a connection with the target vehicle based on the communication request, obtaining a parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle includes:

[0023] In response to establishing a connection with the target vehicle based on the communication request, sending guidance information to the target vehicle and / or other devices connected to the target vehicle, wherein the guidance information is configured to guide a user to describe an intention after parking;

[0024] receiving a user description sent by any one of the target vehicle, other devices connected to the target vehicle, and a user of the target vehicle in response to the guidance information;

[0025] According to the user description, based on a generative large model, the parking demand of the target vehicle is obtained.

[0026] Optionally, obtaining the parking demand of the target vehicle according to the user description based on a generative large model includes:

[0027] According to the user description, obtaining a semantic analysis result of the user description;

[0028] According to the semantic analysis result, the semantic analysis result is input into a generative macro model connected to an embedded database to obtain a recommended destination, wherein the recommended destination is recorded in the embedded database;

[0029] The parking requirement of the target vehicle is obtained according to the recommended destination.

[0030] Optionally, obtaining the parking demand of the target vehicle according to the user description based on a generative large model includes:

[0031] According to the user description, obtaining a semantic analysis result of the user description;

[0032] Obtaining the model of the target vehicle according to the vehicle image of the target vehicle;

[0033] According to the semantic analysis result and the model of the target vehicle, the semantic analysis result and the model of the target vehicle are input into a generative large model connected to an embedded database to obtain a recommended destination, wherein the recommended destination is recorded in the embedded database;

[0034] The parking requirement of the target vehicle is obtained according to the recommended destination.

[0035] Optionally, according to the parking demand of the target vehicle, planning a target parking space and / or a target parking route for the target vehicle, and guiding the target vehicle to park according to the target parking space and / or the target parking route, including:

[0036] According to the parking demand of the target vehicle, obtaining a target parking area corresponding to the parking demand of the target vehicle;

[0037] According to the target parking area, obtaining the availability of the target parking area;

[0038] According to the availability of the target parking area, a target parking space and / or a target parking route are planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route.

[0039] In another aspect, a multimodal parking navigation system includes an information collection platform, a data analysis platform, and a path planning platform;

[0040] The information collection platform is configured as follows:

[0041] In response to detecting a target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle;

[0042] The data analysis platform is configured to:

[0043] In response to establishing a connection with the target vehicle based on the communication request, obtaining a parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle;

[0044] The path planning platform is configured as follows:

[0045] According to the parking demand of the target vehicle, a target parking space and / or a target parking route is planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route.

[0046] On the other hand, a device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the above methods.

[0047] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and a processor executes the computer program to implement any of the above methods.

[0048] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0049] The present invention discloses a parking navigation method, system and device based on multimodality, including: in response to detecting a target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle; in response to establishing a connection with the target vehicle based on the communication request, obtaining the parking demand of the target vehicle from the target vehicle and / or other devices connected to the target vehicle; according to the parking demand of the target vehicle, planning a target parking space and / or a target parking route for the target vehicle, and guiding the target vehicle to park according to the target parking space and / or the target parking route. At least the problem that the existing parking lot management system can manage parking spaces in a refined manner and allocate parking spaces to users is solved, but because the existing parking lot management system can generally only rely on image recognition technology to identify vehicle-related information, the parking space allocation logic can only be planned based on the vehicle path and parking space size in the field, and it is impossible to allocate parking spaces to users according to user needs to facilitate users' subsequent activities. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the specific implementation of the present application or the technical solution in the prior art, the following is a brief introduction to the drawings required for the specific implementation or the prior art description. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, each element or part is not necessarily drawn according to the actual scale.

[0051] Figure 1 A schematic diagram of the process of the multi-modal parking navigation method in this application;

[0052] Figure 2 This is a schematic diagram of the structure of a computer device in this application.

[0053] Markings in the figure: 101 - processor, 102 - communication bus, 103 - network interface, 104 - user interface, 105 - memory.

[0054] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the scheme of the present disclosure, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present disclosure.

[0056] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0057] Example 1

[0058] like Figure 1 As shown, a parking navigation method based on multimodality includes:

[0059] S1. In response to detecting a target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle.

[0060] Optionally, the parking lot entrance and exit gate or other equipment installed at the entrance and exit detects the entrance and exit. When a vehicle is detected entering the gate or the parking lot entrance passage, the vehicle is regarded as a target vehicle.

[0061] Optionally, a parking lot access gate or other equipment installed at the entrance or exit collects a vehicle image of the target vehicle and / or initiates a communication request to the target vehicle.

[0062] Optionally, a communication request is initiated to the target vehicle via one or more protocols such as Bluetooth, Wi-Fi, V2X, NFC or OBD-II.

[0063] Optionally, the vehicle model is identified through the vehicle image, and a suitable communication protocol is selected according to the vehicle model to initiate a communication request to the target vehicle.

[0064] S2. In response to establishing a connection with the target vehicle based on the communication request, obtaining a parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle.

[0065] Optionally, obtaining the parking requirement of the target vehicle from the target vehicle may be obtaining navigation information of the target vehicle from a vehicle computer of the target vehicle, and obtaining the parking requirement according to the navigation information.

[0066] Optionally, obtaining the parking requirement of the target vehicle from the target vehicle may be obtaining navigation information of the target vehicle from other devices connected to the target vehicle, such as a mobile phone, and obtaining the parking requirement based on the navigation information.

[0067] Optionally, obtaining the parking requirement of the target vehicle from the target vehicle may be obtaining an advertisement identifier of the mobile phone from other devices connected to the target vehicle, such as a mobile phone, and obtaining the parking requirement according to the advertisement identifier of the mobile phone.

[0068] Optionally, the parking requirement is configured as a destination where the target vehicle user will perform activities or go after parking.

[0069] S3. According to the parking demand of the target vehicle, a target parking space and / or a target parking route is planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route.

[0070] Optionally, according to the parking demand of the target vehicle, a target parking space and / or a target parking route is planned for the target vehicle according to parking space availability and / or vehicle model.

[0071] Optionally, a target parking route is planned for the target vehicle according to the target parking space.

[0072] Optionally, the method of guiding the target vehicle to park according to the target parking space and / or the target parking route may be to send the parking space navigation information to the vehicle computer of the target vehicle.

[0073] Optionally, the method of guiding the target vehicle to park according to the target parking space and / or the target parking route may be to guide through guidance equipment such as guide lights and guide display screens arranged in the parking lot.

[0074] By adopting the above scheme, the parking requirements of the target vehicle user are obtained by communicating with the target vehicle at the entrance of the parking lot, and the target parking space is planned for the target vehicle according to the parking requirements, so that the user can easily go to his final destination after getting off the car, avoiding the distance between the user's parking place and the user's final destination being too far, and optimizing the user's parking experience. At least it solves the problem that although the existing parking lot management system can manage parking spaces in a refined manner and allocate parking spaces to users, since the existing parking lot management system generally can only rely on image recognition technology to identify vehicle-related information, the parking space allocation logic can only be planned based on the vehicle path and parking space size in the field, and it is impossible to allocate parking spaces to users according to user needs to facilitate users' subsequent activities.

[0075] Example 2

[0076] This embodiment is based on the embodiment 1. S1: in response to detecting a target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle.

[0077] Optionally, the parking lot entrance and exit gate or other equipment installed at the entrance and exit detects the entrance and exit. When a vehicle is detected entering the gate or the parking lot entrance passage, the vehicle is regarded as a target vehicle.

[0078] Optionally, a parking lot access gate or other equipment installed at the entrance or exit collects a vehicle image of the target vehicle and / or initiates a communication request to the target vehicle.

[0079] Optionally, a communication request is initiated to the target vehicle via one or more protocols such as Bluetooth, Wi-Fi, V2X, NFC or OBD-II.

[0080] Optionally, the vehicle model is identified through the vehicle image, and a suitable communication protocol is selected according to the vehicle model to initiate a communication request to the target vehicle.

[0081] Optionally, in response to detecting the target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle includes:

[0082] In response to detecting the target vehicle, obtaining a vehicle image of the target vehicle;

[0083] According to the vehicle image of the target vehicle, obtaining the model of the target vehicle;

[0084] According to the model of the target vehicle, a corresponding communication protocol is obtained, and a communication request is initiated to the target vehicle according to the corresponding communication protocol.

[0085] Optionally, the method for obtaining the model of the target vehicle according to the vehicle image of the target vehicle may be: according to the vehicle image of the target vehicle, recognizing the vehicle image by a preset image recognition model to obtain the model of the target vehicle;

[0086] It can also be: based on the vehicle image of the target vehicle, the license plate number in the vehicle image is identified through an image recognition model with a preset value, and the model of the target vehicle is obtained in the database based on the identified license plate number. This method is applicable to the situation where the license plate number has been registered in the parking lot management system.

[0087] Optionally, in response to detecting the target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle includes:

[0088] In response to detecting the target vehicle, obtaining a vehicle image of the target vehicle;

[0089] Obtaining a unique identifier of the target vehicle based on the vehicle image of the target vehicle;

[0090] A communication request is initiated to the target vehicle and / or other devices connected to the target vehicle based on the unique identifier of the target vehicle.

[0091] Optionally, the unique identifier may be a license plate number, and a communication request may be initiated to the target vehicle and / or other devices connected to the target vehicle based on the unique identifier of the target vehicle. Information of the target vehicle and / or other devices connected to the target vehicle may be obtained from a local database or a cloud database based on the license plate number of the target vehicle, and a communication request may be initiated to the target vehicle and / or other devices connected to the target vehicle based on the information.

[0092] S2. In response to establishing a connection with the target vehicle based on the communication request, obtaining a parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle.

[0093] Optionally, obtaining the parking requirement of the target vehicle from the target vehicle may be obtaining navigation information of the target vehicle from a vehicle computer of the target vehicle, and obtaining the parking requirement according to the navigation information.

[0094] Optionally, obtaining the parking requirement of the target vehicle from the target vehicle may be obtaining navigation information of the target vehicle from other devices connected to the target vehicle, such as a mobile phone, and obtaining the parking requirement based on the navigation information.

[0095] Optionally, obtaining the parking requirement of the target vehicle from the target vehicle may be obtaining an advertisement identifier of the mobile phone from other devices connected to the target vehicle, such as a mobile phone, and obtaining the parking requirement according to the advertisement identifier of the mobile phone.

[0096] Optionally, the parking requirement is configured as a destination where the target vehicle user will perform activities or go after parking.

[0097] Optionally, in response to establishing a connection with the target vehicle based on the communication request, obtaining a parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle includes:

[0098] In response to establishing a connection with the target vehicle based on the communication request, requesting target navigation data from the target vehicle and / or other devices connected to the target vehicle, the target navigation data being used to indicate a destination of the target vehicle;

[0099] When the target navigation data is received within the specified time, the target navigation data is parsed to obtain the navigation destination pointed to by the target navigation data;

[0100] According to the navigation destination, determining whether the navigation destination is a fuzzy destination or a precise destination, wherein the fuzzy destination is configured as a destination with more than one entity corresponding to the address of the destination, and the precise destination is configured as a destination with only one entity corresponding to the address of the destination;

[0101] When the navigation destination is an accurate destination, the accurate destination is used as the parking requirement of the target vehicle.

[0102] Specifically, the precise address may be information such as a store name or a house number. By searching the precise address in the database, a unique corresponding entity may be obtained.

[0103] With the above solution, when the user's navigation destination is a specific entity, such as a restaurant or a store, we can assume that the user's main purpose of parking in this parking lot is a restaurant or a store, then the navigation destination is a precise destination, corresponding to the parking demand of the target vehicle; and when the user's navigation destination is a shopping mall or a shopping center corresponding to the parking lot, the navigation destination is a fuzzy destination and it is impossible to determine where the user wants to go after parking, so it is impossible to obtain the parking demand of the target vehicle based on the navigation destination.

[0104] Optionally, in response to establishing a connection with the target vehicle based on the communication request, obtaining a parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle includes:

[0105] In response to establishing a connection with the target vehicle based on the communication request, sending guidance information to the target vehicle and / or other devices connected to the target vehicle, the guidance information being configured to guide the user to describe an intention after parking;

[0106] receiving a user description sent by any one of the target vehicle, other devices connected to the target vehicle, and a user of the target vehicle in response to the guidance information;

[0107] According to the user description, the parking requirements of the target vehicle are obtained based on the generative large model.

[0108] Optionally, when the user's navigation destination is an ambiguous destination, it is necessary to send guidance information to the target vehicle or the user on the target vehicle. The guidance information may be such as "Which store do you want to visit first?", "Which store do you want to park closer to?" or "Are you here to eat or to buy something?", etc. After receiving the guidance information, the user will provide feedback on his or her user description. The user description may include at least the user's response to the guidance information, such as "I want to visit a certain store", "I want to eat at a certain restaurant" or "I want to find a place to eat", etc. Based on the user description, we can use the generative large model to obtain the parking demand of the target vehicle, such as "a certain store", "a certain restaurant", etc.

[0109] Optionally, according to the user description, based on the generative large model, the parking demand of the target vehicle is obtained, including:

[0110] According to the user description, a semantic analysis result of the user description is obtained;

[0111] According to the semantic analysis result, the semantic analysis result is input into the generative large model connected to the embedded database to obtain the recommended destination, and the recommended destination is recorded in the embedded database;

[0112] The parking demand of the target vehicle is obtained according to the recommended destination.

[0113] Optionally, the embedded database stores information on all entities corresponding to the parking lot. When the user's description is more complex, simple keyword extraction can no longer satisfy the analysis of obtaining the user's parking needs. For example, the user's description may be "I want to have a barbecue first, and then watch a movie." We cannot use simple regular matching to obtain the parking needs of the target vehicle as "a certain store" or "a certain restaurant." Instead, we can only use semantic analysis to analyze that the user's destination is "a certain barbecue restaurant (which is close to a movie theater)" or "a movie theater." According to this analysis result, the parking needs of the target vehicle are close to both "a certain barbecue restaurant" and "a movie theater."

[0114] Optionally, according to the user description, based on the generative large model, the parking demand of the target vehicle is obtained, including:

[0115] According to the user description, a semantic analysis result of the user description is obtained;

[0116] According to the vehicle image of the target vehicle, obtaining the model of the target vehicle;

[0117] According to the semantic analysis result and the model of the target vehicle, the semantic analysis result and the model of the target vehicle are input into a generative large model connected to an embedded database to obtain a recommended destination, and the recommended destination is recorded in the embedded database;

[0118] The parking demand of the target vehicle is obtained according to the recommended destination.

[0119] Optionally, using the above method, we can judge the user's consumption level based on the model of the target vehicle, and optimize the accuracy of the analysis when analyzing the user's destination and parking needs based on the user's consumption level.

[0120] Optionally, according to the user description, based on the generative large model, the parking demand of the target vehicle is obtained, including:

[0121] According to the user description, a semantic analysis result of the user description is obtained;

[0122] According to the vehicle image of the target vehicle, the license plate number of the target vehicle is obtained;

[0123] According to the license plate number of the target vehicle, the historical behavior data of the owner of the target vehicle is obtained;

[0124] According to the semantic analysis result and the historical behavior data of the owner of the target vehicle, the semantic analysis result and the historical behavior data of the owner of the target vehicle are input into a generative large model connected to an embedded database to obtain a recommended destination, and the recommended destination is recorded in the embedded database;

[0125] The parking demand of the target vehicle is obtained according to the recommended destination.

[0126] Optionally, the guidance information can be played to the target vehicle through a speaker, and then a user description of the user's voice answer can be received through a microphone arranged near the gate, and voice recognition is performed based on the user description of the user's voice answer to obtain the user description.

[0127] Optionally, the passenger composition of the target vehicle may be analyzed through vehicle images and / or other collected data, such as vehicle weight data obtained through a pressure sensor installed at the gate. Optionally, according to the user description, based on the generative large model, the parking demand of the target vehicle is obtained, including:

[0128] According to the user description, a semantic analysis result of the user description is obtained;

[0129] According to the vehicle image of the target vehicle, obtaining the model of the target vehicle;

[0130] Obtain the passenger composition of the target vehicle according to the vehicle weight data, the vehicle image and the model of the target vehicle;

[0131] According to the semantic analysis result, the model of the target vehicle and the passenger composition of the target vehicle, the semantic analysis result, the model of the target vehicle and the passenger composition of the target vehicle are input into a generative large model connected to an embedded database to obtain a recommended destination, and the recommended destination is recorded in the embedded database;

[0132] The parking demand of the target vehicle is obtained according to the recommended destination.

[0133] Optionally, the method for obtaining the passenger composition of the target vehicle according to the vehicle weight data, the vehicle image and the model of the target vehicle includes:

[0134] According to the vehicle weight data and the model of the target vehicle, the total weight of the passengers of the target vehicle is obtained;

[0135] According to the total weight of the passengers of the target vehicle and the number of some passengers and passenger seats recorded in the vehicle image, the passenger composition of the target vehicle is obtained through a pre-trained prediction model. The passenger composition includes passenger types and the number of passengers of each type.

[0136] Specifically, a method based on multimodal input of user description and vehicle image, using generative big model to match destinations for users, this solution is based on generative big model and knowledge base technology, recommending destinations that meet users' needs and spending power. The system is mainly divided into three modules: user description analysis module, vehicle image analysis module, and destination recommendation module, and realizes multimodal interaction and decision-making through an integrated workflow.

[0137] The system modules of the system for implementing the above method include:

[0138] User description analysis module:

[0139] The input is configured as a user-description input by the user;

[0140] The function is configured to extract users' travel needs, such as shopping, dining, parent-child travel, etc., as well as key information such as budget expectations based on natural language processing technology.

[0141] Vehicle image analysis module:

[0142] The input is configured as a vehicle image;

[0143] The function is configured to identify vehicle brands, models and related features (such as luxury level, price range) through computer vision technology, and infer users' spending power and habits.

[0144] Destination recommendation module:

[0145] The input is configured as user description and vehicle image analysis results;

[0146] The function is configured to select the best recommendations from pre-defined destinations based on user needs, spending power, and preferences through a generative big model and a pre-stored destination knowledge base.

[0147] Knowledge Base Modules:

[0148] The data store is configured to contain information about all predefined destinations, such as price, features, and suitable groups, and supports dynamic updates;

[0149] Use a graph database, such as Neo4j, or an embedded vector database, such as Pinecone, for fast querying and matching.

[0150] Specifically, the underlying technologies of the user description analysis module include:

[0151] Natural language processing models (NLP), such as language models based on the GPT architecture, for intent understanding;

[0152] Keyword extraction and named entity recognition (NER) technology are used to extract details such as travel needs and budget;

[0153] Semantic matching technology, such as using BERT or sentence embedding models, understands the user's ambiguous intent.

[0154] The module is configured to call the language model according to the input user description, convert the text into structured information, such as destination type, budget, etc., and output the semantic analysis result according to the converted structured information.

[0155] The following is pseudo code to implement this method:

[0156] Python

[0157] def analyze_user_intent(user_text):

[0158] #Load NLP model

[0159] model=load_model("gpt-intent-analysis")

[0160] #Extract intent using a generative model

[0161] intent_data=model.generate_structured_data(user_text)

[0162] #Extract key information of intent

[0163] visit_type=intent_data["visit_type"]

[0164] budget=intent_data["budget"]

[0165] preferences=intent_data["preferences"]

[0166] return{"visit_type":visit_type,"budget":budget,"preferences":preferences}.

[0167] Specifically, the underlying technologies of the vehicle image analysis module include:

[0168] Deep learning computer vision models (CV), such as ResNet and EfficientNet, for vehicle image classification;

[0169] Pre-trained car classification models, such as Open Images, YOLO models, or Fine-tuned models, for identifying vehicle brands and models;

[0170] Combine the database to map vehicle models and price ranges to infer user spending power.

[0171] This module is configured to extract features from vehicle images through a deep learning computer vision model, identify the brand and model of the target vehicle using a classification model based on the extraction results, query the vehicle price range and consumption habits from a mapping database based on the brand and model of the target vehicle, and output vehicle analysis results based on the vehicle price range and consumption habits.

[0172] The following is pseudo code to implement this method:

[0173] Python

[0174] def analyze_vehicle_photo(photo):

[0175] #Load the pre-trained vehicle classification model

[0176] vehicle_model=load_model("vehicle-classification")

[0177] # Image preprocessing

[0178] processed_photo=preprocess_image(photo)

[0179] #Classification prediction

[0180] prediction=vehicle_model.predict(processed_photo)

[0181] # Extract vehicle make and model

[0182] brand = prediction["brand"]

[0183] model = prediction["model"]

[0184] #Query the vehicle price range from the database

[0185] price_range=query_vehicle_database(brand,model)

[0186] return{"brand":brand,"model":model,"price_range":price_range}.

[0187] Specifically, the underlying technologies of the destination recommendation module include:

[0188] Generative large models, such as GPT-4, are used to integrate multimodal information and generate recommendation results;

[0189] Embedded databases, such as Pinecone, store destination vectors for efficient similarity retrieval;

[0190] Decision logic: Generate destination recommendations based on user description, spending power, and predefined rules.

[0191] The module is configured to take user descriptions and vehicle analysis results as input, retrieve the knowledge base based on the input user descriptions and vehicle analysis results, match destinations that match user needs and capabilities, and use a generative model to generate personalized recommendation copy.

[0192] The following is pseudo code to implement this method:

[0193] Python

[0194] def recommend_destination(user_intent,vehicle_info,knowledge_base):

[0195] #Integrate user description and vehicle analysis information

[0196] query={

[0197] "visit_type":user_intent["visit_type"],

[0198] "budget":user_intent["budget"],

[0199] "price_range":vehicle_info["price_range"],

[0200] "preferences":user_intent["preferences"]

[0201] }

[0202] # Retrieve destinations that meet the criteria from the knowledge base

[0203] matching_destinations=knowledge_base.search(query)

[0204] #Call the generative model to generate recommendation reasons

[0205] model=load_model("gpt-recommendation")

[0206] recommendation=model.generate_response(matching_destinations,query)

[0207] return recommendation.

[0208] Specifically, the underlying technologies of the knowledge base module include:

[0209] Graph databases, such as Neo4j, or vector databases, such as Pinecone, are used to store and retrieve destination information; the database structure includes: nodes, i.e., destinations, and attributes, such as price, features, and suitable crowds.

[0210] The module is configured to perform predefined destination information entry, storage, dynamic update and rapid retrieval.

[0211] The following is pseudo code to implement this method:

[0212] Python

[0213]

[0214] S3. According to the parking demand of the target vehicle, a target parking space and / or a target parking route is planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route.

[0215] Optionally, according to the parking demand of the target vehicle, a target parking space and / or a target parking route is planned for the target vehicle according to parking space availability and / or vehicle model.

[0216] Optionally, a target parking route is planned for the target vehicle according to the target parking space.

[0217] Optionally, the method of guiding the target vehicle to park according to the target parking space and / or the target parking route may be to send the parking space navigation information to the vehicle computer of the target vehicle.

[0218] Optionally, the method of guiding the target vehicle to park according to the target parking space and / or the target parking route may be to guide through guidance equipment such as guide lights and guide display screens arranged in the parking lot.

[0219] Optionally, according to the parking demand of the target vehicle, a target parking space and / or a target parking route is planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route, including:

[0220] According to the parking demand of the target vehicle, a target parking area corresponding to the parking demand of the target vehicle is obtained;

[0221] According to the target parking area, obtaining the availability of the target parking area;

[0222] According to the availability of the target parking area, a target parking space and / or a target parking route are planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route.

[0223] Optionally, according to the parking demand of the target vehicle, a target parking space and / or a target parking route is planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route, including:

[0224] According to the parking requirements of the target vehicle, the vehicle size of the target vehicle is identified; specifically, the vehicle size: according to the length, width, height, minimum turning radius and other parameters of the vehicle, the size of the parking space suitable for parking is determined. For example, a compact car can be parked in a smaller parking space, while an SUV, MPV or large truck requires a larger parking space;

[0225] According to the vehicle size and parking requirements of the target vehicle, at least one target parking area and a target parking space are determined. The target parking area can be matched with the destination in the parking requirement through a mapping table with a priority order. Specifically, a set of adaptation algorithms is used to select the best parking space according to the vehicle model and size, and a most suitable parking space is selected according to factors such as the distance from the target area and the utilization rate of the parking space.

[0226] Perform path planning according to the target parking space to obtain the target parking route;

[0227] Specifically, path planning is performed according to the target parking space to obtain the target parking route, including:

[0228] Parking lot layout modeling, building or reading parking lot map models;

[0229] According to the map model, the optimal driving route from the current position of the target vehicle to the parking space is calculated through the path planning algorithm; commonly used path planning algorithms include: A* algorithm, Dijkstra algorithm, RRT algorithm, etc.

[0230] Example 3

[0231] A parking navigation system based on multi-modality, including an information collection platform, a data analysis platform and a path planning platform;

[0232] The information collection platform is configured as:

[0233] In response to detecting a target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle;

[0234] The data analysis platform is configured as:

[0235] In response to establishing a connection with the target vehicle based on the communication request, obtaining a parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle;

[0236] The path planning platform is configured as:

[0237] According to the parking demand of the target vehicle, a target parking space and / or a target parking route is planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route.

[0238] Optionally, in response to detecting the target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle includes:

[0239] In response to detecting the target vehicle, obtaining a vehicle image of the target vehicle;

[0240] According to the vehicle image of the target vehicle, obtaining the model of the target vehicle;

[0241] According to the model of the target vehicle, a corresponding communication protocol is obtained, and a communication request is initiated to the target vehicle according to the corresponding communication protocol.

[0242] Optionally, in response to detecting the target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle includes:

[0243] In response to detecting the target vehicle, obtaining a vehicle image of the target vehicle;

[0244] Obtaining a unique identifier of the target vehicle based on the vehicle image of the target vehicle;

[0245] A communication request is initiated to the target vehicle and / or other devices connected to the target vehicle based on the unique identifier of the target vehicle.

[0246] Optionally, in response to establishing a connection with the target vehicle based on the communication request, obtaining a parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle includes:

[0247] In response to establishing a connection with the target vehicle based on the communication request, requesting target navigation data from the target vehicle and / or other devices connected to the target vehicle, the target navigation data being used to indicate a destination of the target vehicle;

[0248] When the target navigation data is received within the specified time, the target navigation data is parsed to obtain the navigation destination pointed to by the target navigation data;

[0249] According to the navigation destination, determining whether the navigation destination is a fuzzy destination or a precise destination, wherein the fuzzy destination is configured as a destination with more than one entity corresponding to the address of the destination, and the precise destination is configured as a destination with only one entity corresponding to the address of the destination;

[0250] When the navigation destination is an accurate destination, the accurate destination is used as the parking requirement of the target vehicle.

[0251] Optionally, in response to establishing a connection with the target vehicle based on the communication request, obtaining a parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle includes:

[0252] In response to establishing a connection with the target vehicle based on the communication request, sending guidance information to the target vehicle and / or other devices connected to the target vehicle, the guidance information being configured to guide the user to describe an intention after parking;

[0253] receiving a user description sent by any one of the target vehicle, other devices connected to the target vehicle, and a user of the target vehicle in response to the guidance information;

[0254] According to the user description, the parking requirements of the target vehicle are obtained based on the generative large model.

[0255] Optionally, according to the user description, based on the generative large model, the parking demand of the target vehicle is obtained, including:

[0256] According to the user description, a semantic analysis result of the user description is obtained;

[0257] According to the semantic analysis result, the semantic analysis result is input into the generative large model connected to the embedded database to obtain the recommended destination, and the recommended destination is recorded in the embedded database;

[0258] The parking demand of the target vehicle is obtained according to the recommended destination.

[0259] Optionally, according to the user description, based on the generative large model, the parking demand of the target vehicle is obtained, including:

[0260] According to the user description, a semantic analysis result of the user description is obtained;

[0261] According to the vehicle image of the target vehicle, obtaining the model of the target vehicle;

[0262] According to the semantic analysis result and the model of the target vehicle, the semantic analysis result and the model of the target vehicle are input into a generative large model connected to an embedded database to obtain a recommended destination, and the recommended destination is recorded in the embedded database;

[0263] The parking demand of the target vehicle is obtained according to the recommended destination.

[0264] Optionally, according to the parking demand of the target vehicle, a target parking space and / or a target parking route is planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route, including:

[0265] According to the parking demand of the target vehicle, a target parking area corresponding to the parking demand of the target vehicle is obtained;

[0266] According to the target parking area, obtaining the availability of the target parking area;

[0267] According to the availability of the target parking area, a target parking space and / or a target parking route are planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route.

[0268] Example 4

[0269] This embodiment provides a device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement any of the above methods.

[0270] Specifically, Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of an electronic device of the hardware operating environment involved in the embodiment of the present application. The electronic device may include: a processor 101, such as a central processing unit (CPU), a communication bus 102, a user interface 104, a network interface 103, and a memory 105. Among them, the communication bus 102 is used to realize the connection and communication between these components. The user interface 104 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 104 may also include a standard wired interface and a wireless interface. The network interface 103 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 105 may optionally be a storage device independent of the aforementioned processor 101. The memory 105 may be a high-speed random access memory (RAM) memory, or it may be a stable non-volatile memory (NVM), such as at least one disk storage; the processor 101 may be a general-purpose processor, including a central processing unit, a network processor, etc., or it may be a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0271] Those skilled in the art will appreciate that Figure 2 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0272] like Figure 2 As shown, the memory 105 as a storage medium may include an operating system, a network communication module, a user interface module, and an application program for implementing a parking navigation method based on multi-modality.

[0273] exist Figure 2 In the electronic device shown, the network interface 103 is mainly used for data communication with a network server; the user interface 104 is mainly used for data interaction with a user; the processor 101 and the memory 105 in the present application can be set in the electronic device, and the electronic device calls the application stored in the memory 105 for implementing a multimodal parking navigation method through the processor 101 to implement the above method.

[0274] Example 5

[0275] This embodiment provides a computer-readable storage medium, on which a computer program is stored, and a processor executes the computer program to implement any of the above methods.

[0276] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including intelligent terminals and servers.

[0277] In the above embodiments of the present disclosure, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0278] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0279] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0280] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0281] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a non-volatile storage medium, including a number of instructions to enable a device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present disclosure. The aforementioned non-volatile storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0282] The above are only preferred embodiments of the present disclosure. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present disclosure. These improvements and modifications should also be regarded as the protection scope of the present disclosure.

Claims

1. A parking navigation method based on multimodality, characterized in that: include: In response to detecting a target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle; In response to establishing a connection with the target vehicle based on the communication request, obtaining a parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle; According to the parking demand of the target vehicle, a target parking space and / or a target parking route is planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route.

2. A parking navigation method based on multimodality as claimed in claim 1, characterized in that: In response to detecting the target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle comprises: In response to detecting the target vehicle, obtaining a vehicle image of the target vehicle; Obtaining the model of the target vehicle according to the vehicle image of the target vehicle; According to the model of the target vehicle, a corresponding communication protocol is obtained, and a communication request is initiated to the target vehicle according to the corresponding communication protocol.

3. The multi-modal parking navigation method according to claim 1, characterized in that: In response to detecting the target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle comprises: In response to detecting the target vehicle, obtaining a vehicle image of the target vehicle; Obtaining a unique identifier of the target vehicle according to the vehicle image of the target vehicle; A communication request is initiated to the target vehicle and / or other devices connected to the target vehicle according to the unique identifier of the target vehicle.

4. The multimodal parking navigation method according to claim 1, characterized in that: The step of obtaining the parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle in response to establishing the connection with the target vehicle based on the communication request includes: In response to establishing a connection with the target vehicle based on the communication request, requesting target navigation data from the target vehicle and / or other devices connected to the target vehicle, the target navigation data being used to indicate a destination of the target vehicle; When the target navigation data is received within a specified time, parsing the target navigation data to obtain a navigation destination pointed to by the target navigation data; According to the navigation destination, determining whether the navigation destination is a fuzzy destination or a precise destination, wherein the fuzzy destination is configured as a destination with more than one entity corresponding to the address of the destination, and the precise destination is configured as a destination with only one entity corresponding to the address of the destination; When the navigation destination is an accurate destination, the accurate destination is used as the parking requirement of the target vehicle.

5. The multimodal parking navigation method according to claim 1, characterized in that: The step of obtaining the parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle in response to establishing the connection with the target vehicle based on the communication request includes: In response to establishing a connection with the target vehicle based on the communication request, sending guidance information to the target vehicle and / or other devices connected to the target vehicle, wherein the guidance information is configured to guide a user to describe an intention after parking; receiving a user description sent by any one of the target vehicle, other devices connected to the target vehicle, and a user of the target vehicle in response to the guidance information; According to the user description, based on a generative large model, the parking demand of the target vehicle is obtained.

6. A parking navigation method based on multimodality as claimed in claim 5, characterized in that: The obtaining, according to the user description and based on the generative large model, the parking demand of the target vehicle includes: According to the user description, obtaining a semantic analysis result of the user description; According to the semantic analysis result, the semantic analysis result is input into a generative macro model connected to an embedded database to obtain a recommended destination, wherein the recommended destination is recorded in the embedded database; The parking requirement of the target vehicle is obtained according to the recommended destination.

7. The multimodal parking navigation method according to claim 5, characterized in that: The obtaining, according to the user description and based on the generative large model, the parking demand of the target vehicle includes: According to the user description, obtaining a semantic analysis result of the user description; Obtaining the model of the target vehicle according to the vehicle image of the target vehicle; According to the semantic analysis result and the model of the target vehicle, the semantic analysis result and the model of the target vehicle are input into a generative large model connected to an embedded database to obtain a recommended destination, wherein the recommended destination is recorded in the embedded database; The parking requirement of the target vehicle is obtained according to the recommended destination.

8. The multi-modal parking navigation method according to claim 6, characterized in that: According to the parking demand of the target vehicle, a target parking space and / or a target parking route are planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route, including: According to the parking demand of the target vehicle, obtaining a target parking area corresponding to the parking demand of the target vehicle; According to the target parking area, obtaining the availability of the target parking area; According to the availability of the target parking area, a target parking space and / or a target parking route are planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route.

9. A parking navigation system based on multimodality, characterized in that: Including information collection platform, data analysis platform and route planning platform; The information collection platform is configured as follows: In response to detecting a target vehicle, obtaining a vehicle image of the target vehicle and initiating a communication request to the target vehicle; The data analysis platform is configured to: In response to establishing a connection with the target vehicle based on the communication request, obtaining a parking requirement of the target vehicle from the target vehicle and / or other devices connected to the target vehicle; The path planning platform is configured as follows: According to the parking demand of the target vehicle, a target parking space and / or a target parking route is planned for the target vehicle, and the target vehicle is guided to park according to the target parking space and / or the target parking route.

10. A device, characterized in that: The device comprises a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the method according to any one of claims 1 to 7.

Images