A multi-modal perception-based high-precision map-free intelligent vehicle searching system and method

Abstract

The application discloses a multi-modal perception-based high-precision map-free intelligent car searching system and method, and relates to the field of intelligent transportation and computer vision.The system comprises a data acquisition module, an index construction module, a positioning verification module, a sequence recording module and a matching navigation module.The application is based on the feature point cloud construction and lightweight semantic recognition of ORB-SLAM3, realizes the real-time modeling of the local environment of a parking lot, and significantly reduces the system construction and maintenance cost.A three-layer positioning verification system is formed by adopting multi-sensor fusion, and a conflict resolution mechanism is set, the positioning accuracy is improved by triggering supplementary visual angle acquisition through secondary identification, and the problems of insufficient accuracy of a single sensor and error accumulation are effectively overcome.The visual semantic feature sequence from the parking space to the exit is automatically recorded after parking, reverse indexing and matching are performed in the car searching stage, an intuitive navigation path is generated in combination with AR and voice prompts, and the user-level high-robustness car searching experience can be realized without a coordinate-based map.

Description

Technical Field

[0001] This invention relates to the fields of intelligent transportation and computer vision technology, and in particular to an intelligent vehicle-finding system and method based on multimodal perception without high-precision maps. Background Technology

[0002] With urbanization, the number of large underground or multi-level parking garages is constantly increasing, making parking and vehicle location problems increasingly prominent. Existing vehicle location solutions mainly include the following categories: deploying a large number of Bluetooth base stations or ultrasonic beacons within the parking lot to locate the vehicle's position through interaction with mobile phones or in-vehicle terminals. This method requires additional hardware, resulting in high deployment and maintenance costs, and the signal is easily interfered with in complex parking environments, leading to insufficient positioning accuracy. Some solutions rely on high-precision digital maps of the parking lot, combining inertial navigation or visual odometry for positioning. However, the production and updating of high-precision maps are expensive, and they are not widely adopted in most underground parking environments, making it difficult for such solutions to be applied on a large scale.

[0003] A common mobile application solution requires users to manually record or upload their parking location when parking, and then use reverse navigation when searching for their car. This method relies heavily on user operation, making it prone to forgetting or misoperation, resulting in a poor user experience.

[0004] In summary, existing car-finding technologies either rely on expensive infrastructure construction or manual operation, lacking a solution that can automatically trigger recording and achieve highly robust car-finding navigation without requiring high-precision maps or additional hardware deployment. Summary of the Invention

[0005] In view of the above problems, the present invention is proposed.

[0006] Therefore, the problem to be solved by this invention is how to achieve a smart car-finding method with low cost, high robustness and good user experience without relying on high-precision maps and large-scale infrastructure transformation.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] In a first aspect, embodiments of the present invention provide a smart vehicle-finding system without high-precision maps based on multimodal perception, comprising: a data acquisition module, used for automatically triggering multimodal data acquisition by the on-board unit when the vehicle is parked or driven to a parking lot;

[0009] The index building module is used to perform ORB-SLAM3-based feature extraction and pose estimation on the panoramic image sequence in the collected data, select stable feature points and combine them with lightweight semantic recognition to generate local feature point cloud and temporary spatial index of the parking session on the same day.

[0010] The localization verification module is used to confirm mapless localization based on local feature point cloud and collected multimodal data, using a three-layer verification method.

[0011] The sequence recording module is used to record the visual semantic feature sequence from the parking space to the nearest exit in chronological order after the vehicle is parked and turned off, and to perform reverse indexing on the sequence.

[0012] The matching navigation module is used to generate a reverse path constrained by the feature sequence by performing multi-scale similarity matching between the real-time sequence on the mobile device and the reverse feature sequence stored in the vehicle, and combining the three-layer verification results.

[0013] As a preferred embodiment of the intelligent vehicle-finding system without high-precision maps based on multimodal perception described in this invention, the multimodal data includes a sequence of panoramic images from four wide-angle fisheye cameras, inertial measurement data from the vehicle-mounted IMU, area identification information received by the UWB chip, and air pressure values ​​from a high-precision air pressure sensor.

[0014] As a preferred embodiment of the intelligent vehicle-finding system without high-precision maps based on multimodal perception described in this invention, the system comprises: performing ORB-SLAM3-based feature extraction and pose estimation on the panoramic image sequence from the collected data; selecting stable feature points and combining them with lightweight semantic recognition to generate a local feature point cloud and a temporary spatial index for the parking session on that day, including:

[0015] ORB-SLAM3 is run independently for each camera. Each camera outputs keyframes and local map points and camera poses in the camera's reference frame. The poses of each camera are uniformly transformed to the vehicle coordinate system using camera-to-vehicle external parameters, and the point clouds are merged.

[0016] Lightweight YOLOv8n is used for object detection; lightweight OCR is run on the detected parking space number area to extract text and confidence scores; the bounding boxes output by detection and OCR are spatially correlated with the coordinates of stable feature points.

[0017] By combining visual matching confidence, detection confidence, and tracking length, semantic confidence is calculated. When multiple labels are obtained for the same point, the label with the highest confidence or semantic priority is used.

[0018] For each selected feature point, triangulation is performed using at least two viewpoints and the corresponding camera pose to obtain 3D coordinates; the depth uncertainty of each triangulated point is estimated and recorded as cov; and similar points are merged in the vehicle coordinate system.

[0019] As a preferred embodiment of the intelligent vehicle-finding system without high-precision maps based on multimodal perception described in this invention, the system performs map-free positioning confirmation based on local feature point clouds and collected multimodal data, using a three-layer verification process, including:

[0020] Floor level verification: The floor level is verified by fusion and correction of the height difference between visually recognized floor signs and barometric pressure sensors; Area level verification: The area ID obtained by UWB beacons is cross-verified with visually recognized area signs; Trajectory and attitude level verification: The trajectory consistency is checked by comparing the IMU sequence with the visual odometry; When the three-level verification conflict, the secondary recognition process is automatically triggered to collect supplementary perspectives and re-fuse and determine the result.

[0021] As a preferred embodiment of the intelligent vehicle-finding system without high-precision maps based on multimodal perception described in this invention, when a conflict arises in the three-layer verification, a secondary recognition process is automatically triggered to collect supplementary perspectives and re-fuse and determine the solution, including:

[0022] Calculate confidence and estimate for each layer: the visual layer includes pose. vis and conf vis The pressure layer includes the floor. est and conf pres The UWB layer includes pos uwb and conf uwb ;

[0023] When any of the triggering conditions is met, the secondary recognition process begins, including:

[0024] Condition 1: If ||pose vis -pos uwb ||>d thresh and conf vis >c min and conf uwb >c min ; or floor est !=floor vis and conf pres >0.7 and conf vis >0.5;

[0025] Condition 2: The difference between the three confidence levels is less than Δc, but the inference results are different, Δc=0.2;

[0026] Condition 3: The pose covariance trace after fusion > σ thresh .

[0027] As a preferred embodiment of the intelligent vehicle-finding system without high-precision maps based on multimodal perception described in this invention, the system includes: after the vehicle is parked and the engine is turned off, recording the visual semantic feature sequence from the parking space to the nearest exit in chronological order, and performing reverse indexing processing on this sequence, including:

[0028] A continuous sequence of image frames {It|t=1,2,…,T} from the parking space to the nearest exit is acquired using an in-vehicle panoramic camera, and semantic target features are extracted from it based on a lightweight deep learning model: ;

[0029] in, ( F is a semantic feature extraction operator. t Let d be the semantic feature vector corresponding to the t-th frame image;

[0030] The semantic feature vectors mentioned above are arranged in chronological order to form a forward feature sequence of the parking session. The forward sequence is then rearranged in reverse order, and each feature is assigned a reverse index during the rearrangement process to form the feature sequence required for the reverse navigation path.

[0031] In the sequence, for significant semantic objects, anchor points are marked on their corresponding feature vectors. When a user enters the car-finding mode, based on the current perceived features F... cur With reverse indexing sequence S reverse Perform similarity matching: ;

[0032] Among them, cos( ) represents cosine similarity.

[0033] As a preferred embodiment of the intelligent vehicle-finding system without high-precision maps based on multimodal perception described in this invention, the system generates a reverse path constrained by the feature sequence by performing multi-scale similarity matching between the real-time sequence on the mobile device and the reverse feature sequence stored in the vehicle, combined with three-layer verification results.

[0034] Serial number of the phone Sequence alignment with candidate session W is performed, and the normalized distance D is obtained using DTW. DTW ( Convert W to similarity:

[0035] ;

[0036] Where λ is the control sensitivity;

[0037] If a matching anchor point is determined, the steps to generate the reverse path are as follows:

[0038] Retrieve the parking space index po in reverse order from the matching position pos. scar Using the estimated rigid body transformation T, the local transformation of each token is... xyz Mapping to the current coordinate system of the mobile phone, if there is no complete T, use a relative turning sequence for coarse alignment; generate continuous paths from discrete token points using splines or Elastic Band interpolation, constrained to not cross obstacles in the locally occupied grid of the mobile phone SLAM; if a path segment is occupied, perform local replanning combined with the real-time SLAM point cloud.

[0039] Secondly, embodiments of the present invention provide a method for intelligent vehicle finding without high-precision maps based on multimodal perception, including:

[0040] When a vehicle is parked or driven to a parking lot, the on-board unit automatically triggers multimodal data acquisition;

[0041] The panoramic image sequence in the collected data is subjected to feature extraction and pose estimation based on ORB-SLAM3. Stable feature points are selected and combined with lightweight semantic recognition to generate local feature point cloud and temporary spatial index of the parking session on the same day.

[0042] Based on local feature point cloud and collected multimodal data, map-free localization confirmation is performed using a three-layer verification method.

[0043] After the vehicle is parked and the engine is turned off, the visual semantic feature sequence from the parking space to the nearest exit is recorded in chronological order, and the sequence is then reverse-indexed.

[0044] By performing multi-scale similarity matching between the real-time sequence on the mobile device and the reverse feature sequence stored in the vehicle, and combining the three-layer verification results, a reverse path constrained by the feature sequence is generated.

[0045] Thirdly, embodiments of the present invention provide a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program instructions are executed by the processor, they implement the steps of the intelligent vehicle-finding system without high-precision maps based on multimodal perception as described in the first aspect of the present invention.

[0046] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program instructions are executed by a processor, they implement the steps of the intelligent vehicle-finding system without high-precision maps based on multimodal perception as described in the first aspect of the present invention.

[0047] The beneficial effects of this invention are as follows: Based on ORB-SLAM3 feature point cloud construction and lightweight semantic recognition, this invention achieves real-time modeling of the local environment of a parking lot, without relying on pre-built high-precision maps or large-scale deployment of beacons or QR codes, significantly reducing system construction and maintenance costs. It employs a three-layer positioning verification system formed by the fusion of multiple sensors, including vision, barometric pressure, UWB, and IMU, and sets up a conflict resolution mechanism. Secondary recognition triggers supplementary viewpoint acquisition, improving positioning accuracy and effectively overcoming the problems of insufficient accuracy and error accumulation from single sensors. By automatically recording the visual semantic feature sequence from the parking space to the exit after parking, and performing reverse indexing and matching during the car-finding stage, combined with AR and voice prompts, an intuitive navigation path is generated, achieving a highly robust user-level car-finding experience without the need for a coordinate map. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of a multimodal perception-based intelligent vehicle-finding system without a high-precision map. Figure 2 The flowchart shows a method for intelligent vehicle finding without high-precision maps based on multimodal perception. Figure 3 This is a diagram of a computer device for a multimodal perception-based intelligent vehicle-finding system and method without a high-precision map. Detailed Implementation

[0050] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0051] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0052] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0053] Example 1

[0054] Reference Figures 1-3 This is the first embodiment of the present invention, which provides a multimodal perception-based intelligent vehicle-finding system without high-precision maps. The system consists of a data acquisition module, an index construction module, a positioning verification module, a sequence recording module, and a matching navigation module.

[0055] Specifically, the data acquisition module is used to automatically trigger multimodal data acquisition when the vehicle is parked or driven to a parking lot.

[0056] Furthermore, the multimodal data includes a sequence of panoramic images from four wide-angle fisheye cameras, inertial measurement data from the vehicle-mounted IMU, area identification information received by the UWB chip, and air pressure values ​​from a high-precision barometer.

[0057] Specifically, the index building module is used to perform ORB-SLAM3-based feature extraction and pose estimation on the panoramic image sequence in the collected data, select stable feature points and combine them with lightweight semantic recognition to generate local feature point clouds and temporary spatial indexes for the parking session on that day.

[0058] Furthermore, ORB-SLAM3 is run independently for each camera. Each camera outputs keyframes and local map points and camera poses in its reference frame. The camera poses are then uniformly transformed to the vehicle coordinate system using external parameters from the camera to the vehicle body, and the point clouds are merged.

[0059] Lightweight YOLOv8n is used for object detection; lightweight OCR is run on the detected parking space number area to extract text and confidence scores; the bounding boxes output by detection and OCR are spatially correlated with the coordinates of stable feature points.

[0060] By combining visual matching confidence, detection confidence, and tracking length, semantic confidence is calculated. When multiple labels are obtained for the same point, the label with the highest confidence or semantic priority is used.

[0061] For each selected feature point, triangulation is performed using at least two viewpoints and the corresponding camera pose to obtain 3D coordinates; the depth uncertainty of each triangulated point is estimated and recorded as cov; and similar points are merged in the vehicle coordinate system.

[0062] It should be noted that each vehicle-mounted wide-angle camera independently runs the ORB-SLAM3 algorithm, outputting keyframes, local map points, and camera pose in that camera's reference frame. Furthermore, using pre-calibrated extrinsic parameters from the camera to the vehicle, the poses of each camera are uniformly transformed to the vehicle's coordinate system, thereby achieving the merging of point clouds from different cameras in the same coordinate system. Compared to existing schemes using only a single camera or single-view SLAM, this method effectively avoids the feature sparsity and occlusion problems caused by the limited field of view of a single camera.

[0063] Furthermore, while extracting feature points, a lightweight YOLOv8n model is invoked to perform object detection on keyframes, locating salient semantic targets such as parking space numbers, floor markings, and area signs. For the detected parking space number regions, a lightweight OCR module is further run to extract text content and confidence scores, and the output bounding box is spatially correlated with the coordinates of surrounding stable feature points, forming a bimodal binding relationship of "semantic label - geometric feature". Compared with existing simple geometric feature point clouds, this method can endow point clouds with stronger semantic information.

[0064] Furthermore, for each candidate semantic feature point, a global semantic confidence score is calculated by comprehensively considering visual matching confidence, object detection confidence, and feature point tracking length. If the same spatial point is assigned multiple semantic labels, the decision is made according to the principle of maximizing confidence or a preset semantic priority. For example, floor identifiers take precedence over parking space numbers to ensure semantic consistency in the navigation sequence. This fusion mechanism effectively overcomes the interference caused by misjudgment or detection drift in traditional OCR, improving the robustness of semantic annotation.

[0065] Furthermore, for the selected semantic feature points, triangulation is performed using at least two viewpoints and their corresponding camera poses to obtain the three-dimensional coordinates of the point in the vehicle coordinate system. Simultaneously, the depth uncertainty of each triangulated point is estimated and quantified using covariance (cov) to ensure a dynamic balance between accuracy and reliability during subsequent path indexing and multimodal fusion.

[0066] Furthermore, under a unified vehicle coordinate system, triangulated points with similar spatial locations are aggregated and merged, redundant feature points are removed, and a dense local feature point cloud with semantic labels and depth uncertainty is finally formed. This point cloud is stored as a temporary spatial index for the parking session of the day and used for reverse path generation during the vehicle search phase.

[0067] Specifically, the location verification module is used to confirm mapless location based on local feature point cloud and collected multimodal data, using a three-layer verification method.

[0068] Further, floor level verification: the height difference between visually recognized floor signs and barometric pressure sensors is fused and corrected; area level verification: the area ID obtained from UWB beacons is cross-verified with visually recognized area signs; trajectory and attitude level verification: trajectory consistency is checked by comparing IMU sequences with visual odometry; when conflicts occur in the three-level verification, a secondary recognition process is automatically triggered to collect supplementary perspectives and re-fuse and determine the results.

[0069] Furthermore, confidence scores and estimates are calculated for each layer: the visual layer includes pose. vis and conf vis The pressure layer includes the floor. est and conf pres The UWB layer includes pos uwb and conf uwb ;

[0070] When any of the triggering conditions is met, the secondary recognition process begins, including:

[0071] Condition 1: If ||pose vis -pos uwb ||>d thresh and conf vis >c min and conf uwb >c min ; or floor est !=floor vis and conf pres >0.7 and conf vis >0.5;

[0072] Condition 2: The difference between the three confidence levels is less than Δc, but the inference results are different, Δc=0.2;

[0073] Condition 3: The pose covariance trace after fusion > σ thresh .

[0074] It should be noted that, taking a 20-story office building as an application scenario, the mobile robot needs to accurately determine its floor, area, and trajectory when performing indoor inspection tasks. To ensure the reliability of the positioning results, this invention provides a multi-level verification method. The robot first identifies the floor sign "12F" in the elevator lobby using a visual camera mounted on its head. Simultaneously, its built-in barometric pressure sensor measures the current air pressure as 986.2 hPa. Using the standard atmospheric pressure altitude conversion formula, the current height is calculated to be approximately 34.5m.

[0075] Based on the building's standard floor height (approximately 3 meters per floor), the calculated air pressure corresponds to the 11th-12th floor range. The system integrates both results, determining the current floor as 12th floor and assigning a confidence level of 0.95. Upon entering a corridor, the robot detects the UWB beacon's emission area ID=A305, corresponding to "12th Floor - East Corridor" in the database. Simultaneously, the camera captures the area sign "12th Floor - A305" on the wall.

[0076] After cross-validation, both methods were consistent, and the determination result was 12F-East Corridor, with a confidence level of 0.97. During the robot's 20m journey through the 12F corridor, the accelerometer and gyroscope data provided by the IMU sensor calculated the trajectory as "straight + 90° left turn + straight for 5m"; simultaneously, the visual odometry (VO) obtained the trajectory based on corridor feature point matching as "straight + 90° left turn + straight for 5.3m". The deviation between the two trajectories was 0.3m, lower than the set threshold (0.5m), therefore the trajectory verification passed. Upon entering the conference area, the system detected a conflict: the visual recognition identified the sign as "12F-B201", but the UWB beacon feedback area ID corresponded to "13F-B201". At this point, the system automatically triggers a secondary recognition process: the robot rotates its perspective to re-acquire multiple sign images, and the visual recognition result is still "12F-B201"; the system reads the barometric pressure sensor data again, and the barometric pressure value still corresponds to the height range of 12F; after fusion judgment, the current area is finally identified as 12F-B201, and the UWB beacon signal is marked as abnormal drift.

[0077] After multiple tests, this embodiment shows that when the data at all levels is consistent, the system can quickly complete the positioning.

[0078] When a conflict occurs, the secondary identification process can effectively eliminate errors from a single sensor, ensuring a positioning accuracy of >99%; compared with traditional methods that rely solely on UWB or visual recognition, the overall robustness is improved by about 25%.

[0079] Specifically, the sequence recording module is used to record the visual semantic feature sequence from the parking space to the nearest exit in chronological order after the vehicle is parked and turned off, and to perform reverse indexing processing on the sequence.

[0080] Furthermore, a continuous sequence of image frames {It|t=1,2,…,T} from the parking space to the nearest exit is acquired using an in-vehicle surround-view camera, and semantic target features are extracted from it based on a lightweight deep learning model:

[0081] ;

[0082] in, ( F is a semantic feature extraction operator.t Let d be the semantic feature vector corresponding to the t-th frame image;

[0083] The semantic feature vectors mentioned above are arranged in chronological order to form a forward feature sequence of the parking session. The forward sequence is then rearranged in reverse order, and each feature is assigned a reverse index during the rearrangement process to form the feature sequence required for the reverse navigation path.

[0084] In the sequence, for significant semantic objects, anchor points are marked on their corresponding feature vectors. When a user enters the car-finding mode, based on the current perceived features F... cur With reverse indexing sequence S reverse Perform similarity matching:

[0085] Among them, cos( ) represents cosine similarity.

[0086] It should be noted that, taking a large underground parking lot as an example, after a vehicle is parked in parking space A053 on level B2, the system automatically begins to acquire a continuous sequence of image frames {It|t=1,2,…,T} via the vehicle-mounted surround-view cameras (four fisheye cameras stitched together to form a panoramic view). After the vehicle is turned off, the system automatically triggers a short recording, acquiring a video sequence along the path from parking space A053 to the nearest exit, totaling 280 frames (approximately 14 seconds). After acquisition, the system performs basic preprocessing on the frame images, including: image distortion correction; brightness normalization; and scaling the image size to 224×224 pixels to meet the input requirements of a lightweight model.

[0087] A lightweight convolutional neural network is used as φ( ) Operation operators, from each frame image I t Extract a d=128-dimensional semantic feature vector. The feature content includes significant semantic targets such as parking space number signs, wall color partitions, fire extinguisher locations, and entrance / exit signs.

[0088] Furthermore, the system will {F1,F2,…,F T Arranged in chronological order, the positive feature sequence S is obtained. forward Subsequently, the sequence is reversed and rearranged; and during the reversal process, a reverse index τ is assigned to each feature vector, i.e., FT→τ=1, FT-1→τ=2,…,F1→τ=T, thus constructing a feature reference sequence for the reverse navigation path.

[0089] Within the sequence, the system automatically assigns anchor points to the feature vectors of significant semantic objects. For example, "Parking space 053 in area A" → anchor point A1, index τ=280; "Fire extinguisher box" → anchor point A2, index τ=160;

[0090] “Elevator lobby entrance B2-E1” → Anchor point A3, index τ=1. The anchor point is used to improve the robustness of subsequent similarity matching.

[0091] Furthermore, an example of the matching process: If the current user is near a fire extinguisher, the extracted F... cur The feature similarity corresponding to anchor point A2 is the highest (cosine similarity = 0.92), and the system determines that the user is at position τ = 160 on the reverse path. Based on this, the system deduces that "there are approximately 120 frames left to reach the target parking space A053" and generates a reverse guidance prompt on the navigation interface. The system displays the user's current position and the corresponding index on the reverse path in real time, providing visual navigation prompts: "Turn left 20 meters ahead to enter area A"; "Go straight for 30 meters to reach parking space A053". In this embodiment, the reverse indexed path feature sequence can: preserve semantic continuity and avoid drift caused by lighting or occlusion in traditional geometric path matching; enhance the robustness of key position recognition by using anchor points; in actual tests, the average accuracy of cosine similarity matching reaches 96.8%, which is significantly better than positioning methods that rely solely on GPS and UWB.

[0092] Specifically, the matching navigation module is used to generate a reverse path constrained by the feature sequence by performing multi-scale similarity matching between the real-time sequence on the mobile device and the reverse feature sequence stored in the vehicle, and combining the three-layer verification results.

[0093] Furthermore, the phone serial number Sequence alignment with candidate session W is performed, and the normalized distance D is obtained using DTW. DTW ( Convert W to similarity:

[0094] ;

[0095] Where λ is the control sensitivity;

[0096] If a matching anchor point is determined, the steps to generate the reverse path are as follows:

[0097] Retrieve the parking space index po in reverse order from the matching position pos. scar Using the estimated rigid body transformation T, the local transformation of each token is... xyz Mapping to the current coordinate system of the mobile phone, if there is no complete T, use a relative turning sequence for coarse alignment; generate continuous paths from discrete token points using splines or Elastic Band interpolation, constrained to not cross obstacles in the locally occupied grid of the mobile phone SLAM; if a path segment is occupied, perform local replanning combined with the real-time SLAM point cloud.

[0098] It should be noted that in matching sessions In the process, the system determines the user's current anchor point location pos=180 (corresponding to "B2 floor fire extinguisher identifier"). The vehicle parking space index is poscar=280 (corresponding to "Area A, parking space 053"). The reverse path generation steps are as follows: Starting from the matching location pos=180, the feature subsequence {w} of the parking space index poscar=280 is obtained in reverse order. 180 ,w 181 ,…,w 280 If the rigid body transformation T between the camera and the vehicle is known, then the local coordinates of each token are mapped to the current coordinate system of the mobile SLAM; if T cannot be fully obtained at present, then the user's relative turning sequence (Δθ) is used. i Roughly align with the walking direction.

[0099] Furthermore, the paths of discrete token points are generated into continuous curves using cubic spline interpolation. To improve smoothness and feasibility, the Elastic Band method is used for secondary optimization of the paths, with the following constraints:

[0100] The path must not traverse obstacles within the locally occupied grid constructed in real-time by mobile SLAM; the path curvature must not exceed κ. max =0.4 m -1 When a path segment is detected as being "occupied by construction area" by the SLAM point cloud, the system performs local replanning: using obstacle boundary points as new constraints, it calls A... * The algorithm searches for alternative paths on the local SLAM map; these alternative paths are then merged locally and reassembled back into the reverse path. Finally, the system displays the reverse path navigation on the mobile device, prompting the user: "Please go straight for 15 meters, turn left through the B2-E1 elevator lobby, and you will reach parking space A053 in 30 meters ahead." In this embodiment, through DTW alignment and reverse path generation, the average matching accuracy reaches 95.7%; even with temporary obstacles, local replanning can be completed within 1.8 seconds, significantly improving navigation continuity and robustness.

[0101] Example 2

[0102] This embodiment is the second embodiment of the present invention. This embodiment provides an intelligent vehicle-finding system without high-precision maps based on multimodal perception. In order to verify the beneficial effects of the present invention, a simulation experiment is conducted for scientific demonstration.

[0103] Specifically, a two-level underground parking garage with approximately 800 parking spaces was selected. The environment included varying lighting conditions, dynamic pedestrians, and some construction obstacles. After a vehicle is parked, the onboard unit automatically triggers a surround-view image capture, generating a parking session feature point cloud and an inverse feature sequence. Three methods were compared: Method A: QR code identification vehicle location scheme; Method B: Single-vision SLAM vehicle location scheme; Method C: This invention (multimodal fusion + inverse feature sequence).

[0104] Furthermore, after the vehicle is parked and the engine is turned off, the onboard panoramic camera acquires a continuous sequence of image frames; ORB-SLAM3 is run to extract feature points and YOLOv8n+OCR is used to identify parking space numbers and floor markings; position correction is completed through three layers of verification (barotropic layer, UWB layer, and visual layer), and a temporary spatial index is stored; the semantic feature sequence of the path from the parking space to the exit is recorded in chronological order and reverse indexing is performed.

[0105] Furthermore, after returning to the parking lot, the user launches the mobile app and enters the car-finding mode; the mobile app collects semantic feature sequences of the driving process in real time. And use DTW to perform similarity matching with the session sequence W; when the similarity Sim( When W exceeds the threshold (0.8), the corresponding reverse navigation anchor point is located.

[0106] Furthermore, reverse the sequence from the matching position pos to the parking space index poscar, and extract the reverse feature subsequence;

[0107] If the rigid body transformation T is known, the coordinates of the feature points are mapped to the mobile phone coordinate system; if T is missing, a relative turning sequence is used for coarse alignment; cubic spline interpolation + Elastic Band optimization is used to generate a continuous navigation path and constrain it not to cross the occupied grid obstacles of the mobile phone SLAM.

[0108] Furthermore, when a segment of the path is detected to be blocked by a dynamic obstacle, local replanning is triggered; based on A * The algorithm generates alternative paths on the local point cloud of SLAM; the alternative paths are then smoothly stitched back into the overall reverse path.

[0109] Furthermore, the performance of the three methods was statistically analyzed under six indicators: average positioning error, floor recognition accuracy, vehicle search time, success rate without network, robustness in dynamic environments, and energy consumption. Each method was tested 50 times, and the experimental data were recorded and the mean was calculated.

[0110] Table 1 Comparative Experiment Table

[0111]

[0112] Table 1 illustrates that the method of this invention reduces the average positioning error to less than 0.5 meters, which is far superior to the comparative methods; the vehicle search time is shortened to less than 3 minutes, significantly improving efficiency; the success rate reaches 86.4% in a network-free environment, meeting the actual needs of underground parking lots; the robustness score in a dynamic obstacle environment reaches 4.7 / 5, which is significantly better than the comparative scheme; and the use of a lightweight model and low-power design reduces energy consumption to 7%, improving the user experience.

[0113] This embodiment also provides a computer device applicable to the case of a multimodal perception-based intelligent vehicle search system without a high-precision map, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the multimodal perception-based intelligent vehicle search system without a high-precision map as proposed in the above embodiment.

[0114] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0115] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the intelligent vehicle-finding system without high-precision maps based on multimodal perception as proposed in the above embodiments.

[0116] In summary, this invention utilizes ORB-SLAM3's feature point cloud construction and lightweight semantic recognition to achieve real-time modeling of the local parking environment. It eliminates the need for pre-built high-precision maps and large-scale deployment of beacons or QR codes, significantly reducing system construction and maintenance costs. A three-layer positioning verification system is formed by fusing multiple sensors, including vision, barometric pressure, UWB, and IMU, and a conflict resolution mechanism is implemented. Secondary recognition triggers supplementary viewpoint acquisition, improving positioning accuracy and effectively overcoming the problems of insufficient accuracy and error accumulation from single sensors. By automatically recording the visual and semantic feature sequence from the parking space to the exit after parking, and performing reverse indexing and matching during the vehicle search phase, combined with AR and voice prompts, an intuitive navigation path is generated, achieving a highly robust user-level vehicle search experience without the need for a coordinate-based map.

[0117] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A multimodal perception-based intelligent vehicle-finding system without a high-precision map, characterized in that, include: The data acquisition module is used to automatically trigger multimodal data acquisition by the on-board unit when the vehicle is parked or driven to a parking lot; The index building module is used to perform ORB-SLAM3-based feature extraction and pose estimation on the panoramic image sequence in the collected data, select stable feature points and combine them with lightweight semantic recognition to generate local feature point clouds and temporary spatial indexes for the parking session on the same day. The localization verification module is used to confirm mapless localization based on local feature point cloud and collected multimodal data, using a three-layer verification method. The sequence recording module is used to record the visual semantic feature sequence from the parking space to the nearest exit in chronological order after the vehicle is parked and turned off, and to perform reverse indexing on the sequence. The matching navigation module is used to generate a reverse path constrained by the feature sequence by performing multi-scale similarity matching between the real-time sequence on the mobile device and the reverse feature sequence stored in the vehicle, and combining the three-layer verification results. The method of map-free localization confirmation based on local feature point clouds and collected multimodal data, using a three-layer verification process, includes: Floor level verification: The floor level is verified by fusion and correction between visual recognition of floor markings and height differences measured by barometric pressure sensors; Regional-level verification: Cross-validation is performed between the regional ID obtained through UWB beacons and the visually recognized regional signs; Trajectory and attitude-level verification: Trajectory consistency is checked by comparing the IMU sequence with the visual odometry; When conflicts occur in the three-level verification, a secondary recognition process is automatically triggered to collect supplementary perspectives and re-fuse and determine the results.

2. The intelligent vehicle-finding system without high-precision maps based on multimodal perception as described in claim 1, characterized in that, The multimodal data includes a sequence of panoramic images from four wide-angle fisheye cameras, inertial measurement data from the vehicle-mounted IMU, area identification information received by the UWB chip, and air pressure values ​​from a high-precision barometer.

3. The intelligent vehicle-finding system without high-precision maps based on multimodal perception as described in claim 1, characterized in that: The process involves performing ORB-SLAM3-based feature extraction and pose estimation on the panoramic image sequences from the collected data, selecting stable feature points, and combining them with lightweight semantic recognition to generate a local feature point cloud and temporary spatial index for the parking session on that day. ORB-SLAM3 is run independently for each camera. Each camera outputs keyframes and local map points and camera poses in the camera's reference frame. The poses of each camera are uniformly transformed to the vehicle coordinate system using camera-to-vehicle external parameters, and the point clouds are merged. Lightweight YOLOv8n is used for object detection; lightweight OCR is run on the detected parking space number area to extract text and confidence scores; the bounding boxes output by detection and OCR are spatially correlated with the coordinates of stable feature points. By combining visual matching confidence, detection confidence, and tracking length, semantic confidence is calculated. When multiple labels are obtained for the same point, the highest confidence or semantic priority is used for adjudication. For each selected feature point, triangulation is performed using at least two viewpoints and the corresponding camera pose to obtain 3D coordinates; the depth uncertainty of each triangulated point is estimated and recorded as cov; and similar points are merged in the vehicle coordinate system.

4. The intelligent vehicle-finding system without high-precision maps based on multimodal perception as described in claim 1, characterized in that: When a conflict arises in the three-layer verification, a secondary recognition process is automatically triggered to collect supplementary perspectives and re-fuse and determine the result, including: Calculate confidence and estimate for each layer: the visual layer includes pose. vis and conf vis The pressure layer includes the floor. est and conf pres The UWB layer includes pos uwb and conf uwb ; When any of the triggering conditions is met, the secondary recognition process begins, including: Condition 1: If ||pose vis -pos uwb ||>d thresh and conf vis >c min and conf uwb >c min ; or floor est !=floor vis and conf pres >0.7 and conf vis >0.5; Condition 2: The difference between the three confidence levels is less than Δc, but the inference results are different, Δc=0.2; Condition 3: The pose covariance trace after fusion > σ thresh .

5. The intelligent vehicle-finding system without high-precision maps based on multimodal perception as described in claim 1, characterized in that: After the vehicle is parked and the engine is turned off, the visual semantic feature sequence from the parking space to the nearest exit is recorded in chronological order, and the sequence is then reverse-indexed, including: A continuous sequence of image frames {It|t=1,2,…,T} from the parking space to the nearest exit is acquired using an in-vehicle panoramic camera, and semantic target features are extracted from it based on a lightweight deep learning model: ； in, ( F is the semantic feature extraction operator. t Let d be the semantic feature vector corresponding to the t-th frame image; The semantic feature vectors mentioned above are arranged in chronological order to form a positive feature sequence of the parking session. The positive feature sequence is then rearranged in reverse order, and a reverse index is assigned to each feature during the rearrangement process to form the feature sequence required for the reverse navigation path. In the sequence, for significant semantic objects, anchor points are marked on their corresponding feature vectors. When a user enters the car-finding mode, based on the current perceived features F... cur With reverse indexing sequence S reverse Perform similarity matching: ； Among them, cos( ) represents cosine similarity.

6. The intelligent vehicle-finding system without high-precision maps based on multimodal perception as described in claim 1, characterized in that: The process involves performing multi-scale similarity matching between the real-time sequence from the mobile device and the reverse feature sequence stored in the vehicle, combined with three-layer verification results, to generate a reverse path constrained by the feature sequence, including: Serial number of the phone Sequence alignment with candidate session W is performed, and the normalized distance D is obtained using DTW. DTW ( Convert W to similarity: ； Where λ is the control sensitivity; If a matching anchor point is determined, the steps to generate the reverse path are as follows: Retrieve the parking space index po in reverse order from the matching position pos. scar Using the estimated rigid body transformation T, the local transformation of each token is... xyz Mapping to the current coordinate system of the mobile phone, if there is no complete T, use a relative turning sequence for coarse alignment; generate continuous paths from discrete token points using splines or Elastic Band interpolation, constrained to not cross obstacles in the locally occupied grid of the mobile phone SLAM; if a path segment is occupied, perform local replanning combined with the real-time SLAM point cloud.

7. A method for intelligent vehicle finding without high-precision maps based on multimodal perception, characterized in that, include: When a vehicle is parked or driven to a parking lot, the on-board unit automatically triggers multimodal data acquisition; The panoramic image sequences in the collected data are subjected to feature extraction and pose estimation based on ORB-SLAM3. Stable feature points are selected and combined with lightweight semantic recognition to generate local feature point clouds and temporary spatial indexes for the parking session on the same day. Based on local feature point clouds and collected multimodal data, map-free localization confirmation is performed using a three-layer verification method, including: Floor level verification: The floor level is verified by fusing and correcting the height difference between visually recognized floor signs and barometric pressure sensors; Area level verification: The area ID obtained by UWB beacons is cross-verified with visually recognized area signs; Trajectory and attitude level verification: The trajectory consistency is checked by comparing IMU sequences with visual odometry; When conflicts occur in the three-level verification, a secondary recognition process is automatically triggered to collect supplementary perspectives and re-fuse and determine the results. After the vehicle is parked and the engine is turned off, the visual semantic feature sequence from the parking space to the nearest exit is recorded in chronological order, and the sequence is then reverse-indexed. By performing multi-scale similarity matching between the real-time sequence on the mobile device and the reverse feature sequence stored in the vehicle, and combining the three-layer verification results, a reverse path constrained by the feature sequence is generated.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the intelligent vehicle-finding system without high-precision maps based on multimodal perception as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the intelligent vehicle-finding system without high-precision maps based on multimodal perception as described in any one of claims 1 to 6.

Images