Vehicle parking method and system, and vehicle
By aligning and collaboratively analyzing multimodal data, and using neural network models to precisely control vehicle parking, the problems of response delay and low success rate of parking in narrow spaces in automatic parking systems under dynamic obstacle scenarios are solved, thus achieving efficient parking.
Patent Information
- Application Number
- PCT/CN2025/148276
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-08-21
- Filing Date
- 2025-12-31
- Publication Date
- 2026-11-12
AI Technical Summary
Automatic parking systems suffer from response delays in dynamic obstacle interaction scenarios, resulting in a high rate of false emergency braking, poor environmental adaptability, low success rate in parking in narrow spaces, and low parking efficiency.
By acquiring and aligning multimodal data, and using neural network architecture search and knowledge distillation models for collaborative analysis, the parking spaces and driving routes of vehicles are determined, and vehicle parking is precisely controlled by combining trajectory information.
It improves parking efficiency in complex environments, reduces accidental emergency braking, and increases the success rate of parking in narrow spaces.
Smart Images

Figure CN2025148276_12112026_PF_FP_ABST
Abstract
Description
Vehicle parking methods, systems and vehicles
[0001] Cross-reference
[0002] This disclosure claims priority to Chinese Patent Application No. 202511177836.3, filed on August 21, 2025, entitled "Parking Method, System and Vehicle", the entire contents of which are incorporated herein by reference. Technical Field
[0003] This disclosure relates to the field of vehicle technology, and more specifically, to a parking method, system, and vehicle. Background Technology
[0004] When a vehicle is in a dynamic obstacle interaction scenario, the prediction response delay of the automatic parking system to sudden moving targets (such as children, pets, etc.) generally exceeds 200 milliseconds (ms), resulting in an emergency braking false trigger rate of 12%-15%. In addition, the system has poor environmental adaptability and a low success rate in parking in narrow spaces (such as width < vehicle width + 35 cm), resulting in technical problems of low parking efficiency.
[0005] There is currently no good solution to the above problems. Summary of the Invention
[0006] This disclosure provides a parking method, system, and vehicle to at least address the technical problem of low parking efficiency.
[0007] According to one aspect of the present disclosure, a method for parking a vehicle is provided. The method includes: acquiring initial multimodal data of the vehicle's environment, wherein the initial multimodal data represents different types of environmental data collected by different types of environmental sensing devices; performing alignment processing on the initial multimodal data to obtain target multimodal data, wherein the target multimodal data represents different types of environmental data aligned in time and / or space; invoking a target model to perform collaborative analysis on the target multimodal data to obtain vehicle information recognition results and environment information recognition results; determining a parking space for the vehicle from the environment based on the vehicle information recognition results, the environment information recognition results, and trajectory information matching the vehicle's current execution parameters, and determining a driving path for the vehicle to travel to the parking space from the environment, wherein the current execution parameters characterize the execution performance of actuators in the vehicle, and the trajectory information predicts the vehicle's trajectory during movement; and controlling the vehicle to drive to the parking space and park according to the driving path.
[0008] According to another aspect of the embodiments of this disclosure, a vehicle parking system is also provided, comprising: multiple sensing devices, respectively used to acquire target location data, target point cloud data, and target image data of the environment in which the vehicle is located, wherein the target location data is used to represent the positions of different objects in the environment in which the vehicle is located, the target point cloud data is used to represent the three-dimensional space of the environment, and the target image data is used to represent the two-dimensional images of different objects in the environment; a processor, used to perform alignment processing on the target location data, target point cloud data, and target image data in time and / or space to obtain target multimodal data; calling a target model to perform collaborative analysis on the target multimodal data to obtain vehicle information recognition results and environment information recognition results; based on the vehicle information recognition results, the environment information recognition results, and trajectory information matching the current execution parameters of the vehicle, determining the parking space to be parked in the environment, and determining the driving path of the vehicle to be driven to the parking space from the environment, wherein the current execution parameters are used to characterize the execution performance of the actuators in the vehicle, and the trajectory information is used to predict the trajectory of the vehicle during movement; and a controller, used to control the vehicle to drive to the parking space for parking according to the driving path.
[0009] According to another aspect of the present disclosure, a vehicle is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods of various embodiments of the present disclosure when it runs.
[0010] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform the methods of the various embodiments of the present disclosure.
[0011] According to another aspect of the embodiments of this disclosure, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this disclosure.
[0012] According to another aspect of the embodiments of this disclosure, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods in various embodiments of this disclosure.
[0013] According to another aspect of the embodiments of this disclosure, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this disclosure.
[0014] In this embodiment, a collaborative analysis of multimodal data is employed. The initial multimodal data is aligned to obtain target multimodal data representing different types of environmental data aligned in time and / or space. Then, a target model is invoked to perform collaborative analysis on the target multimodal data, yielding vehicle and environmental information identification results. Based on these results, along with trajectory information matching the vehicle's current execution parameters, the parking space and driving path are determined. Following this path, the vehicle is controlled to drive to the parking space, achieving the purpose of parking. In other words, in this embodiment of the present disclosure, since the target model is used to perform collaborative analysis on the multimodal data of the vehicle to obtain the vehicle information recognition result and the environmental information recognition result, and then based on the vehicle information recognition result, the environmental information recognition result, and the vehicle's trajectory information, the parking space and driving path of the vehicle can be determined, so that the vehicle can be controlled to drive to the parking space according to the driving path, so as to accurately control the vehicle to complete the parking, avoiding the inability to park in time in complex scenarios, thereby achieving the technical effect of improving the parking efficiency of the vehicle and solving the technical problem of low parking efficiency of the vehicle. Attached Figure Description
[0015] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the present disclosure and are used to explain the disclosure, but do not constitute an undue limitation of the disclosure. In the drawings:
[0016] Figure 1 is a flowchart of a vehicle parking method according to an embodiment of the present disclosure;
[0017] Figure 2 is a flowchart of another vehicle parking method according to an embodiment of the present disclosure;
[0018] Figure 3 is a schematic diagram of a parking system for a vehicle according to an embodiment of the present disclosure. Detailed Implementation
[0019] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.
[0020] It should be noted that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or components is not necessarily limited to those steps or components that are explicitly listed, but may include other steps or components that are not explicitly listed or that are inherent to such process, method, product, or device.
[0021] According to embodiments of this disclosure, a method embodiment for parking a vehicle is provided, wherein the steps shown in the flowcharts of the accompanying drawings are executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases the steps shown or described are executed in a different order than that shown here.
[0022] This embodiment provides a vehicle parking method. Figure 1 is a flowchart of a vehicle parking method according to an embodiment of the present disclosure. As shown in Figure 1, the method includes the following steps.
[0023] Step S101: Obtain initial multimodal data of the environment in which the vehicle is located, wherein the initial multimodal data is used to represent different types of environmental data collected by different types of environmental sensing devices.
[0024] In the technical solution proposed in step S101 of the present disclosure embodiment, initial multimodal data of the vehicle's environment is acquired through different types of environmental sensing devices. The environmental sensing devices include surround-view cameras, ultrasonic probes, lidar, etc., and the initial multimodal data is referred to as sensing data or perception data.
[0025] Optionally, the aforementioned surround-view camera provides visual information about the vehicle's surroundings, enabling the vehicle's Automated Parking Assist (APA) system to identify parking spaces, obstacles, and road conditions. This APA system includes multiple high-resolution cameras mounted at the front, rear, left, and right of the vehicle to capture 360-degree images of its surroundings.
[0026] Optionally, the ultrasonic probe is used to detect obstacles close to the vehicle, i.e., when the vehicle is extremely close to surrounding obstacles during parking. The ultrasonic probe can be mounted on the front and rear bumpers of the vehicle to emit ultrasonic pulses. When these ultrasonic pulses are reflected back by an obstacle, the ultrasonic probe can receive and measure the echo time, thereby calculating the distance between the vehicle and the obstacle.
[0027] Optionally, lidar provides high-precision depth information and three-dimensional (3D) point cloud data in automated parking systems. LiDAR captures high-density point cloud data of the surrounding environment by emitting laser pulses and receiving the reflected signals, which are used to measure the distance between the vehicle and obstacles.
[0028] For example, this embodiment utilizes environmental sensing devices such as surround-view cameras, ultrasonic probes, and lidar to collect perception data of the vehicle's environment, thereby facilitating accurate parking of the vehicle later.
[0029] The above is a preferred embodiment for obtaining initial multimodal data of the vehicle's environment, and does not specifically limit the process and method for obtaining initial multimodal data of the vehicle's environment.
[0030] Step S102: Align the initial multimodal data to obtain target multimodal data, wherein the target multimodal data is used to represent different types of environmental data aligned in time and / or space.
[0031] In the technical solution proposed in step S102 of the present disclosure embodiment, after obtaining the initial multimodal data of the vehicle's environment, the initial multimodal data can be aligned using the time-space synchronization engine in the vehicle in order to obtain the target multimodal data.
[0032] Optionally, the initial multimodal data is aligned temporally and / or spatially to obtain target multimodal data. This alignment process addresses the synchronization issues between data acquired by different sensors in time and / or space, ensuring that the multimodal data can be effectively integrated and utilized, thereby improving the overall sensing accuracy and robustness of the system. The target multimodal data is referred to as the aligned sensing data (corresponding to the sensing data mentioned above).
[0033] Optionally, the alignment process includes aligning the data in time, that is, ensuring that data acquired from different sensors can be compared and fused at the same point in time; the alignment process also includes aligning the initial multimodal data in space, that is, ensuring that the coordinate information acquired by different sensors can be matched with each other to establish a unified coordinate system.
[0034] For example, by using a time-space synchronization engine to align the perception data, we can obtain aligned perception data. Then, we can perform collaborative analysis on the aligned perception data to achieve the goal of accurate parking of the vehicle.
[0035] Step S103: Call the target model to perform collaborative analysis on the target multimodal data to obtain the vehicle information recognition result and the environment information recognition result.
[0036] In the technical solution proposed in step S103 of this embodiment, the target model can be invoked to perform collaborative analysis on the target multimodal data obtained in the above steps to obtain vehicle information recognition results and environmental information recognition results. The target model is a Neural Architecture Search (NAS) model and / or a Knowledge Distillation (KD) model.
[0037] Alternatively, NAS models automatically explore the design space through algorithms to find the architecture that performs best on a given task. Knowledge distillation models are used to transfer knowledge from a large, complex model (called the teacher model) to a smaller, simpler model (called the student model), thereby reducing computation and storage requirements while maintaining performance.
[0038] Optionally, the vehicle information recognition result is called the vehicle body positioning result, or simply vehicle body positioning. The environmental information recognition result is called the obstacle recognition result, or simply obstacle recognition. The collaborative analysis is called intelligent analysis.
[0039] Optionally, a neural network architecture search model and a knowledge distillation model can be invoked to perform collaborative analysis on the target multimodal data to obtain vehicle information recognition results and environmental information recognition results. For example, through a neural network architecture search (NAS) model and a knowledge distillation model, intelligent analysis can be performed on the perception data after alignment processing to complete the vehicle's body localization and obstacle recognition in the vehicle's environment, and output the distance between obstacles and the vehicle body in real time.
[0040] Step S104: Based on the vehicle information recognition results, the environment information recognition results, and the trajectory information matching the vehicle's current execution parameters, determine the parking space for the vehicle and the driving path for the vehicle to drive to the parking space from the environment.
[0041] In the technical solution proposed in step S104 of the present disclosure embodiment, after obtaining the vehicle information identification result and the environment information identification result, the vehicle information identification result, the environment information identification result, and the trajectory information matching the vehicle's current execution parameters can be used to determine the vehicle's parking space and the vehicle's driving path to the parking space in the environment.
[0042] Optionally, the current execution parameters are used to characterize the performance of the actuators in the vehicle, and the trajectory information is used to predict the vehicle's trajectory during movement. The current execution parameters, also called current actuator parameters or simply actuator parameters, characterize the specific performance indicators and operational characteristics of controlling vehicle movement (including forward, reverse, steering, acceleration, deceleration, and braking) when the vehicle executes commands from the automatic parking system. The driving path is called the real-time planned path, or simply the real-time path. The parking space to be parked is called the available parking space.
[0043] In this embodiment, based on the vehicle's information recognition results, the environment's information recognition results, and the vehicle's trajectory information, the parking space to be occupied and the driving path to the parking space are determined. The driving path includes a real-time planned path. For example, after completing vehicle positioning and obstacle recognition in the vehicle's environment, inertial navigation calculation (Dead Reckoning, or DR) can be performed using actuator parameters to predict the vehicle's trajectory. A dynamic collision risk prediction model is then used to assess potential risks in the process, thereby automatically selecting available parking spaces and generating a real-time planned path to precisely control the vehicle to complete parking.
[0044] Step S105: According to the driving path, control the vehicle to drive to the parking space for parking.
[0045] In the technical solution proposed in step S105 of the present disclosure embodiment, the vehicle can be controlled to drive to the parking space according to the driving path. For example, after obtaining the real-time planned path of the vehicle, the vehicle can be controlled to drive to the parking space according to the real-time planned path, so as to realize the control of the vehicle to complete precise parking.
[0046] It is understood that this is a preferred embodiment of controlling a vehicle to park, and the control method and process of controlling the vehicle to park are not specifically limited. As long as the process and method of controlling the vehicle to drive to the parking space according to the driving path are within the protection scope of this disclosure, they will not be listed here.
[0047] In the vehicle parking method of this embodiment, the vehicle also interacts with roadside equipment and terminal equipment. Optionally, the vehicle sends an information subscription request to the roadside equipment. This message subscription request includes specific types of information that the vehicle needs to receive, such as road conditions, traffic signal status, and obstacle warnings ahead. In response to the information subscription request, the roadside equipment sends roadside perception information to the vehicle. For example, the roadside equipment will filter out roadside perception information that meets the vehicle's needs based on its own perception capabilities and stored information, and send it to the vehicle at a certain frequency. In addition to communicating with the roadside equipment, the vehicle also receives driving scenario switching instructions transmitted by the terminal equipment through the network. For example, the driving scenario switching instructions are used to switch the vehicle to energy-saving mode, sport mode, autonomous driving mode, etc., so that the vehicle can adapt to new driving scenarios.
[0048] In steps S101 to S105 of this embodiment, the initial multimodal data is aligned to obtain target multimodal data representing different types of environmental data aligned in time and / or space. Then, a target model is invoked to perform collaborative analysis on the target multimodal data, yielding vehicle and environmental information recognition results. Based on these results, along with trajectory information matching the vehicle's current execution parameters, the parking space and driving path are determined. Following the driving path, the vehicle is controlled to reach the parking space, achieving the purpose of parking. Because this embodiment utilizes a target model to perform collaborative analysis on the vehicle's multimodal data to obtain vehicle and environmental information recognition results, and then determines the parking space and driving path based on these results and the vehicle's trajectory information, the vehicle can be controlled to drive to the parking space according to the driving path, precisely controlling the vehicle to complete parking. This improves parking efficiency and solves the problem of low parking efficiency.
[0049] As an optional implementation method, the target model is invoked to perform collaborative analysis on the target multimodal data to obtain vehicle information recognition results and environment information recognition results. This includes: invoking the target model to perform collaborative analysis on the feature fusion results of the target multimodal data to obtain vehicle information recognition results and environment information recognition results, wherein the feature fusion results are used to represent the fusion results of the features of the target multimodal data.
[0050] In this embodiment, an attention mechanism is used to map features extracted from different modalities in the target multimodal data to the same space, thereby achieving feature fusion of the target multimodal data and obtaining feature fusion results. Then, a neural network architecture search model and a knowledge distillation model are invoked to perform collaborative analysis on the feature fusion results, so as to obtain the information recognition results of the vehicle and the information recognition results of the environment.
[0051] For example, after collecting multimodal data of the target using devices such as surround-view cameras, ultrasonic probes, and lidar, feature-level fusion can be performed on the multimodal data to obtain the feature fusion result. A neural network architecture search model and a knowledge distillation model are then used to intelligently analyze the feature fusion result, enabling vehicle positioning and obstacle recognition, thus improving the obstacle recognition rate of the vehicle under low-light conditions.
[0052] As an optional embodiment, the method further includes: preprocessing the target multimodal data on an edge device of the vehicle; and fusing features of the following data in the preprocessed target multimodal data on the edge device to obtain a feature fusion result: target location data, target point cloud data, and target image data, wherein the target location data is used to represent the location of different objects in the environment, the target point cloud data is used to represent the three-dimensional space of the environment, and the target image data is used to represent the two-dimensional images of different objects in the environment.
[0053] In this embodiment, after obtaining the target multimodal data, the target multimodal data can be preprocessed on the edge device of the vehicle, and the following features in the preprocessed target multimodal data can be fused using an attention mechanism to obtain the feature fusion result: target location data, target point cloud data, and target image data.
[0054] Optionally, the edge device includes an edge computing-based fusion preprocessing component for preprocessing raw data from different sensors (e.g., cameras, radar, lidar, etc.). This preprocessing includes steps such as data cleaning, noise reduction, format conversion, and standardization to efficiently and in real-time process and fuse data from vehicle sensors, enabling more accurate and faster environmental perception.
[0055] Optionally, the target location data is 3D coordinate information, which is simply referred to as 3D coordinates. The target point cloud data is depth point cloud information, which can be simply referred to as depth point cloud. The target image data is image information, which is RGB image information, which can be simply referred to as RGB image.
[0056] For example, the target multimodal data can be preprocessed using an edge computing-based fusion preprocessing component, and then the 3D coordinate information, depth point cloud information and RGB image information in the preprocessed target multimodal data can be fused to obtain the feature fusion result.
[0057] As an optional embodiment, the method further includes at least one of the following: using a temporal convolutional network model to extract features from the target location data with timestamps to obtain the features of the target location data; projecting the voxel features of the target point cloud data onto the plane of a two-dimensional image to obtain the features of the target point cloud data; using an attention mechanism to extract features from consecutive image frames in the target image data; calling a graph convolutional neural network model to extract key point geometric features from the color information in the target image data; and calling a residual convolutional neural network model to perform information recognition and information classification on the target image data to obtain the features of the target image data.
[0058] In this embodiment, a Temporal Convolutional Network (TCN) model can be used to extract features from the timestamped target location data, obtaining the features of the target location data. Then, the voxel features of the target point cloud data are projected onto the plane of the two-dimensional image using the camera's projection matrix, obtaining the features of the target point cloud data. Next, an attention mechanism is used to extract features from consecutive image frames in the target image data, and a Graph Convolutional Network (GCN) model is invoked to extract key point geometric features from the color information in the target image data. Simultaneously, a residual convolutional neural network model is invoked to perform information recognition and classification on the target image data, thereby achieving the acquisition of the target image data features.
[0059] Optionally, the temporal convolutional network model can be simply referred to as a temporal convolutional network. The attention mechanism is a spatio-temporal joint attention mechanism (Spatio-Temporal Transformer, abbreviated as ST-Transformer). The graph convolutional neural network model can be simply referred to as a graph convolutional neural network. Color information can be represented using RGB information.
[0060] Optionally, temporal convolutional networks are used to process sequential data. By using causal convolution and dilated convolution, the receptive field can be expanded without losing any sequential information, effectively capturing long-range dependencies in sequential data, such as sound, text, and video frames. A spatiotemporal joint attention mechanism is used to simultaneously focus on important features of the data in both spatial and temporal dimensions. That is, this mechanism can identify key objects in each frame (corresponding to spatial attention) and track changes in objects over time (corresponding to temporal attention) to provide a more comprehensive understanding of the information.
[0061] For example, a Temporal Convolutional Network (TCN) is used to extract features from 3D coordinates with timestamps to obtain the features of the 3D coordinates. Then, the feature centers of each voxel in the depth point cloud are projected onto the image plane using the camera's projection matrix to obtain the features of the depth point cloud. Further, a spatiotemporal joint attention mechanism is established, and this mechanism is used to extract features from consecutive image frames as input. Then, a graph convolutional neural network is used to extract keypoint geometric features from the RGB information in the image, and a residual convolutional neural network model is combined to identify and classify the image information.
[0062] As an optional implementation, the target model includes a target student model. The target model is invoked to perform collaborative analysis on the feature fusion results of the target multimodal data to obtain vehicle information recognition results and environment information recognition results. This includes: invoking a neural network architecture search model to determine a neural network model that matches the feature fusion results, wherein the performance index of the neural network model under the target task type and target constraints is greater than a performance index threshold. The target task type represents the type of task allowed to be processed by the neural network architecture search model under the environment, the target constraints represent the constraints imposed by the neural network architecture search model on the vehicle's hardware performance, and the performance index represents the degree of performance of the neural network model; using the neural network model as a teacher model and invoking a knowledge distillation model to compress or transfer the knowledge information of the teacher model to the initial student model to obtain the target student model; and using the target student model to perform collaborative analysis on the feature fusion results to obtain vehicle information recognition results and environment information recognition results.
[0063] In this embodiment, a neural network architecture search model can be invoked to determine a neural network model that matches the feature fusion result. This neural network model is then used as the teacher model, and a knowledge distillation model is invoked to compress or transfer the knowledge information from the teacher model to the initial student model, resulting in the target student model within the target model. Furthermore, multiple parallel network branches within the target student model are utilized to perform collaborative analysis of the feature fusion result, achieving the goal of obtaining vehicle and environmental information recognition results.
[0064] Optionally, the neural network model is the neural network architecture that performs optimally under specific task and hardware constraints. The type of target task is called the specific task, such as image classification or object detection. Image classification can be simply referred to as image classification, and object detection as object detection. The target student model is simply referred to as the student model.
[0065] For example, firstly, a Neural Architecture Search (NAS) model is used to automatically design the optimal neural network architecture under specific tasks (e.g., image classification, object detection) and hardware constraints (corresponding to actual computational load, memory, and latency). Then, the NAS model provides a neural network architecture design that performs well under the target task and constraints. Next, a Knowledge Distillation (KD) model is used to jointly optimize the parking perception and control model provided by the NAS model, "compressing" or "transferring" the knowledge of a large, complex, but high-performance model (the teacher model) into a smaller, simpler model (the student model). This allows the student model to maintain high performance while having lower computational overhead and faster inference speed. Finally, the student model can be used to intelligently analyze the feature fusion results obtained above to achieve vehicle positioning and obstacle recognition in the vehicle's environment.
[0066] As an optional embodiment, the vehicle information recognition result includes the vehicle's positioning information, which indicates the vehicle's location in the environment. The environment information recognition result includes the recognition result of obstacles corresponding to the vehicle in the environment, which indicates the location of obstacles in the environment. The vehicle information recognition result and the environment information recognition result are obtained by collaboratively analyzing the feature fusion result using a target student model. This includes: using a target student model to collaboratively analyze the feature fusion result to obtain the positioning information and obstacle recognition result; determining the parking space and the driving path from the environment based on the vehicle information recognition result, the environment information recognition result, and the trajectory information matching the vehicle's current execution parameters; determining the target distance between the location corresponding to the positioning information and the location corresponding to the obstacle recognition result; determining the vehicle's risk analysis result based on the target distance and trajectory information, where the risk analysis result indicates the probability of a collision; and using the risk analysis result, identifying the parking space from the region of interest in the environment and determining the driving path from the region of interest, where the region of interest is generated using scene features of the parking scenario, and different regions of interest are matched with different computational resources.
[0067] In this embodiment, after obtaining the target student model, multiple parallel network branches in the target student model are used to perform collaborative analysis on the feature fusion results in order to obtain the location information and obstacle recognition results.
[0068] Optionally, the vehicle information recognition results include the vehicle's positioning information, which represents the vehicle's position in its environment, such as vehicle coordinates. The environmental information recognition results include the recognition results of obstacles corresponding to the vehicle in the environment, where the obstacle recognition results represent the obstacle's position in the environment, such as obstacle coordinates.
[0069] For example, different environmental data are collected by different environmental sensing devices. After integrating the environmental data collected from different environmental sensing devices, the integrated data is intelligently analyzed by a student model to complete the vehicle's body positioning and obstacle recognition in the environment.
[0070] Optionally, after obtaining the location information, the position corresponding to the location information can be determined first, and then the target distance between the position corresponding to the location information and the position corresponding to the obstacle identification result can be determined. Based on the target distance and trajectory information, a risk analysis result representing the probability of a vehicle collision can be determined. Finally, using this risk analysis result, parking spaces and driving paths are identified from the region of interest in the environment. Here, the target distance is simply referred to as distance. The region of interest is called the dynamic region of interest (ROI).
[0071] For example, the system outputs the target distance between the vehicle and obstacles in real time. Then, based on this target distance and trajectory information, the system predicts the vehicle's trajectory and determines a risk analysis result representing the likelihood of a collision. The trajectory information is matched with actuator parameters. Based on the risk analysis results, a dynamic region of interest (ROI) partitioning strategy is designed for typical parking scenarios. Computational resources are adjusted through adaptive resolution to automatically select available parking spaces and generate a real-time planned path. Finally, the system precisely controls the vehicle to park in an available space according to the real-time planned path.
[0072] As an optional embodiment, the method includes: establishing a collision risk prediction model using trajectory information, the motion states of multiple obstacles in the environment, and semantic information of the environment, wherein the semantic information is used to describe the environment; determining the risk analysis result of the vehicle based on the target distance and trajectory information, including: inputting the target distance and trajectory information into the collision risk prediction model; and using the collision risk prediction model to analyze the target distance and trajectory information to obtain the risk analysis result.
[0073] In this embodiment, a collision risk prediction model can be established using trajectory information, the motion states of multiple obstacles in the environment, and semantic information of the environment. This collision risk prediction model is referred to as a dynamic collision risk prediction model. The motion states are referred to as kinematic states. Semantic information is simply called semantics. Target distance and trajectory information are input into the collision risk prediction model for risk prediction analysis to obtain risk analysis results.
[0074] For example, a collision risk prediction model is established based on trajectory information, the kinematic state of multiple obstacles, and the semantics of the surrounding environment. The trajectory information is derived from the DR (Dash Trajectory) trajectory. Then, the trajectory information, along with the target distance between the vehicle and the obstacle, is used as input to the collision risk prediction model. This dynamic collision risk prediction model assesses the potential risks to the vehicle during parking to obtain the risk analysis results.
[0075] As an optional embodiment, controlling a vehicle to drive to a parking space according to a driving path includes: detecting the slope of the road surface the vehicle is traveling on while controlling the vehicle to drive to the parking space according to the driving path; compensating the current control signal of the vehicle using the slope to obtain a target control signal, wherein the current control signal includes the vehicle's driving parameters; and controlling the vehicle to drive to the parking space in response to the target control signal.
[0076] In this embodiment, during the process of controlling the vehicle to move towards the parking space according to the driving path, it is necessary to detect the slope of the road surface on which the vehicle is traveling. The current control signal of the vehicle is compensated using the slope to obtain a compensated target control signal. The current control signal includes the vehicle's driving parameters, including the steering angle. Based on this target control signal, the vehicle is controlled to move to the parking space to achieve precise parking.
[0077] For example, as a vehicle travels along a real-time planned path to a parking space, the slope of the road surface can be determined. A road slope feedforward compensation mechanism is then introduced to compensate for the vehicle's steering angle. Based on this compensated steering angle, the vehicle is controlled to precisely park in the designated parking space.
[0078] It is understood that this is a preferred implementation method for controlling a vehicle to drive to a parking space for parking, and no specific limitations are made on the process and method of controlling a vehicle to drive to a parking space for parking.
[0079] In this embodiment, the initial multimodal data is aligned to obtain target multimodal data representing different types of environmental data aligned in time and / or space. Then, a target model is invoked to perform collaborative analysis on the target multimodal data, yielding vehicle and environmental information recognition results. Based on the obtained vehicle and environmental information recognition results, and trajectory information matching the vehicle's current execution parameters, the parking space and driving path are determined. Following the driving path, the vehicle can be controlled to drive to the parking space, achieving the purpose of parking. Because this embodiment utilizes a target model to perform collaborative analysis on the vehicle's multimodal data to obtain vehicle and environmental information recognition results, and then determines the parking space and driving path based on these results and the vehicle's trajectory information, the vehicle can be controlled to drive to the parking space according to the driving path, precisely controlling the vehicle to complete parking. This achieves the technical effect of improving vehicle parking efficiency and solves the technical problem of low vehicle parking efficiency.
[0080] The multimodal data involved in this disclosure (including but not limited to data used for alignment processing, collaborative analysis, etc.) are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points shall be provided for users to choose to authorize or refuse.
[0081] The parking method for vehicles will be further explained below with reference to preferred embodiments.
[0082] With the rapid development of intelligent driving technology, especially the automatic parking system as an important component of intelligent driving technology, its performance tuning is becoming a focus of industry competition. When achieving Level 2 (L2+) and above automatic parking functions, the APA system faces challenges from two main technical approaches: multi-sensor fusion solutions and pure vision solutions. While multi-sensor fusion solutions can provide higher environmental perception accuracy, their high hardware costs (approximately 3000 RMB more per vehicle) weaken the market competitiveness of the vehicle model. Furthermore, they present obstacles in terms of data fusion complexity, such as issues with time synchronization, spatial calibration, and semantic alignment. Particularly in rainy or foggy weather, the false alarm rate of millimeter-wave radar increases (to 15%-20%), and conflicts with visual perception results lead to decision delays. On the other hand, pure vision solutions are limited by the inherent defects of cameras, and have bottlenecks in dynamic obstacle response, near-field blind zone coverage, and extreme scene recognition. For example, in low-light or high dynamic range scenarios, the image signal-to-noise ratio drops significantly, leading to a surge in parking line detection error rate (40%-60%). At the same time, the perception blind zone of surround-view cameras and the feature extraction failure rate under extreme weather conditions also limit the applicability of the system in complex environments.
[0083] When vehicles interact with dynamic obstacles, the automatic parking system's prediction response delay to sudden moving targets (such as children or pets) generally exceeds 200ms, resulting in an emergency braking false trigger rate of 12%-15%. Furthermore, the system has poor environmental adaptability, a low success rate in parking in narrow spaces, and suffers from low parking efficiency.
[0084] To address the aforementioned issues, this disclosure proposes a vehicle parking method. This method aligns initial multimodal data to obtain target multimodal data representing different types of environmental data aligned in time and / or space. A target model is then invoked to perform collaborative analysis on the target multimodal data, yielding vehicle and environmental information recognition results. Based on these results, along with trajectory information matching the vehicle's current execution parameters, the parking space and driving path are determined. Following this path, the vehicle is controlled to reach the parking space, achieving the parking objective. By utilizing a target model for collaborative analysis of the vehicle's multimodal data to obtain vehicle and environmental information recognition results, and then determining the parking space and driving path based on these results and the vehicle's trajectory information, the method allows the vehicle to be controlled to reach the parking space according to the chosen path, precisely controlling the vehicle to complete parking. This significantly improves parking efficiency and solves the problem of low parking efficiency.
[0085] Figure 2 is a flowchart of another vehicle parking method according to an embodiment of the present disclosure. As shown in Figure 2, the method may include the following steps.
[0086] Step S201: Identify the three sensing devices: surround view camera, ultrasonic probe, and lidar.
[0087] In this embodiment, three sensing devices are first identified: a surround-view camera, an ultrasonic sensor, and a lidar sensor. These three devices are then used to collect perception data about the vehicle's environment, facilitating accurate parking of the vehicle subsequently.
[0088] Step S202: Collect 3D coordinate information, depth point cloud information, and RGB image information.
[0089] In this embodiment, 3D coordinate information, depth point cloud information, and RGB image information are acquired through a surround-view camera, an ultrasonic probe, and a LiDAR, respectively. Different environmental information is provided for each type of sensor to generate depth point cloud information, which in turn provides RGB image information.
[0090] Step S203: Analyze the 3D coordinate information, depth point cloud information, and RGB image information using a neural network architecture search model and a knowledge distillation model.
[0091] In this embodiment, a Neural Architecture Search (NAS) model is first used to automatically design the optimal neural network architecture under specific tasks (e.g., image classification, object detection) and hardware constraints (corresponding to actual computational load, memory, and latency). Then, the NAS model provides a neural network architecture design that performs well under the target task and constraints. Next, a Knowledge Distillation (KD) model is used to jointly adjust the parking perception and control model provided by the NAS model, "compressing" or "transferring" the knowledge of a large, complex, but high-performance model (e.g., a teacher model) into a smaller, simpler model (e.g., a student model). This allows the student model to maintain high performance while having lower computational overhead and faster inference speed. Finally, the student model can be used to intelligently analyze the feature fusion results obtained above to achieve vehicle positioning and obstacle recognition in the vehicle's environment.
[0092] Optionally, a spatiotemporal joint attention mechanism can be established, with consecutive frame images used as input. Graph convolutional neural networks can extract key point geometric features from the RGB information in images, and combine them with residual convolutional neural network structures to identify and classify image information. Furthermore, during the data preprocessing stage, the obtained 3D coordinate information, depth information, and timestamps are integrated to improve the diversity and completeness of the dataset.
[0093] Step S204: Determine the vehicle coordinates and the obstacle recognition results.
[0094] In this embodiment, after integrating the obtained 3D coordinate information, depth point cloud information, and RGB image information, the integrated 3D coordinate information, depth point cloud information, and RGB image information are intelligently analyzed through a neural network architecture search model and a knowledge distillation model to complete vehicle positioning and obstacle recognition.
[0095] Step S205: Output the distance between the obstacle and the vehicle body.
[0096] In this embodiment, after intelligent analysis of the integrated data through a neural network architecture search model and a knowledge distillation model, the distance between the obstacle and the vehicle body can be output in real time.
[0097] Step S206: Determine the trajectory information based on the actuator parameters.
[0098] In this embodiment, the DR trajectory can be calculated by combining the actuator parameters to determine the trajectory information.
[0099] Step S207: Invoke the dynamic collision risk prediction model.
[0100] In this embodiment, a dynamic collision risk prediction model is invoked to analyze the trajectory information and the distance between the obstacle and the vehicle body. The dynamic collision risk prediction model is used to analyze the trajectory information and the distance between the obstacle and the vehicle body to assess potential risks and obtain risk analysis results.
[0101] Step S208: Determine available parking spaces and real-time routes.
[0102] In this embodiment, based on the risk analysis results, available parking spaces can be automatically selected, and a real-time planned route can be generated to guide the vehicle to the available parking space.
[0103] Step S209, parking complete.
[0104] In this embodiment, the vehicle can be controlled to drive to a parking space according to a real-time planned path, achieving precise control of the vehicle to complete parking and ensuring the safety and efficiency of parking.
[0105] In this embodiment, multiple network models are employed to extract the obtained information. A Generative Convolutional Network (GCN) is used for dense sampling at different locations in the image. Convolution is used to predict different feature maps to extract the test results. Through convolution, a homogeneous transformation matrix is obtained to establish the relationship between the 2D mask and the 3D points. The projection matrix of the camera projects the feature center of each voxel onto the image plane. A Temporal Convolutional Network (TCN) is used to extract features from 3D coordinates with timestamps. By stacking multiple TCN layers, the network model learns spatial features at different scales. A collision risk prediction model is established, and based on the performance of the actual vehicle actuators, DR trajectory estimation is performed to obtain trajectory information. Based on the trajectory information, the kinematic state of multiple obstacles, and the semantics of the surrounding environment, a collision risk prediction model is established. In a parking lot environment without a positioning system, real-time road slope compensation is used to avoid trajectory distortion in sloping scenes.
[0106] In this embodiment, an intelligent parking system for high-level autonomous driving is proposed. The system includes: a multimodal fusion perception subsystem, a hierarchical progressive decision architecture subsystem, a high-precision execution control subsystem, a functional safety assurance subsystem, a system integration and verification subsystem, and an evolution roadmap subsystem. The above subsystems are further described below.
[0107] The multimodal fusion perception subsystem employs a heterogeneous sensor collaborative perception module, configured with multispectral visual perception components, a millimeter-wave radar array, and a solid-state LiDAR network. The vision system consists of a surround-view array of six 2-megapixel fisheye cameras, employing optical distortion correction algorithms to eliminate wide-angle distortion and adaptive white balance technology to ensure all-weather imaging quality. The radar system integrates 12 fifth-generation ultrasonic sensors and four 77 GHz millimeter-wave radars, using a spatial diversity deployment strategy to eliminate detection blind spots. Furthermore, by introducing a solid-state LiDAR point cloud compensation mechanism, feature-level fusion improves obstacle recognition rates under low-light conditions.
[0108] Optionally, after obtaining the perceived data, the perceived data can be aligned using a time-space synchronization engine, and an improved probabilistic correlation algorithm based on edge computing fusion preprocessing components is used to solve the multi-target tracking problem. The Transformer attention mechanism is innovatively introduced into multi-source data for feature extraction, constructing a hierarchical feature representation space. For typical parking scenarios, a dynamic Region of Interest (ROI) partitioning strategy is designed, adjusting and optimizing computational resources through adaptive resolution.
[0109] The hierarchical, progressive decision-making architecture subsystem adopts a three-tiered module structure: "Scene Understanding - Behavioral Decision-Making - Motion Planning." The scene understanding module extracts structured information such as parking lines and traffic signs through a semantic segmentation network, and combines this with historical trajectory analysis to build an environmental cognitive map. The behavioral decision-making module applies a hybrid decision-making model combining finite state machines and reinforcement learning. In structured scenarios, a rule engine ensures safety, while in unstructured scenarios, a machine learning algorithm (Q-learning algorithm) enhances adaptability.
[0110] Optionally, the motion planning module proposes a spatiotemporal joint optimization method. The global planner employs an improved random sampling subgraph search algorithm to generate an initial path while satisfying vehicle kinematic constraints. Local planning introduces an elastic band deformation mechanism, dynamically avoids obstacles through a potential field function, and achieves trajectory smoothing by combining a model predictive control framework.
[0111] The high-precision execution control subsystem employs a dual-loop proportional-integral-derivative (PID) controller architecture for longitudinal control. The inner loop controls the drive motor torque, while the outer loop adjusts vehicle speed and position deviation, incorporating a road slope feedforward compensation mechanism. Lateral control is based on a feedforward-feedback composite control steering model. The feedforward component calculates the theoretical steering angle based on the path curvature, while the feedback component achieves trajectory tracking through heading deviation PID adjustment.
[0112] The functional safety assurance subsystem employs a heterogeneous dual-computing platform architecture at the hardware level. The main control component utilizes a multi-core processor, while the safety monitoring component is equipped with an independent microprocessor. Dual redundancy verification is achieved through heartbeat detection and output comparison. Regarding expected functional safety, a multi-dimensional test verification matrix is established. This matrix includes an environmental dimension covering 12 weather conditions such as rain, fog, and strong light; a scenario dimension encompassing over 200 special operating conditions such as irregularly shaped parking spaces and dynamic obstacles; and a fault dimension simulating 54 abnormal states such as sensor failure and communication interruption. Boundary test cases are generated using a formal verification toolchain, and probabilistic risk assessment is conducted using the Monte Carlo method.
[0113] The system integration and verification subsystem develops a modular simulation testing platform and constructs a digital twin system including a physics engine, sensor models, and vehicle dynamics models. The virtual test field imports a global library of typical parking lot 3D models, supporting realistic environment reconstruction such as sensor noise injection and communication latency simulation. Real-vehicle testing employs a scenario-based automated testing framework, using a cloud-based test management platform for remote test case distribution and real-time data feedback. The verification process follows the integration testing flow on the right side of the development and testing model (e.g., the V-model): component testing covers over 97% of the condition coverage standards (MC / DC criterion) in software testing; integration testing focuses on verifying interface consistency between modules; system testing assesses functional completeness and performance indicators; and acceptance testing completes over 5000 parking cycle verifications in a real parking lot.
[0114] The evolution roadmap subsystem will focus on breakthroughs in the following technological directions for the next generation of the system: a cognitive decision-making architecture based on neural symbol fusion, combining knowledge graphs and deep learning to achieve human-like reasoning; a vehicle-parking collaborative perception network, using vehicle communication technology to obtain high-precision semantic maps of parking lots; quantum-inspired optimization algorithms to improve real-time planning capabilities in complex scenarios; a self-supervised learning framework to reduce data annotation costs and enhance model generalization; and an interpretable intelligent module to build a toolchain for visual analysis of the decision-making process.
[0115] In this embodiment, a multimodal fusion perception subsystem, a hierarchical progressive decision architecture subsystem, a high-precision execution control subsystem, a functional safety assurance subsystem, a system integration and verification subsystem, and an evolution path subsystem are used to address the technical bottlenecks of traditional automatic parking systems in terms of environmental adaptability, decision intelligence, and control precision.
[0116] In this embodiment, the initial multimodal data is aligned to obtain target multimodal data representing different types of environmental data aligned in time and / or space. Then, a target model is invoked to perform collaborative analysis on the target multimodal data, yielding vehicle and environmental information recognition results. Based on the obtained vehicle and environmental information recognition results, and trajectory information matching the vehicle's current execution parameters, the parking space and driving path are determined. Following the driving path, the vehicle can be controlled to drive to the parking space, achieving the purpose of parking. Because this embodiment utilizes a target model to perform collaborative analysis on the vehicle's multimodal data to obtain vehicle and environmental information recognition results, and then determines the parking space and driving path based on these results and the vehicle's trajectory information, the vehicle can be controlled to drive to the parking space according to the driving path, precisely controlling the vehicle to complete parking. This achieves the technical effect of improving vehicle parking efficiency and solves the technical problem of low vehicle parking efficiency.
[0117] According to an embodiment of this disclosure, a system embodiment of a vehicle parking system is provided. It should be noted that this system is used to execute the above-described vehicle parking method.
[0118] This embodiment provides a vehicle parking system. FIG3 is a schematic diagram of a vehicle parking system according to an embodiment of the present disclosure. As shown in FIG3, the vehicle parking system includes: multiple sensing devices 301, a processor 302 and a controller 303.
[0119] Multiple sensing devices 301 are used to acquire target location data, target point cloud data and target image data of the environment in which the vehicle is located. The target location data is used to represent the location of different objects in the environment in which the vehicle is located, the target point cloud data is used to represent the three-dimensional space of the environment, and the target image data is used to represent the two-dimensional images of different objects in the environment.
[0120] The processor 302 is used to align target location data, target point cloud data, and target image data in time and / or space to obtain target multimodal data; call the target model to perform collaborative analysis on the target multimodal data to obtain vehicle information recognition results and environment information recognition results; based on the vehicle information recognition results, environment information recognition results, and trajectory information matching the vehicle's current execution parameters, determine the vehicle's parking space and the vehicle's driving path to the parking space from the environment, wherein the current execution parameters are used to characterize the execution performance of the actuators in the vehicle, and the trajectory information is used to predict the vehicle's trajectory during movement.
[0121] The controller 303 is used to control the vehicle to drive to the parking space according to the driving path.
[0122] Optionally, the processor 302 is further configured to invoke the target model to perform collaborative analysis on the target multimodal data through the following steps to obtain vehicle information recognition results and environment information recognition results: perform collaborative analysis on the feature fusion results of the target multimodal data to obtain vehicle information recognition results and environment information recognition results, wherein the feature fusion results are used to represent the fusion results of the features of the target multimodal data.
[0123] Optionally, the processor 302 is further configured to preprocess the target multimodal data on an edge device of the vehicle; on the edge device, the features of the following data in the preprocessed target multimodal data are fused to obtain the feature fusion result: target location data, target point cloud data and target image data, wherein the target location data is used to represent the location of different objects in the environment, the target point cloud data is used to represent the three-dimensional space of the environment, and the target image data is used to represent the two-dimensional images of different objects in the environment.
[0124] Optionally, the processor 302 is further configured to: utilize a temporal convolutional network model to extract features from the time-stamped target location data to obtain the features of the target location data; project the voxel features of the target point cloud data onto the plane of the two-dimensional image to obtain the features of the target point cloud data; utilize an attention mechanism to extract features from consecutive image frames in the target image data; call a graph convolutional neural network model to extract key point geometric features from the color information in the target image data; and call a residual convolutional neural network model to perform information recognition and information classification on the target image data to obtain the features of the target image data.
[0125] Optionally, the target model includes a target student model. The processor 302 is further configured to invoke the target model through the following steps to perform collaborative analysis on the feature fusion results of the target multimodal data, thereby obtaining vehicle information recognition results and environment information recognition results: Invoking a neural network architecture search model to determine a neural network model that matches the feature fusion results, wherein the performance index of the neural network model under the target task type and target constraints is greater than a performance index threshold, the target task type being used to represent the type of task allowed to be processed by the neural network architecture search model under the environment, the target constraints being used to represent the constraints imposed by the neural network architecture search model on the vehicle's hardware performance, and the performance index being used to represent the degree of performance of the neural network model; using the neural network model as a teacher model and invoking a knowledge distillation model to compress or transfer the knowledge information of the teacher model to the initial student model, thereby obtaining the target student model; and using the target student model to perform collaborative analysis on the feature fusion results, thereby obtaining vehicle information recognition results and environment information recognition results.
[0126] Optionally, the vehicle information recognition result includes the vehicle's positioning information, which indicates the vehicle's location in the environment. The environment information recognition result includes the recognition result of obstacles corresponding to the vehicle in the environment, which indicates the location of obstacles in the environment. The processor 302 is further configured to perform collaborative analysis on the feature fusion result using the target student model through the following steps to obtain the vehicle information recognition result and the environment information recognition result: performing collaborative analysis on the feature fusion result using the target student model to obtain the positioning information and obstacle recognition result; the processor 302 is further configured to perform collaborative analysis on the feature fusion result using the target student model to obtain the positioning information and obstacle recognition result; the processor 302 is further configured to perform the following steps based on the vehicle information recognition result and the environment information... The system uses the vehicle's current execution parameters and the identified location information, along with the trajectory information, to determine the parking space and the driving path from the environment. It also determines the target distance between the location information and the identified location of any obstacle. Based on the target distance and trajectory information, it determines the vehicle's risk analysis results, which represent the likelihood of a collision. Using the risk analysis results, it identifies the parking space from the region of interest (ROI) in the environment and determines the driving path. The ROI is generated using scene features of the parking scenario, and different ROIs require different computational resources.
[0127] Optionally, the processor 302 is further configured to control the vehicle to drive to the parking space for parking according to the driving path through the following steps: establishing a collision risk prediction model using trajectory information, the motion state of multiple obstacles in the environment, and semantic information of the environment, wherein the semantic information is used to describe the environment; the processor 302 is further configured to determine the risk analysis result of the vehicle based on the target distance and trajectory information through the following steps: inputting the target distance and trajectory information into the collision risk prediction model; and using the collision risk prediction model to analyze the target distance and trajectory information to obtain the risk analysis result.
[0128] Optionally, the controller 303 is further configured to detect the slope of the road surface on which the vehicle is traveling during the process of controlling the vehicle to travel towards the parking space according to the driving path; compensate the current control signal of the vehicle using the slope to obtain a target control signal, wherein the current control signal includes the driving parameters of the vehicle; and control the vehicle to travel to the parking space for parking in response to the target control signal.
[0129] In this embodiment, multiple sensing devices are used to acquire target location data, target point cloud data, and target image data of the vehicle's environment. The target location data represents the positions of different objects in the environment, the target point cloud data represents the three-dimensional space of the environment, and the target image data represents the two-dimensional images of different objects in the environment. A processor aligns the target location data, target point cloud data, and target image data in time and / or space to obtain target multimodal data. A target model is invoked to perform collaborative analysis on the target multimodal data, yielding vehicle information recognition results and environment information recognition results. Based on the vehicle information recognition results, the environment information recognition results, and trajectory information matching the vehicle's current execution parameters, the parking space to be occupied and the driving path to the parking space are determined from the environment. The current execution parameters characterize the execution performance of the actuators in the vehicle, and the trajectory information predicts the vehicle's trajectory during movement. A controller then controls the vehicle to drive to the parking space according to the driving path, thereby improving the vehicle's parking efficiency and solving the technical problem of low parking efficiency.
[0130] Embodiments of this disclosure also provide a vehicle, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods described in various embodiments of this disclosure when it runs.
[0131] Embodiments of this disclosure also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium resides to perform the methods of the various embodiments of this disclosure.
[0132] Embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of this disclosure.
[0133] Embodiments of this disclosure also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods described in various embodiments of this disclosure.
[0134] Embodiments of this disclosure also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of this disclosure.
[0135] In the above embodiments of this disclosure, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0136] In the several embodiments provided in this disclosure, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are illustrative; for example, the division of components is a logical functional division, and in actual implementation, there may be other division methods, such as multiple components or components combined or integrated into another system, or some features being ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed between components or modules is through some interface, indirect coupling or communication connection, and is electrical or other forms.
[0137] The components described as separate parts may or may not be physically separate. The parts shown as components may or may not be physical components; that is, they may be located in one place or distributed across multiple components. Some or all of the components are selected according to actual needs to achieve the purpose of this embodiment.
[0138] Furthermore, in the various embodiments of this disclosure, the functional components are integrated into a single processing component, or each component exists physically separately, or two or more components are integrated into one component. The integrated components are implemented either in hardware or as software functional components.
[0139] If the integrated components are implemented as software functional components and sold or used as independent products, they are stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, is embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media for storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.
[0140] The above description represents the preferred embodiments of this disclosure. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this disclosure, and these improvements and modifications should also be considered within the scope of protection of this disclosure. Industrial applicability
[0141] One or more embodiments of this disclosure provide a vehicle parking method. This method employs a collaborative analysis of multimodal data. Initial multimodal data is aligned to obtain target multimodal data representing different types of environmental data aligned in time and / or space. A target model is then invoked to perform collaborative analysis on the target multimodal data, yielding vehicle and environmental information identification results. Based on these results, along with trajectory information matching the vehicle's current execution parameters, the parking space and driving path are determined. Following the driving path, the vehicle is controlled to drive to the parking space, achieving the purpose of parking. In other words, because the embodiments of this disclosure utilize the target model to perform collaborative analysis on the multimodal data of the vehicle to obtain the vehicle information recognition results and the environmental information recognition results, and then based on the vehicle information recognition results, the environmental information recognition results, and the vehicle's trajectory information, the parking space and driving path of the vehicle can be determined, so that the vehicle can be controlled to drive to the parking space according to the driving path, so as to accurately control the vehicle to complete the parking, avoiding the inability to park in a timely manner when the vehicle is in a complex scenario, thereby achieving the technical effect of improving the parking efficiency of the vehicle and solving the technical problem of low parking efficiency of the vehicle.
Claims
1. A method for parking a vehicle, comprising: Acquire initial multimodal data of the vehicle's environment, wherein the initial multimodal data is used to represent different types of environmental data collected by different types of environmental sensing devices; The initial multimodal data is aligned to obtain target multimodal data, wherein the target multimodal data is used to represent different types of environmental data aligned in time and / or space; The target model is invoked to perform collaborative analysis on the target multimodal data to obtain the vehicle information recognition result and the environment information recognition result; Based on the vehicle information recognition result, the environment information recognition result, and the trajectory information matching the vehicle's current execution parameters, the parking space for the vehicle is determined from the environment, and the driving path for the vehicle to drive to the parking space is determined from the environment. The current execution parameters are used to characterize the execution performance of the actuators in the vehicle, and the trajectory information is used to predict the trajectory of the vehicle during its movement. According to the driving path, control the vehicle to drive to the parking space for parking.
2. The method according to claim 1, wherein, The target model is invoked to perform collaborative analysis on the target multimodal data to obtain the vehicle information recognition result and the environment information recognition result, including: The target model is invoked to perform collaborative analysis on the feature fusion results of the target multimodal data to obtain the vehicle information recognition results and the environment information recognition results, wherein the feature fusion results are used to represent the fusion results of the features of the target multimodal data.
3. The method according to claim 2, wherein, The method further includes: The target multimodal data is preprocessed on the edge device of the vehicle; On the edge device, features of the following data in the preprocessed target multimodal data are fused to obtain the feature fusion result: target location data, target point cloud data, and target image data. The target location data is used to represent the location of different objects in the environment, the target point cloud data is used to represent the three-dimensional space of the environment, and the target image data is used to represent the two-dimensional images of different objects in the environment.
4. The method according to claim 3, wherein, The method further includes: By using a temporal convolutional network model, feature extraction is performed on the target location data with timestamps to obtain the features of the target location data.
5. The method according to claim 3, wherein, The method further includes: The voxel features of the target point cloud data are projected onto the plane of the two-dimensional image to obtain the features of the target point cloud data.
6. The method according to claim 3, wherein, The method further includes: By utilizing an attention mechanism, feature extraction is performed on consecutive image frames in the target image data. A graph convolutional neural network model is invoked to extract key point geometric features from the color information in the target image data. Additionally, a residual convolutional neural network model is invoked to perform information recognition and classification on the target image data, thereby obtaining the features of the target image data.
7. The method according to claim 2, wherein, The target model includes a target student model. By calling the target model, the feature fusion results of the target multimodal data are collaboratively analyzed to obtain the vehicle information recognition results and the environment information recognition results, including: A neural network architecture search model is invoked to determine a neural network model that matches the feature fusion result. The neural network model has a performance index greater than a performance index threshold under the target task type and target constraints. The target task type is used to represent the type of task allowed to be processed by the neural network architecture search model in the environment. The target constraints are used to represent the constraints imposed by the neural network architecture search model on the hardware performance of the vehicle. The performance index is used to represent the degree of performance of the neural network model. The neural network model is used as the teacher model, and the knowledge distillation model is invoked to compress or transfer the knowledge information of the teacher model to the initial student model to obtain the target student model. The target student model is used to perform collaborative analysis on the feature fusion results to obtain the vehicle information recognition results and the environment information recognition results.
8. The method according to claim 7, wherein, The vehicle information recognition result includes the vehicle's location information, which indicates the vehicle's position in the environment. The environment information recognition result includes the recognition result of obstacles corresponding to the vehicle in the environment, which indicates the location of the obstacles in the environment. The target student model is used to perform collaborative analysis on the feature fusion result to obtain the vehicle information recognition result and the environment information recognition result, including: The target student model is used to perform collaborative analysis on the feature fusion results to obtain the positioning information and the obstacle recognition results.
9. The method according to claim 8, wherein, The target student model is used to perform collaborative analysis on the feature fusion results to obtain the location information and the obstacle recognition results, including: By utilizing multiple parallel network branches in the target student model, the feature fusion results are collaboratively analyzed to obtain the positioning information and the obstacle recognition results.
10. The method according to claim 8, wherein, Based on the vehicle's information recognition results, the environment's information recognition results, and trajectory information matching the vehicle's current execution parameters, the system determines the vehicle's parking space from the environment and the vehicle's driving path to the parking space from the environment, including: Determine the target distance between the location corresponding to the positioning information and the location corresponding to the obstacle identification result; Based on the target distance and the trajectory information, a risk analysis result for the vehicle is determined, wherein the risk analysis result is used to represent the degree of probability of the vehicle being involved in a collision; Using the risk analysis results, the parking space to be parked is identified from the region of interest in the environment, and the driving path is determined from the region of interest in the environment. The region of interest is generated using the scene features of the parking scenario, and different regions of interest are matched with different computing resources.
11. The method according to claim 10, wherein, Based on the target distance and the trajectory information, the risk analysis result of the vehicle is determined, including: The target distance and the trajectory information are input into the collision risk prediction model; The collision risk prediction model is used to analyze the target distance and the trajectory information to obtain the risk analysis results.
12. The method according to claim 11, wherein, The method includes: The collision risk prediction model is established using the trajectory information, the motion states of multiple obstacles in the environment, and the semantic information of the environment, wherein the semantic information is used to describe the environment.
13. The method according to any one of claims 1 to 12, wherein, According to the driving path, controlling the vehicle to drive to the parking space for parking includes: While controlling the vehicle to drive towards the parking space according to the driving path, the slope of the road surface on which the vehicle is driving is detected. The current control signal of the vehicle is compensated using the slope to obtain a target control signal, wherein the current control signal includes the vehicle's driving parameters; In response to the target control signal, the vehicle is controlled to drive to the parking space for parking.
14. A parking system for a vehicle, comprising: Multiple sensing devices are used to acquire target location data, target point cloud data, and target image data of the environment in which the vehicle is located. The target location data is used to represent the location of different objects in the environment in which the vehicle is located. The target point cloud data is used to represent the three-dimensional space of the environment. The target image data is used to represent the two-dimensional images of different objects in the environment. The processor is configured to perform alignment processing on the target location data, the target point cloud data, and the target image data in time and / or space to obtain target multimodal data; invoke a target model to perform collaborative analysis on the target multimodal data to obtain information recognition results of the vehicle and information recognition results of the environment; based on the information recognition results of the vehicle and the information recognition results of the environment, and trajectory information matching the current execution parameters of the vehicle, determine the parking space of the vehicle in the environment, and determine the driving path of the vehicle to the parking space in the environment, wherein the current execution parameters are used to characterize the execution performance of the actuators in the vehicle, and the trajectory information is used to predict the trajectory of the vehicle during movement; The controller is used to control the vehicle to drive to the parking space for parking according to the driving path.
15. A vehicle comprising: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 13.
