Vehicle parking method and system and vehicle

Abstract

The embodiment of the invention provides a vehicle parking method and system and a vehicle. The method comprises the steps that initial multi-modal data of the environment where the vehicle is located is obtained; performing alignment processing on the initial multi-modal data to obtain target multi-modal data; calling a target model to carry out collaborative analysis on the target multi-modal data to obtain an information identification result of the vehicle and an information identification result of the environment; on the basis of the information identification result of the vehicle, the information identification result of the environment and track information matched with current execution parameters of the vehicle, a to-be-parked parking space of the vehicle and a driving path of the vehicle to be driven to the to-be-parked parking space are determined from the environment, and the current execution parameters are used for representing execution performance of an actuator in the vehicle; the track information is used for predicting the track of the vehicle in the moving process; and according to the driving path, the vehicle is controlled to drive to the to-be-parked space for parking. The technical problem that the parking efficiency of the vehicle is low is solved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of vehicle technology, and in particular, to a vehicle parking method, system, and vehicle. Background Art

[0002] When the vehicle is in a dynamic obstacle interaction scenario, the automatic parking system's predicted response delay to sudden moving targets (such as children, pets, etc.) generally exceeds 200 milliseconds (ms), resulting in the vehicle's emergency braking false trigger rate reaching 12%-15%. The system also has poor environmental adaptability and a low parking success rate for narrow parking spaces (for example, width < vehicle width + 35 cm), resulting in a technical problem of low vehicle parking efficiency.

[0003] There is currently no good solution to the above problems. Summary of the Invention

[0004] Embodiments of the present application provide a vehicle parking method, system, and vehicle to at least solve the technical problem of low vehicle parking efficiency.

[0005] According to one aspect of an embodiment of the present application, a vehicle parking method is provided. The method may include: acquiring initial multimodal data of an environment in which the vehicle is located, wherein the initial multimodal data is used to represent different types of environmental data collected by different types of environmental sensing devices; performing alignment processing on the initial multimodal data to obtain target multimodal data, wherein the target multimodal data is used to represent different types of environmental data aligned in time and / or space; calling a target model to perform collaborative analysis on the target multimodal data to obtain vehicle information recognition results and environment information recognition results; based on the vehicle information recognition results, the environment information recognition results, and track information matching the vehicle's current execution parameters, determining the vehicle's parking space and the vehicle's driving path to the parking space from the environment, wherein the current execution parameters are used to characterize the execution performance of actuators in the vehicle, and the track information is used to predict the vehicle's trajectory during motion; and controlling the vehicle to drive to the parking space according to the driving path for parking.

[0006] Furthermore, the target model is called to perform collaborative analysis on the target multimodal data to obtain vehicle information recognition results and environment information recognition results, including: fusing the features of the target multimodal data to obtain feature fusion results; calling the target model to perform collaborative analysis on the feature fusion results of the target multimodal data to obtain vehicle information recognition results and environment information recognition results, wherein the feature fusion results are used to represent the fusion results of the features of the target multimodal data.

[0007] Furthermore, the method also includes: preprocessing the target multimodal data on the edge device of the vehicle; and fusing the features of the following data in the preprocessed target multimodal data on the edge device to obtain a feature fusion result: target position data, target point cloud data, and target image data, wherein the target position data is used to represent the positions of different objects in the environment, the target point cloud data is used to represent the three-dimensional space of the environment, and the target image data is used to represent the two-dimensional images of different objects in the environment.

[0008] Furthermore, the method also includes at least one of the following: using a temporal convolutional network model to extract features of the target position data with timestamps to obtain features of the target position data; projecting the voxel features of the target point cloud data onto the plane of the two-dimensional image to obtain features of the target point cloud data; using an attention mechanism to extract features of continuous image frames in the target image data, calling a graph convolutional neural network model to extract key point geometric features from the color information in the target image data, and calling a residual convolutional neural network model to perform information recognition and information classification on the target image data to obtain features of the target image data.

[0009] Furthermore, the target model includes a target student model, and the target model is called to collaboratively analyze the feature fusion results of the target multimodal data to obtain the vehicle information recognition results and the environment information recognition results, including: calling the neural network architecture search model to determine the neural network model that matches the feature fusion results, wherein the performance index of the neural network model under the target task type and target constraint conditions is greater than the performance index threshold, the target task type is used to indicate the type of task in the environment that the neural network architecture search model is allowed to process, the target constraint condition is used to indicate the constraint of the neural network architecture search model on the hardware performance of the vehicle, and the performance index is used to indicate the quality of the performance of the neural network model; the neural network model is used as a teacher model, and the knowledge distillation model is called to compress or transfer the knowledge information of the teacher model to the initial student model to obtain the target student model; the target student model is used to collaboratively analyze the feature fusion results to obtain the vehicle information recognition results and the environment information recognition results.

[0010] Furthermore, the vehicle information recognition result includes the vehicle's positioning information, which is used to indicate the vehicle's position in the environment. The environment information recognition result includes the recognition result of an obstacle corresponding to the vehicle in the environment, which is used to indicate the position of the obstacle in the environment. The target student model is used to collaboratively analyze the feature fusion result to obtain the vehicle information recognition result and the environment information recognition result, including: using the target student model to collaboratively analyze the feature fusion result to obtain the positioning information and the obstacle recognition result; based on the vehicle information recognition result, the environment information recognition result, and the track information matched with the vehicle's current execution parameters, determining the vehicle's parking space and the vehicle's driving path to the parking space from the environment, including: determining a target distance between the position corresponding to the positioning information and the position corresponding to the obstacle recognition result; determining a risk analysis result of the vehicle based on the target distance and the track information, wherein the risk analysis result is used to indicate the probability of a collision risk of the vehicle; using the risk analysis result, identifying the parking space from a region of interest in the environment and determining the driving path, wherein the region of interest is generated using scene features of the parking scene, and different regions of interest are matched with different computing resources.

[0011] Furthermore, the method includes: establishing a collision risk prediction model using track information, the motion status of multiple obstacles in the environment and semantic information of the environment, wherein the semantic information is used to describe the environment; determining the risk analysis result of the vehicle based on the target distance and track information, including: inputting the target distance and track information into the collision risk prediction model; using the collision risk prediction model to analyze the target distance and track information to obtain the risk analysis result.

[0012] Furthermore, the vehicle is controlled to travel to a parking space to be parked according to the driving path, including: in the process of controlling the vehicle to travel to the parking space to be parked according to the driving path, detecting the slope of the road on which the vehicle is traveling; using the slope to compensate for a current control signal of the vehicle to obtain a target control signal, wherein the current control signal includes a driving parameter of the vehicle; and in response to the target control signal, controlling the vehicle to travel to the parking space to be parked.

[0013] According to another aspect of the embodiments of the present application, a parking system of a vehicle is also provided, comprising: a plurality of sensing devices respectively configured to obtain target position data, target point cloud data and target image data of an environment in which the vehicle is located, wherein the target position data is used to represent positions of different objects in the environment, the target point cloud data is used to represent a three-dimensional space of the environment, and the target image data is used to represent two-dimensional images of different objects in the environment; a processor configured to perform alignment processing on the target position data, the target point cloud data and the target image data in time and / or space to obtain target multi-modal data; invoke a target model to perform collaborative analysis on the target multi-modal data to obtain information recognition results of the vehicle and information recognition results of the environment; determine a parking space to be parked in the vehicle from the environment based on the information recognition results of the vehicle, the information recognition results of the environment, and track information matched with a current execution parameter of the vehicle, and a driving path of the vehicle to be driven to the parking space to be parked in, wherein the current execution parameter is used to represent an execution performance of an executor in the vehicle, and the track information is used to predict a trajectory of the vehicle in a motion process; and a controller configured to control the vehicle to drive to the parking space to be parked in according to the driving path.

[0014] According to another aspect of the embodiments of the present application, a vehicle is also provided, comprising: a memory storing an executable program; and a processor configured to run the program, wherein the program performs the method in the embodiments of the present application when the program is run.

[0015] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, comprising a stored executable program, wherein the computer readable storage medium controls a device in which the computer readable storage medium is located to perform the method in the embodiments of the present application when the executable program is run.

[0016] According to another aspect of the embodiments of the present application, a computer program product is also provided, comprising a computer program, which, when executed by a processor, implements the method in the embodiments of the present application.

[0017] According to another aspect of the embodiments of the present application, a computer program product is also provided, comprising a non-volatile computer readable storage medium storing a computer program, which, when executed by a processor, implements the method in the embodiments of the present application.

[0018] According to another aspect of the embodiments of the present application, a computer program is also provided, which, when executed by a processor, implements the method in the embodiments of the present application.

[0019] In the embodiment of the present application, the multi-modal data is analyzed in a collaborative manner, the initial multi-modal data is aligned to obtain target multi-modal data representing different types of environment data aligned in time and / or space, and then the target model is called to collaboratively analyze the target multi-modal data, so as to obtain the information recognition result of the vehicle and the information recognition result of the environment. The information recognition result of the vehicle, the information recognition result of the environment, and the track information matching the current execution parameter of the vehicle are determined to determine the parking space and the driving path of the vehicle. According to the driving path, the vehicle can be controlled to drive to the parking space, thereby achieving the purpose of parking the vehicle. That is, in the present application, the multi-modal data of the vehicle is analyzed by the target model to obtain the information recognition result of the vehicle and the information recognition result of the environment. Then, the information recognition result of the vehicle, the information recognition result of the environment, and the track information of the vehicle are used to determine the parking space and the driving path of the vehicle. The vehicle can be controlled to drive to the parking space according to the driving path, so as to accurately control the vehicle to complete parking. The technical problem of low parking efficiency of the vehicle is solved, and the technical effect of improving the parking efficiency of the vehicle is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of this application and help to explain the present application, but do not limit the present application in any way. In the drawings:

[0021] Figure 1 is a flowchart of a vehicle parking method according to an embodiment of the present application;

[0022] Figure 2 is a flowchart of another vehicle parking method according to an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of a vehicle parking system according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] According to an embodiment of the present application, a method embodiment of a vehicle parking method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0027] In this embodiment, a vehicle parking method is provided. Figure 1 is a flow chart of a vehicle parking method according to an embodiment of the present application, such as Figure 1 As shown, the process may include the following steps.

[0028] Step S101 : acquiring initial multimodal data of the environment in which the vehicle is located, wherein the initial multimodal data is used to represent different types of environmental data collected by different types of environmental sensing devices.

[0029] In the technical solution proposed in the above-mentioned step S101 of the embodiment of the present application, initial multimodal data of the vehicle's environment is obtained through different types of environmental sensing devices, wherein the environmental sensing devices can be surround-view cameras, ultrasonic probes, lidars, etc., and the initial multimodal data can be called sensing data or perception data.

[0030] Optionally, the surround-view camera is used to provide visual information about the vehicle's surroundings, enabling the vehicle's automated parking system to identify parking spaces, obstacles, and road conditions. This automated parking system consists of multiple high-resolution cameras, typically mounted on the front, back, left, and right sides of the vehicle, to capture a 360-degree image of the vehicle's surroundings.

[0031] Optionally, ultrasonic probes are used to detect close-range obstacles of the vehicle, i.e., in the case where the vehicle is extremely close to the surrounding obstacles during parking. The ultrasonic probes can be installed on the front and rear bumpers of the vehicle to emit ultrasonic pulses. When these pulses are reflected back by the obstacles, the probes can receive and measure the echo time, thereby calculating the distance between the vehicle and the obstacles.

[0032] Optionally, a laser radar provides high-precision depth information and three-dimensional (3D) point cloud data in the automatic parking system. The laser radar measures the distance between the vehicle and the obstacles by emitting laser pulses and receiving the reflected signals, and can capture high-density point cloud data of the surrounding environment.

[0033] For example, this embodiment can use environmental sensing devices such as surround-view cameras, ultrasonic probes, laser radars, etc. to collect perception data of the environment in which the vehicle is located, to facilitate subsequent accurate parking of the vehicle. It should be noted that this is only one preferred implementation of obtaining initial multi-modal data of the environment in which the vehicle is located, and the process and method of obtaining initial multi-modal data of the environment in which the vehicle is located are not specifically limited.

[0034] In step S102, the initial multi-modal data is aligned to obtain target multi-modal data, wherein the target multi-modal data is used to represent different types of environmental data aligned in time and / or space.

[0035] In the technical solution of step S102 of the above embodiment of the present application, after obtaining the initial multi-modal data of the environment in which the vehicle is located, the time-space synchronization engine in the vehicle can be used to align the initial multi-modal data to obtain target multi-modal data.

[0036] Optionally, the initial multi-modal data is aligned in time and space to obtain target multi-modal data, wherein the alignment can be used to solve the synchronization problem of data collected by different sensors in time and space, to ensure that the multi-modal data can be effectively integrated and utilized, and to improve the overall perception accuracy and robustness of the system. The target multi-modal data can be referred to as sensing data (perception data) after alignment.

[0037] Optionally, the alignment includes aligning the data in time, i.e., to ensure that the data obtained from different sensors can be compared and fused at the same time point; the alignment also includes aligning the initial multi-modal data in space, i.e., to ensure that the coordinate information obtained by different sensors can be matched with each other to establish a unified coordinate system.

[0038] For example, the perception data is aligned through the time-space synchronization engine to obtain the aligned perception data, which can then be collaboratively analyzed to achieve the purpose of accurate parking of the vehicle.

[0039] Step S103 : calling the target model to perform collaborative analysis on the target multimodal data to obtain vehicle information recognition results and environment information recognition results.

[0040] In the technical solution proposed in step S103 of the embodiment of the present application, a target model can be invoked to collaboratively analyze the target multimodal data obtained in the above step to obtain vehicle information recognition results and environment information recognition results. The target model can be a Neural Architecture Search (NAS) model and / or a Knowledge Distillation (KD) model.

[0041] Alternatively, a NAS model automatically explores the design space through an algorithm to find the architecture that performs best on a given task. A knowledge distillation model is used to transfer the knowledge of a large, complex model (called the teacher model) to a smaller, simpler model (called the student model), thereby reducing computational and storage requirements while maintaining performance.

[0042] Alternatively, the vehicle information recognition result may be referred to as the vehicle body positioning result, or simply referred to as vehicle body positioning. The environment information recognition result may be referred to as the obstacle recognition result, or simply referred to as obstacle recognition. Collaborative analysis may be referred to as intelligent analysis.

[0043] Optionally, the neural network architecture search model and the knowledge distillation model are called to perform collaborative analysis on the target multimodal data to obtain the information recognition results of the vehicle and the environment. For example, the perception data after alignment processing is intelligently analyzed through the neural network architecture search (NAS) model and the knowledge distillation model to complete the vehicle body positioning and obstacle recognition of the vehicle environment, and output the distance between the obstacle and the vehicle body in real time.

[0044] Step S104 , based on the vehicle information recognition result, the environment information recognition result, and the track information matching the current execution parameters of the vehicle, the vehicle's parking space and the vehicle's driving path to the parking space are determined from the environment.

[0045] In the technical solution proposed in the above-mentioned step S104 of the embodiment of the present application, after obtaining the vehicle information recognition results and the environment information recognition results, the vehicle's parking space and the vehicle's driving path to the parking space can be determined in the environment based on the vehicle information recognition results, the environment information recognition results, and the track information matching the vehicle's current execution parameters.

[0046] Optionally, the current execution parameters are used to characterize the performance of the actuators in the vehicle, and the track information is used to predict the vehicle's trajectory during motion. The current execution parameters, which can be referred to as current actuator parameters or simply actuator parameters, characterize the specific performance indicators and operational characteristics of controlling vehicle motion (including forward, reverse, steering, acceleration, deceleration, and braking) when the vehicle executes the instructions of the automated parking system. The driving path can be referred to as the real-time planned path or simply the real-time path. The spaces to be parked can be referred to as available spaces.

[0047] In this embodiment, the vehicle's parking space and the vehicle's travel path to the parking space are determined based on the vehicle's information recognition results, the environment's information recognition results, and the vehicle's track information. For example, after completing the vehicle's body positioning and obstacle identification in the vehicle's environment, inertial navigation dead reckoning (DR) can be performed in combination with actuator parameters to predict the vehicle's motion trajectory. The potential risks in the above process are evaluated through a dynamic collision risk prediction model to automatically screen available parking spaces and generate a real-time planned path, thereby achieving the purpose of accurately controlling the vehicle to complete parking.

[0048] Step S105: Control the vehicle to travel to a parking space according to the driving route and park.

[0049] In the technical solution proposed in the above-mentioned step S105 of the embodiment of the present application, the vehicle can be controlled to travel to the parking space to be parked according to the driving path. For example, after obtaining the real-time planned path of the vehicle, the vehicle can be controlled to travel to the parking space according to the real-time planned path, so as to achieve control of the vehicle to complete precise parking.

[0050] It can be understood that this is only a preferred implementation method for controlling the parking of the vehicle, and does not specifically limit the control method and process for controlling the parking of the vehicle. As long as the process and method of controlling the vehicle to drive to the parking space for parking according to the driving path are within the scope of protection of this application, they are not listed here.

[0051] It should be noted that in the parking method of the vehicle in this embodiment, the vehicle can also interact with the roadside device and the terminal device. Optionally, the vehicle can send an information subscription request to the roadside device, and the information subscription request can include specific types of information that the vehicle needs to receive, such as road conditions, traffic signal states, front obstacle warnings, etc. The roadside device can send roadside perception information to the vehicle in response to the information subscription request, for example, the roadside device can filter out roadside perception information that meets the needs of the vehicle according to its own perception capabilities and stored information, and send it to the vehicle at a certain frequency. In addition to communication with the roadside device, the vehicle can also receive driving scene switching instructions transmitted by the terminal device through the network, such as instructions for switching the vehicle to an energy-saving mode, a sports mode, an automatic driving mode, etc., so that the vehicle adapts to the new driving scene.

[0052] In steps S101-S105 of the embodiments of the present application, the obtained initial multi-modal data is aligned to obtain target multi-modal data representing different types of environment data aligned in time and / or space, and then the target model is called to cooperatively analyze the target multi-modal data, so as to obtain the information recognition result of the vehicle and the information recognition result of the environment. According to the above obtained information recognition result of the vehicle and the information recognition result of the environment, and the track information matched with the current execution parameter of the vehicle, the parking space to be parked and the driving path of the vehicle are determined, and the vehicle can be controlled to drive to the parking space to be parked according to the above driving path, so as to achieve the purpose of parking the vehicle. Since the target model is used to cooperatively analyze the multi-modal data of the vehicle to obtain the information recognition result of the vehicle and the information recognition result of the environment, and then the information recognition result of the vehicle and the information recognition result of the environment are used to determine the parking space to be parked and the driving path of the vehicle, so that the vehicle can be controlled to drive to the parking space to be parked according to the driving path, so as to accurately control the vehicle to complete parking, thereby achieving the technical effect of improving the parking efficiency of the vehicle, and solving the technical problem of low parking efficiency of the vehicle.

[0053] As an optional embodiment, the target model is called to cooperatively analyze the target multi-modal data to obtain the information recognition result of the vehicle and the information recognition result of the environment, including: fusing the features of the target multi-modal data to obtain a feature fusion result; calling the target model to cooperatively analyze the feature fusion result of the target multi-modal data to obtain the information recognition result of the vehicle and the information recognition result of the environment, wherein the feature fusion result is used to represent the fusion result of the features of the target multi-modal data.

[0054] In this embodiment, the attention mechanism is used to map the features extracted from different modal data in the target multimodal data to the same space, so as to achieve feature-level fusion of the features of the target multimodal data and obtain feature fusion results. Then, the neural network architecture search model and the knowledge distillation model are called to collaboratively analyze the feature fusion results to achieve the purpose of obtaining vehicle information recognition results and environment information recognition results.

[0055] For example, after collecting the target multimodal data through surround-view cameras, ultrasonic probes, lidar and other equipment, the target multimodal data can be fused at the feature level to obtain feature fusion results, and then the neural network architecture search model and knowledge distillation model can be used to intelligently analyze the feature fusion results to complete vehicle body positioning and obstacle recognition, thereby improving the vehicle's obstacle recognition rate under low-light conditions.

[0056] As an optional implementation method, the method also includes: preprocessing the target multimodal data on the edge device of the vehicle; and fusing the features of the following data in the preprocessed target multimodal data on the edge device to obtain a feature fusion result: target position data, target point cloud data, and target image data, wherein the target position data is used to represent the positions of different objects in the environment, the target point cloud data is used to represent the three-dimensional space of the environment, and the target image data is used to represent the two-dimensional images of different objects in the environment.

[0057] In this embodiment, after obtaining the target multimodal data, the target multimodal data can be preprocessed on the edge device of the vehicle, and the attention mechanism can be used to perform feature-level fusion on the features of the following data in the preprocessed target multimodal data to obtain a feature fusion result: target position data, target point cloud data, and target image data.

[0058] Optionally, the edge device includes a fusion preprocessing unit based on edge computing, which is used to preprocess the raw data from different sensors (such as cameras, radars, lidars, etc.), including data cleaning, denoising, format conversion and standardization, so as to efficiently and in real time process and fuse data from vehicle sensors to achieve more accurate and faster environmental perception.

[0059] Optionally, the target position data may be 3D coordinate information, which may be referred to as 3D coordinates. The target point cloud data may be depth point cloud information, which may be referred to as depth point cloud. The target image data may be image (RGB image) information, which may be referred to as RGB image.

[0060] For example, the target multimodal data is preprocessed by a fusion preprocessing unit based on edge computing, and then the 3D coordinate information, depth point cloud information and RGB image information in the preprocessed target multimodal data are feature fused to achieve the purpose of obtaining feature fusion results.

[0061] As an optional embodiment, the method also includes at least one of the following: using a temporal convolutional network model to extract features of the target position data with timestamps to obtain features of the target position data; projecting the voxel features of the target point cloud data onto the plane of the two-dimensional image to obtain features of the target point cloud data; using an attention mechanism to extract features of continuous image frames in the target image data, calling a graph convolutional neural network model to extract key point geometric features from the color information in the target image data, and calling a residual convolutional neural network model to perform information recognition and information classification on the target image data to obtain features of the target image data.

[0062] In this embodiment, a temporal convolutional network (TCN) model can be used to perform feature extraction on the target position data with timestamps to obtain the features of the target position data, and then the voxel features of the target point cloud data are projected onto the plane of the two-dimensional image through the projection matrix of the camera to obtain the features of the target point cloud data. Then, a spatio-temporal joint attention mechanism (ST-Transformer) is used to perform feature extraction on continuous image frames in the target image data, and a graph convolutional neural network (GCN) model is called to extract key point geometric features from the color information in the target image data. At the same time, a residual convolutional neural network model is called to perform information recognition and information classification on the target image data to obtain the features of the target image data.

[0063] Alternatively, the temporal convolutional network model can be simply referred to as a temporal convolutional network. It should be noted that the spatiotemporal joint attention mechanism is a type of attention mechanism. The graph convolutional neural network model can be simply referred to as a graph convolutional neural network. Color information can be represented using RGB information.

[0064] Optionally, a temporal convolutional network is used to process sequential data. By using causal convolution and dilated convolution, it can expand the receptive field without losing any sequence information, effectively capturing long-range dependencies in sequence data, such as sound, text, and video frames. The spatiotemporal joint attention mechanism is used to simultaneously focus on important features of the data in both spatial and temporal dimensions. That is, this attention mechanism can identify key objects in each frame (spatial attention) and track changes in objects over time (temporal attention) to provide a more comprehensive understanding of the information.

[0065] For example, a temporal convolutional network (TCN) is used to extract features of 3D coordinates with timestamps to obtain the features of the 3D coordinates. Then, the feature center of each voxel in the depth point cloud is projected onto the image plane through the projection matrix of the camera to obtain the features of the depth point cloud. Furthermore, a spatiotemporal joint attention mechanism is established, and the spatiotemporal joint attention mechanism is used to extract features in continuous image frames with continuous image frames as input. Then, a graph convolutional neural network is used to extract key point geometric features from the RGB information in the image, and the residual convolutional neural network model is combined to identify and classify the image information.

[0066] As an optional implementation method, the target model includes a target student model, and the target model is called to collaboratively analyze the feature fusion results of the target multimodal data to obtain vehicle information recognition results and environment information recognition results, including: calling a neural network architecture search model to determine a neural network model that matches the feature fusion results, wherein the performance index of the neural network model under the target task type and target constraint conditions is greater than the performance index threshold, the target task type is used to indicate the type of task in the environment that the neural network architecture search model is allowed to process, the target constraint condition is used to indicate the constraint of the neural network architecture search model on the hardware performance of the vehicle, and the performance index is used to indicate the quality of the performance of the neural network model; the neural network model is used as a teacher model, and the knowledge distillation model is called to compress or transfer the knowledge information of the teacher model to the initial student model to obtain a target student model; the target student model is used to collaboratively analyze the feature fusion results to obtain vehicle information recognition results and environment information recognition results.

[0067] In this embodiment, a neural network architecture search model can be called to determine a neural network model that matches the feature fusion result, and the neural network model at this time is used as a teacher model. The knowledge distillation model is called to compress or transfer the knowledge information of the teacher model to the initial student model to obtain the target student model in the target model. Furthermore, multiple parallel network branches in the target student model are used to realize collaborative analysis of the feature fusion results, so as to achieve the purpose of obtaining vehicle information recognition results and environment information recognition results.

[0068] Optionally, the neural network model can be a neural network architecture that optimizes performance under specific task and hardware constraints. The target task type can be referred to as a specific task, such as an image classification task or an object detection task. Image classification tasks can be referred to as image classification, and object detection tasks can be referred to as object detection. The target student model can be referred to as the student model.

[0069] For example, a neural network architecture search (NAS) model is first used to automatically design the neural network architecture with the best performance for specific tasks (such as image classification, object detection, etc.) and hardware constraints (existing actual computing power, memory, and latency). The NAS model then provides a neural network architecture design that performs well under the target task and constraints. The knowledge distillation (KD) model is then used to jointly optimize the parking perception and control model provided by the existing NAS model. This "compresses" or "transfers" the knowledge of a large, complex, but high-performance model (the teacher model) into a smaller, simpler model (the student model). This allows the student model to maintain high performance while having lower computational overhead and faster inference speed. Finally, the student model can be used to intelligently analyze the feature fusion results obtained above to achieve vehicle positioning and obstacle recognition in the vehicle's environment.

[0070] As an optional embodiment, the vehicle information recognition result includes the vehicle's positioning information, where the positioning information is used to indicate the vehicle's position in the environment. The environment information recognition result includes the recognition result of an obstacle corresponding to the vehicle in the environment, where the obstacle recognition result is used to indicate the obstacle's position in the environment. A target student model is used to collaboratively analyze the feature fusion result to obtain the vehicle information recognition result and the environment information recognition result, including: collaboratively analyzing the feature fusion result using the target student model to obtain the positioning information and the obstacle recognition result; determining a parking space for the vehicle and a driving path for the vehicle to the parking space from the environment based on the vehicle information recognition result, the environment information recognition result, and track information matched with current execution parameters of the vehicle, including: determining a target distance between a position corresponding to the positioning information and a position corresponding to the obstacle recognition result; determining a risk analysis result for the vehicle based on the target distance and the track information, where the risk analysis result is used to indicate the likelihood of a collision risk for the vehicle; and using the risk analysis result, identifying a parking space from a region of interest in the environment and determining a driving path, where the region of interest is generated using scene features of the parking scene, and different regions of interest are matched with different computing resources.

[0071] In this embodiment, after obtaining the target student model, the feature fusion result is analyzed cooperatively by using multiple parallel network branches in the target student model to obtain positioning information and an identification result of the obstacle.

[0072] Optionally, the information identification result of the vehicle includes positioning information of the vehicle, where the positioning information is used to represent a position of the vehicle in the environment, such as a body coordinate, etc. The information identification result of the environment includes an identification result of the obstacle corresponding to the vehicle in the environment, where the identification result of the obstacle is used to represent a position of the obstacle in the environment, such as an obstacle coordinate, etc.

[0073] For example, different environment data is collected by different environment sensing devices, and after the environment data collected from the different environment sensing devices is integrated, the integrated data is intelligently analyzed by the student model to complete the body positioning of the vehicle and the obstacle identification in the environment.

[0074] Optionally, after obtaining the positioning information, a target distance between a position corresponding to the positioning information and a position corresponding to the identification result of the obstacle is determined, and then according to the target distance and the trajectory information, a risk analysis result used to represent a possibility degree of a collision risk of the vehicle is determined, and finally the risk analysis result is used to identify a parking space and a driving path from a region of interest of the environment. The target distance can be referred to as a distance. The region of interest can be referred to as a dynamic region of interest (ROI).

[0075] For example, a distance between the body and the obstacle is output in real time, and then according to the distance between the body and the obstacle, trajectory information obtained by combining the actuator parameters, a motion trajectory of the vehicle is predicted, a risk analysis result used to represent a possibility degree of a collision risk of the vehicle is determined, a division strategy of the dynamic region of interest (ROI) is designed based on the risk analysis result for a typical parking scene of the vehicle, and adaptive resolution adjustment is used to optimize the computing resources to automatically screen a parking space and generate a real-time planning path, and finally the vehicle is precisely controlled to complete parking.

[0076] As an optional embodiment, the method includes: establishing a collision risk prediction model by using the trajectory information, motion states of a plurality of obstacles in the environment, and semantic information of the environment, where the semantic information is used to describe the environment; determining a risk analysis result of the vehicle based on the target distance and the trajectory information, including: inputting the target distance and the trajectory information into the collision risk prediction model; and analyzing the target distance and the trajectory information by using the collision risk prediction model to obtain the risk analysis result.

[0077] In this embodiment, a collision risk prediction model can be established using trajectory information, the motion states of multiple obstacles in the environment, and semantic information about the environment. The target distance and trajectory information are then input into the collision risk prediction model for risk prediction analysis, thereby obtaining risk analysis results. The collision risk prediction model can be referred to as a dynamic collision risk prediction model. The motion state can be referred to as a kinematic state. The semantic information can be referred to as simply "semantics."

[0078] For example, a collision risk prediction model is established based on the track information obtained by DR trajectory calculation, the kinematic state of multiple obstacles, and the semantics of the surrounding environment. The track information and the distance between the vehicle body and the obstacle are then used as inputs of the collision risk prediction model. The potential risk of the vehicle during parking is evaluated through the dynamic collision risk prediction model to obtain risk analysis results.

[0079] As an optional embodiment, controlling the vehicle to travel to a parking space to be parked according to the driving path includes: detecting the slope of the road on which the vehicle is traveling while controlling the vehicle to travel to the parking space to be parked according to the driving path; compensating the vehicle's current control signal using the slope to obtain a target control signal, wherein the current control signal includes the vehicle's driving parameters; and controlling the vehicle to travel to the parking space to be parked in response to the target control signal.

[0080] In this embodiment, when controlling the vehicle to move toward the parking space according to the driving path, the slope of the road on which the vehicle is traveling is detected. The slope is used to compensate the vehicle's current control signal to obtain a compensated target control signal. Based on this target control signal, the vehicle is controlled to move toward the parking space to achieve precise parking. The current control signal includes the vehicle's driving parameters, which may be steering angles.

[0081] For example, when a vehicle is traveling to a parking space according to a real-time planned path, the slope of the road on which the vehicle is traveling can be determined. At this time, a road slope feedforward compensation mechanism is introduced to compensate the vehicle's steering angle to ensure that the vehicle can be accurately parked in the parking space.

[0082] It is understandable that this is merely a preferred implementation for controlling a vehicle to travel to a parking space for parking, and does not specifically limit the process and method of controlling a vehicle to travel to a parking space for parking.

[0083] In an embodiment of the present application, initial multimodal data is aligned to obtain target multimodal data representing different types of environmental data aligned in time and / or space. A target model is then invoked to perform collaborative analysis on the target multimodal data to obtain vehicle and environmental information recognition results. Based on the vehicle and environmental information recognition results, as well as track information matching the vehicle's current execution parameters, the vehicle's parking space and driving path are determined. The vehicle is then controlled to travel to the parking space according to the driving path, thereby achieving the purpose of parking the vehicle. By utilizing a target model to collaboratively analyze the vehicle's multimodal data to obtain vehicle and environmental information recognition results, and based on the vehicle and environmental information recognition results, the vehicle's parking space and driving path are determined. The vehicle is then controlled to travel to the parking space according to the driving path, thereby accurately controlling the vehicle to complete parking. This improves parking efficiency and resolves the technical problem of low parking efficiency.

[0084] It should be noted that the multimodal data involved in this application (including but not limited to data used for alignment processing, data for collaborative analysis, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0085] The vehicle parking method will be further described below in conjunction with preferred embodiments.

[0086] With the rapid development of intelligent driving technology for vehicles, especially the Automated Parking Assist (APA) system as an important component of intelligent driving technology, its performance optimization is becoming a focus of industry competition. When implementing Level 2 (L2+) and above automatic parking functions, the APA system faces challenges from two main technical routes: multi-sensor fusion solutions and pure vision solutions. Although the multi-sensor fusion solution can provide higher environmental perception accuracy, its high hardware cost (approximately 3,000 yuan per vehicle) weakens the market competitiveness of the vehicle model, and there are obstacles in the complexity of data fusion, such as time synchronization, spatial calibration and semantic alignment issues. Especially in rainy and foggy weather, the false alarm rate of millimeter-wave radar increases (increased to 15%-20%), and the conflict with the visual perception results leads to decision delays. On the other hand, pure vision solutions are limited by the inherent defects of cameras, and there are bottlenecks in dynamic obstacle response, near-field blind spot coverage and extreme scene recognition. For example, in low-light or high dynamic range scenarios, the image signal-to-noise ratio drops significantly, resulting in a surge in the error rate of parking line detection (40%-60%); at the same time, the surround-view camera's perception blind spots and the feature extraction failure rate under extreme weather conditions also limit the system's applicability in complex environments.

[0087] When the vehicle is in a dynamic obstacle interaction scenario, the automatic parking system's predicted response delay to sudden moving targets (such as children and pets) generally exceeds 200ms, resulting in a false trigger rate of 12%-15% for the vehicle's emergency braking. In addition, the system has poor environmental adaptability and a low parking success rate in narrow parking spaces, resulting in low parking efficiency.

[0088] In order to solve the above problems, the present application proposes a vehicle parking method, which adopts a collaborative analysis method of multimodal data. The initial multimodal data obtained is aligned to obtain target multimodal data for representing different types of environmental data aligned in time and / or space. The target model is then called to perform collaborative analysis on the target multimodal data to obtain vehicle information recognition results and environment information recognition results. Based on the above-mentioned vehicle information recognition results, environment information recognition results, and track information matching the current execution parameters of the vehicle, the vehicle's parking space and driving path are determined. According to the above-mentioned driving path, the vehicle can be controlled to drive to the parking space, thereby achieving the purpose of parking the vehicle. Taking into account the use of target models to collaboratively analyze the vehicle's multimodal data to obtain vehicle information recognition results and environment information recognition results, the vehicle's parking space and driving path can be determined based on the vehicle information recognition results, the environment information recognition results, and the vehicle's track information, so that the vehicle can be controlled to travel to the parking space according to the driving path, so as to accurately control the vehicle to complete parking, thereby achieving the technical effect of improving the vehicle's parking efficiency and solving the technical problem of low vehicle parking efficiency.

[0089] Figure 2 is a flow chart of another vehicle parking method according to an embodiment of the present application, such as Figure 2 As shown, the process may include the following steps:

[0090] Step S201 , determining three sensing devices: a surround view camera, an ultrasonic probe, and a laser radar.

[0091] In this embodiment, three sensing devices, namely, a surround-view camera, an ultrasonic probe and a lidar, are first determined, and the three sensing devices are used to collect perception data of the vehicle's environment to facilitate subsequent accurate parking of the vehicle.

[0092] Step S202: Collect 3D coordinate information, depth point cloud information, and RGB image information.

[0093] In this embodiment, 3D coordinate information, depth point cloud information and RGB image information are collected by a surround-view camera, an ultrasonic probe and a lidar respectively. That is, different types of sensors can provide different environmental information, generate a depth point cloud, and provide an RGB image.

[0094] Step S203: Analyze the 3D coordinate information, depth point cloud information, and RGB image information using a neural network architecture search model and a knowledge distillation model.

[0095] In this embodiment, firstly, a neural network architecture that is optimal in performance under specific tasks (such as image classification, target detection, etc.) and hardware constraints (existing actual computing capacity, memory, delay) is designed automatically by adopting a neural network architecture search (NAS) model, and then a neural network architecture design that performs excellently under the target task and constraints is given by the NAS model. Then, a knowledge distillation (KD) model is used to jointly optimize the parking perception and control model provided by the existing NAS model, and the knowledge of a large and complex but superior performance model (teacher model) is "compressed" or "transferred" to a smaller and simpler model (student model), so that the student model has higher performance while having smaller computing cost and faster reasoning speed. Finally, the student model can be used to intelligently analyze the feature fusion result obtained above to realize vehicle body positioning and obstacle identification of the environment in which the vehicle is located.

[0096] Optionally, a spatio-temporal joint attention mechanism is established, and continuous frame images are taken as inputs. The graph convolutional neural network can extract key point geometric features from the RGB information in the images, and the residual convolutional neural network structure is combined to identify and classify the information of the images. In the data preprocessing stage, the obtained 3D coordinate information, depth information and time stamp are integrated to improve the diversity and integrity of the data set.

[0097] Step S204, determine the identification results of the vehicle body coordinates and the obstacles.

[0098] In this embodiment, after the obtained 3D coordinate information, depth point cloud information and RGB image information are integrated, intelligent analysis is performed by the neural network architecture search model and the knowledge distillation model to complete the vehicle body positioning and obstacle identification.

[0099] Step S205, output the distance between the obstacles and the vehicle body.

[0100] In this embodiment, after the integrated data is intelligently analyzed by the neural network architecture search model and the knowledge distillation model, the distance between the obstacles and the vehicle body can be output in real time.

[0101] Step S206, determine the track information based on the actuator parameters.

[0102] In this embodiment, the DR track is calculated in combination with the actuator parameters to determine the track information.

[0103] Step S207, call the dynamic collision risk prediction model to analyze the track information and the distance between the obstacles and the vehicle body.

[0104] In this embodiment, the track information and the distance between the obstacles and the vehicle body are analyzed by the dynamic collision risk prediction model to evaluate the potential risks, and a risk analysis result is obtained.

[0105] In step S208, the available parking space and the real-time path are determined.

[0106] In this embodiment, based on the risk analysis result, the available parking space can be automatically screened and the real-time planning path is generated.

[0107] In step S209, the parking is completed.

[0108] In this embodiment, the vehicle can be controlled to travel to the available parking space according to the real-time planning path, so that the vehicle can be precisely controlled to complete the parking, and the safety and efficiency of the parking can be ensured.

[0109] In this embodiment, a plurality of network models are used to extract the obtained information, GCN is used to uniformly sample densely at different positions of the picture, convolution is used to predict different feature maps to extract the test results, and through the convolution operation, a homogeneous transformation matrix can be obtained to establish the relationship between the two-dimensional mask and the three-dimensional points. Each voxel feature center is projected onto the image plane through the projection matrix of the camera; a time convolution network (TCN) is used to extract features of the 3D coordinates with time stamps, and by stacking multiple TCN layers, the network can learn spatial features of different scales. A collision risk prediction model is established, DR track calculation is performed based on the performance of the real vehicle actuator, and track information is obtained. A collision risk prediction model is established based on the track information, the kinematic state of the multiple obstacles and the surrounding environment semantics. In the parking lot environment without a positioning system, the trajectory distortion of the slope scene is avoided by real-time compensation of the road slope.

[0110] In an embodiment of the present invention, initial multimodal data is aligned to obtain target multimodal data representing different types of environmental data aligned in time and / or space. A target model is then invoked to perform collaborative analysis on the target multimodal data to obtain vehicle information recognition results and environmental information recognition results. Based on the vehicle information recognition results and environmental information recognition results, as well as track information matching the vehicle's current execution parameters, the vehicle's parking space and driving path are determined. According to the driving path, the vehicle can be controlled to travel to the parking space, thereby achieving the purpose of parking the vehicle. Considering the use of a target model to collaboratively analyze the vehicle's multimodal data to obtain vehicle information recognition results and environmental information recognition results, and then based on the vehicle information recognition results, the vehicle's parking space and driving path are determined, allowing the vehicle to be controlled to travel to the parking space according to the driving path, thereby accurately controlling the vehicle to complete parking, thereby achieving the technical effect of improving vehicle parking efficiency and solving the technical problem of low vehicle parking efficiency.

[0111] In an embodiment of the present application, an intelligent parking system for high-level autonomous driving is proposed. The system includes: a multimodal fusion perception subsystem, a hierarchical progressive decision-making architecture subsystem, a high-precision execution control subsystem, a functional safety assurance subsystem, a system integration and verification subsystem, and an evolution route subsystem. The above subsystems are further explained below.

[0112] The multimodal fusion perception subsystem utilizes a heterogeneous sensor collaborative perception module, equipped with a multispectral visual perception unit, a millimeter-wave radar array, and a solid-state lidar network. The visual system comprises a surround-view array of six 2-megapixel fisheye cameras, using an optical distortion correction algorithm to eliminate wide-angle distortion and adaptive white balancing technology to ensure all-weather imaging quality. The radar system integrates 12 fifth-generation ultrasonic sensors and four 77 GHz millimeter-wave radars, employing a spatial diversity deployment strategy to eliminate detection blind spots. Incorporating a solid-state lidar point cloud compensation mechanism, feature-level fusion improves obstacle recognition in low-light conditions.

[0113] Optionally, after acquiring sensory data, it can be aligned using a time-space synchronization engine. Based on the edge computing fusion preprocessing unit, an improved probabilistic association algorithm is used to address multi-target tracking challenges. The Transformer attention mechanism is innovatively introduced into multi-source data for feature extraction, constructing a hierarchical feature representation space. A dynamic region of interest (ROI) partitioning strategy is designed for typical parking scenarios, optimizing computing resources through adaptive resolution adjustment.

[0114] The hierarchical and progressive decision-making subsystem utilizes a three-tiered modular architecture: scene understanding, behavioral decision-making, and motion planning. The scene understanding module extracts structured information such as parking spaces and traffic signs through a semantic segmentation network and combines historical trajectory analysis to build an environmental cognitive map. The behavioral decision-making module utilizes a hybrid decision-making model that combines a finite state machine with reinforcement learning. In structured scenarios, a rules engine ensures safety, while in unstructured scenarios, a machine learning algorithm (Q-learning) is used to enhance adaptability.

[0115] Optionally, the motion planning module proposes a spatiotemporal joint optimization approach. The global planner uses an improved random sampling subgraph search algorithm to generate an initial path while satisfying the vehicle's kinematic constraints. Local planning incorporates an elastic band deformation mechanism, using potential field functions for dynamic obstacle avoidance, and combines this with a model predictive control framework to achieve trajectory smoothing.

[0116] The high-precision executive control subsystem utilizes a dual-loop proportional-integral-derivative (PID) controller architecture for longitudinal control. The inner loop controls the drive motor torque, while the outer loop adjusts vehicle speed and position deviation, incorporating a feedforward compensation mechanism for road slope. Lateral control is based on a feedforward-feedback composite steering model. The feedforward component calculates the theoretical steering angle based on path curvature, while the feedback component achieves trajectory tracking through PID control of heading deviation.

[0117] The functional safety assurance subsystem utilizes a heterogeneous dual-computing platform architecture at the hardware level. The main control unit utilizes a multi-core processor, while the safety monitoring unit is equipped with an independent microprocessor. Dual redundancy verification is achieved through heartbeat detection and output comparison. To address expected functional safety, a multi-dimensional test and verification matrix was established. The environmental dimension covers 12 weather conditions, including rain, fog, and strong sunlight; the scenario dimension encompasses over 200 special operating conditions, including irregular parking spaces and dynamic obstacles; and the fault dimension simulates 54 abnormal conditions, including sensor failure and communication interruption. Boundary test cases are generated using a formal verification toolchain, and Monte Carlo methods are employed for probabilistic risk assessment.

[0118] The system integrates and verifies the subsystem, develops a modular simulation test platform, and builds a digital twin system that includes a physics engine, sensor models, and vehicle dynamics models. The virtual test field imports a 3D model library of typical parking lots worldwide, supporting real-world environment reconstruction such as sensor noise injection and communication delay simulation. Actual vehicle testing uses a scenario-based automated testing framework, enabling remote delivery of use cases and real-time data transmission via a cloud-based test management platform. The verification process follows the integrated testing flow on the right side of the development and testing model (V model): unit testing covers more than 97% of the condition coverage criteria (MC / DC criteria) in software testing; integration testing focuses on verifying the consistency of interfaces between modules; system testing assesses functional integrity and performance indicators; and acceptance testing completes more than 5,000 parking cycle verifications in a real parking lot.

[0119] For the evolutionary route subsystem, the next-generation system will focus on breakthroughs in the following technical directions: a cognitive decision-making architecture based on neural-symbolic fusion, combining knowledge graphs with deep learning to achieve human-like reasoning; a vehicle-park collaborative perception network, acquiring high-precision semantic maps of parking lots through vehicle communication technology; a quantum-inspired optimization algorithm, improving real-time planning capabilities in complex scenarios; a self-supervised learning framework, reducing data labeling costs and enhancing model generalization; and an explainable intelligent module, building a tool chain for visual analysis of the decision-making process.

[0120] In the embodiments of the present application, efforts are made to address the technical bottlenecks of traditional automatic parking systems in terms of environmental adaptability, decision-making intelligence, and control accuracy through a multimodal fusion perception subsystem, a hierarchical progressive decision-making architecture subsystem, a high-precision execution control subsystem, a functional safety assurance subsystem, a system integration and verification subsystem, and an evolutionary route subsystem.

[0121] In an embodiment of the present application, initial multimodal data is aligned to obtain target multimodal data representing different types of environmental data aligned in time and / or space. A target model is then invoked to perform collaborative analysis on the target multimodal data to obtain vehicle and environmental information recognition results. Based on the vehicle and environmental information recognition results, as well as track information matching the vehicle's current execution parameters, the vehicle's parking space and driving path are determined. The vehicle is then controlled to travel to the parking space according to the driving path, thereby achieving the purpose of parking the vehicle. By utilizing a target model to collaboratively analyze the vehicle's multimodal data to obtain vehicle and environmental information recognition results, and based on the vehicle and environmental information recognition results, the vehicle's parking space and driving path are determined. The vehicle is then controlled to travel to the parking space according to the driving path, thereby accurately controlling the vehicle to complete parking. This improves parking efficiency and resolves the technical problem of low parking efficiency.

[0122] According to an embodiment of the present application, a system embodiment of a vehicle parking system is provided. It should be noted that the system can be used to execute the above-mentioned vehicle parking method.

[0123] In this embodiment, a vehicle parking system is provided. Figure 3 is a schematic diagram of a parking system for a vehicle according to an embodiment of the present application, such as Figure 3 As shown, the parking system of the vehicle includes: multiple sensing devices 301 , a processor 302 and a controller 303 .

[0124] Multiple sensing devices 301 are used to obtain target position data, target point cloud data and target image data of the vehicle's environment, wherein the target position data is used to represent the positions of different objects in the vehicle's environment, the target point cloud data is used to represent the three-dimensional space of the environment, and the target image data is used to represent the two-dimensional images of different objects in the environment.

[0125] Processor 302 is configured to align target position data, target point cloud data, and target image data in time and / or space to obtain target multimodal data; invoke a target model to collaboratively analyze the target multimodal data to obtain vehicle information recognition results and environment information recognition results; and determine a parking space for the vehicle and a travel path for the vehicle to the parking space from the environment based on the vehicle information recognition results, the environment information recognition results, and track information that matches the vehicle's current execution parameters, wherein the current execution parameters are used to characterize the execution performance of the actuator in the vehicle, and the track information is used to predict the trajectory of the vehicle during motion.

[0126] The controller 303 is used to control the vehicle to travel to the parking space for parking according to the driving path.

[0127] Optionally, the processor 302 is also used to fuse the features of the target multimodal data to obtain a feature fusion result; call the target model to perform collaborative analysis on the feature fusion result of the target multimodal data to obtain the vehicle information recognition result and the environment information recognition result, wherein the feature fusion result is used to represent the fusion result of the features of the target multimodal data.

[0128] Optionally, the processor 302 is further used to preprocess the target multimodal data on the edge device of the vehicle; and on the edge device, fuse the features of the following data in the preprocessed target multimodal data to obtain a feature fusion result: target position data, target point cloud data, and target image data, wherein the target position data is used to represent the positions of different objects in the environment, the target point cloud data is used to represent the three-dimensional space of the environment, and the target image data is used to represent the two-dimensional images of different objects in the environment.

[0129] Optionally, the processor 302 is further used to use a temporal convolutional network model to perform feature extraction on the target position data with a timestamp to obtain features of the target position data; project the voxel features of the target point cloud data onto the plane of the two-dimensional image to obtain features of the target point cloud data; use an attention mechanism to perform feature extraction on continuous image frames in the target image data, call a graph convolutional neural network model to extract key point geometric features from the color information in the target image data, and call a residual convolutional neural network model to perform information recognition and information classification on the target image data to obtain features of the target image data.

[0130] Optionally, the processor 302 is further used to call a neural network architecture search model to determine a neural network model that matches the feature fusion result, wherein the performance index of the neural network model under the target task type and target constraint conditions is greater than a performance index threshold, the target task type is used to indicate the type of task in an environment that allows the neural network architecture search model to process, the target constraint conditions are used to indicate the constraints of the neural network architecture search model on the hardware performance of the vehicle, and the performance index is used to indicate the quality of the performance of the neural network model; the neural network model is used as a teacher model, and the knowledge distillation model is called to compress or transfer the knowledge information of the teacher model to the initial student model to obtain a target student model; the target student model is used to collaboratively analyze the feature fusion results to obtain vehicle information recognition results and environment information recognition results.

[0131] Optionally, the processor 302 is further configured to: the vehicle information recognition result includes positioning information of the vehicle, the positioning information is used to indicate the position of the vehicle in the environment; the environment information recognition result includes recognition results of obstacles corresponding to the vehicle in the environment, the obstacle recognition results are used to indicate the positions of the obstacles in the environment; and using the target student model to collaboratively analyze the feature fusion results to obtain vehicle information recognition results and environment information recognition results, including: using the target student model to collaboratively analyze the feature fusion results to obtain positioning information and obstacle recognition results; determining a parking space for the vehicle and a driving path for the vehicle to travel to the parking space from the environment based on the vehicle information recognition result, the environment information recognition result, and track information matched with current execution parameters of the vehicle, including: determining a target distance between a position corresponding to the positioning information and a position corresponding to the obstacle recognition result; determining a risk analysis result for the vehicle based on the target distance and the track information, wherein the risk analysis result is used to indicate the likelihood of a collision risk of the vehicle; and using the risk analysis result to identify a parking space from a region of interest in the environment and determine a driving path, wherein the region of interest is generated using scene features of the parking scene, and different regions of interest are matched with different computing resources.

[0132] Optionally, the processor 302 is also used to establish a collision risk prediction model using track information, the motion status of multiple obstacles in the environment and semantic information of the environment, wherein the semantic information is used to describe the environment; and determine the risk analysis results of the vehicle based on the target distance and track information, including: inputting the target distance and track information into the collision risk prediction model; and using the collision risk prediction model to analyze the target distance and track information to obtain the risk analysis results.

[0133] Optionally, the controller 303 is further used to detect the slope of the road on which the vehicle is traveling while controlling the vehicle to travel to the parking space according to the driving path; use the slope to compensate the current control signal of the vehicle to obtain a target control signal, wherein the current control signal includes the driving parameters of the vehicle; and in response to the target control signal, control the vehicle to travel to the parking space for parking.

[0134] In an embodiment of the present application, multiple sensing devices are used to respectively acquire target position data, target point cloud data, and target image data of a vehicle's environment, wherein the target position data is used to represent the positions of different objects in the vehicle's environment, the target point cloud data is used to represent the three-dimensional space of the environment, and the target image data is used to represent the two-dimensional images of different objects in the environment. A processor aligns the target position data, target point cloud data, and target image data in time and / or space to obtain target multimodal data. A target model is used to perform collaborative analysis on the target multimodal data to obtain vehicle information recognition results and environment information recognition results. Based on the vehicle information recognition results, the environment information recognition results, and track information matching the vehicle's current execution parameters, a parking space for the vehicle and a driving path for the vehicle to reach the parking space are determined from the environment, wherein the current execution parameters are used to characterize the execution performance of actuators in the vehicle, and the track information is used to predict the trajectory of the vehicle during movement. A controller controls the vehicle to drive to the parking space for parking according to the driving path, thereby achieving the technical effect of improving the parking efficiency of the vehicle and solving the technical problem of low parking efficiency of the vehicle.

[0135] An embodiment of the present application further provides a vehicle, comprising: a memory storing an executable program; and a processor for running the program, wherein the method of each embodiment of the present application is executed when the program is running.

[0136] An embodiment of the present application further provides a computer-readable storage medium, which includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present application.

[0137] An embodiment of the present application further provides a computer program product, including a computer program, which implements the methods in various embodiments of the present application when executed by a processor.

[0138] An embodiment of the present application further provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present application is implemented.

[0139] The embodiments of the present application further provide a computer program, which, when executed by a processor, implements the methods in the above-mentioned embodiments of the present application.

[0140] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0142] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0143] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0144] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk or optical disk, etc., various media that can store program code.

[0145] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for parking a vehicle, characterized in that: include: Acquiring initial multimodal data of the environment in which the vehicle is located, wherein the initial multimodal data is used to represent different types of environmental data collected by different types of environmental sensing devices; Performing alignment processing on the initial multimodal data to obtain target multimodal data, wherein the target multimodal data is used to represent the different types of environmental data aligned in time and / or space; Invoking a target model to perform collaborative analysis on the target multimodal data to obtain information recognition results of the vehicle and information recognition results of the environment; Determining a parking space for the vehicle and a travel path for the vehicle to the parking space from the environment based on the vehicle identification results, the environment identification results, and track information matching current execution parameters of the vehicle, wherein the current execution parameters are used to characterize the execution performance of actuators in the vehicle, and the track information is used to predict the trajectory of the vehicle during movement; According to the driving path, the vehicle is controlled to drive to the parking space for parking.

2. The method according to claim 1, characterized in that The calling of the target model to collaboratively analyze the target multimodal data to obtain the vehicle information recognition result and the environment information recognition result includes: fusing the features of the target multimodal data to obtain a feature fusion result; The target model is called to collaboratively analyze the feature fusion results of the target multimodal data to obtain information recognition results of the vehicle and information recognition results of the environment, wherein the feature fusion results are used to represent the fusion results of the features of the target multimodal data.

3. The method according to claim 2, characterized in that The method further comprises: Preprocessing the target multimodal data on an edge device of the vehicle; On the edge device, the features of the following data in the preprocessed target multimodal data are fused to obtain the feature fusion result: target position data, target point cloud data and target image data, wherein the target position data is used to represent the positions of different objects in the environment, the target point cloud data is used to represent the three-dimensional space of the environment, and the target image data is used to represent the two-dimensional images of different objects in the environment.

4. The method according to claim 3, characterized in that The method further comprises at least one of the following: Using a temporal convolutional network model, feature extraction is performed on the target location data with the timestamp to obtain features of the target location data; Projecting the voxel features of the target point cloud data onto the plane of the two-dimensional image to obtain the features of the target point cloud data; The attention mechanism is used to extract features from continuous image frames in the target image data, a graph convolutional neural network model is called to extract key point geometric features from the color information in the target image data, and a residual convolutional neural network model is called to perform information recognition and information classification on the target image data to obtain the features of the target image data.

5. The method according to claim 2, characterized in that The target model includes a target student model, and calling the target model to collaboratively analyze the feature fusion results of the target multimodal data to obtain the information recognition results of the vehicle and the information recognition results of the environment include: Invoking a neural network architecture search model to determine a neural network model that matches the feature fusion result, wherein a performance index of the neural network model under a target task type and a target constraint condition is greater than a performance index threshold, wherein the target task type is used to indicate the type of task in the environment that the neural network architecture search model is allowed to process, the target constraint condition is used to indicate the constraint imposed by the neural network architecture search model on the hardware performance of the vehicle, and the performance index is used to indicate the quality of the performance of the neural network model; The neural network model is used as a teacher model, and a knowledge distillation model is called to compress or transfer the knowledge information of the teacher model to the initial student model to obtain the target student model; The target student model is used to collaboratively analyze the feature fusion results to obtain information recognition results of the vehicle and the environment.

6. The method according to claim 5, characterized in that The vehicle information recognition result includes the vehicle positioning information, which is used to indicate the location of the vehicle in the environment. The environment information recognition result includes the recognition result of the obstacle corresponding to the vehicle in the environment, which is used to indicate the location of the obstacle in the environment. The target student model is used to collaboratively analyze the feature fusion result to obtain the vehicle information recognition result and the environment information recognition result, including: Using the target student model to collaboratively analyze the feature fusion result to obtain the positioning information and the obstacle recognition result; The determining of a parking space for the vehicle and a driving path for the vehicle to travel to the parking space from the environment based on the vehicle information recognition result, the environment information recognition result, and track information matching the current execution parameters of the vehicle includes: Determine a target distance between a position corresponding to the positioning information and a position corresponding to the obstacle identification result; Determining a risk analysis result of the vehicle based on the target distance and the track information, wherein the risk analysis result is used to indicate a degree of possibility of a collision risk of the vehicle; The risk analysis result is used to identify the parking space to be parked from an area of ​​interest in the environment, and to determine the driving path, wherein the area of ​​interest is generated using scene features of the parking scene, and different areas of interest are matched with different computing resources.

7. The method according to claim 6, characterized in that The method comprises: Establishing a collision risk prediction model using the track information, the motion states of multiple obstacles in the environment, and semantic information of the environment, wherein the semantic information is used to describe the environment; The determining of the risk analysis result of the vehicle based on the target distance and the track information includes: Inputting the target distance and the track information into the collision risk prediction model; The collision risk prediction model is used to analyze the target distance and the track information to obtain the risk analysis result.

8. The method according to any one of claims 1 to 7, characterized in that The controlling the vehicle to drive to the parking space for parking according to the driving path includes: In the process of controlling the vehicle to travel toward the parking space according to the travel path, detecting the slope of the road on which the vehicle is traveling; Compensating a current control signal of the vehicle using the slope to obtain a target control signal, wherein the current control signal includes a driving parameter of the vehicle; In response to the target control signal, the vehicle is controlled to travel to the parking space for parking.

9. A vehicle parking system, characterized in that: include: A plurality of sensing devices, respectively used to obtain target position data, target point cloud data, and target image data of the environment in which the vehicle is located, wherein the target position data is used to represent the positions of different objects in the environment in which the vehicle is located, the target point cloud data is used to represent the three-dimensional space of the environment, and the target image data is used to represent the two-dimensional images of different objects in the environment; a processor configured to align the target position data, the target point cloud data, and the target image data in time and / or space to obtain target multimodal data; invoke a target model to perform collaborative analysis on the target multimodal data to obtain information identification results for the vehicle and information identification results for the environment; and determine, from the environment, a parking space for the vehicle and a travel path for the vehicle to the parking space, based on the vehicle information identification results, the environment information identification results, and track information matching current execution parameters of the vehicle, wherein the current execution parameters are used to characterize the execution performance of an actuator in the vehicle, and the track information is used to predict a trajectory of the vehicle during movement. The controller is used to control the vehicle to drive to the parking space for parking according to the driving path.

10. A vehicle, characterized in that: include: a memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 8 when running.

Images