Automatic parking activation method and device, electronic equipment and intelligent driving system
Through the combination of deep learning model and world model, identifying and predicting the environment around the vehicle, the problem of poor automatic parking activation effect in complex and changing environments is solved, and more accurate and stable activation of automatic parking function is achieved.
Patent Information
- Application Number
- CN202411457939.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-18
- Publication Date
- 2025-01-03
AI Technical Summary
The existing automatic parking activation method is not effective in complex and changing environments, and the automatic parking function cannot be activated accurately, especially due to the unstable GPS signal and is only suitable for underground parking lots.
By obtaining the environmental images around the vehicle, vehicle motion status information and positioning information, the deep learning model and pre-trained world model are used to identify and predict the environment, and the continuity of the parking lot environment is determined, thereby activating the automatic parking function. The specific steps include generating virtual environment sample data, building a time series classification model, fusing multimodal data for prediction, and ensuring that the activation conditions are met through threshold judgment.
It improves the accuracy and robustness of automatic parking activation in complex and changing environments, ensures the stable and continuous activation of automatic parking functions in changing environments, reduces data acquisition costs, and enhances the comprehensive coverage of scenarios.
Smart Images
Figure CN119239576A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle intelligent driving technology, and in particular to an automatic parking activation method, device, electronic equipment and intelligent driving system. Background Art
[0002] As a driving assistance technology in intelligent driving, automatic parking technology can realize automatic parking of vehicles and even memorize parking operations through the vehicle's autonomous perception and control system. Automatic parking technology has been widely used in automobiles. In actual application scenarios, how to determine whether the automatic parking function can be activated to further perform mapping or cruise parking functions is one of the important issues.
[0003] Most existing solutions are based on GPS (Global Positioning System) signals to determine whether the vehicle is in an underground parking lot to determine whether the automatic parking function can be activated. However, this method has unstable signals and is only applicable to the single environment of an underground parking lot. It cannot achieve accurate activation effects in complex and changing environments.
[0004] With regard to the problem in related technologies that automatic parking activation is not effective in complex and changing environments, no effective solution has been proposed so far. Summary of the invention
[0005] In this embodiment, an automatic parking activation method, device, electronic device and intelligent driving system are provided to solve the problem of poor automatic parking activation effect in complex and changeable environments in related technologies.
[0006] In a first aspect, an automatic parking activation method is provided in this embodiment, including:
[0007] Acquire the environment image around the vehicle, vehicle motion status information and positioning information;
[0008] Determine a current environment recognition result according to the environment image;
[0009] Based on the environment recognition result, the vehicle motion state information, the positioning information and the pre-trained world model, predict the environment changes within a preset time period in the future to obtain an environment prediction result;
[0010] When it is confirmed that both the environment recognition result and the environment prediction result are parking lot environments and the environment prediction result is continuous, the automatic parking function is activated accordingly.
[0011] In some of the embodiments, determining the current environment recognition result according to the environment image includes:
[0012] Generate a variety of virtual environment sample data based on real environment images;
[0013] Building a time series classification model based on a Transformer network, and training the time series classification model using the virtual environment sample data to obtain an environment recognition model;
[0014] The environment image is input into the environment recognition model, and the environment recognition result is output.
[0015] In some embodiments, the prediction of environmental changes within a preset time period in the future based on the environmental recognition result, the vehicle motion state information, the positioning information and the pre-trained world model to obtain an environmental prediction result includes:
[0016] Performing multimodal fusion on the environment recognition result, the vehicle motion state information and the positioning information to obtain a fusion feature;
[0017] Predicting a complete state of the current environment based on the world model and the fusion features;
[0018] Predicting changes in the state of the environment within the preset time period based on the complete state of the current environment and the vehicle motion state information;
[0019] According to the environmental state change, the environmental prediction result for each time step within the preset time period is determined.
[0020] In some of the embodiments, the continuity determination of the environmental prediction result includes:
[0021] Performing continuity judgment on the environmental prediction result according to the predicted environmental type and the corresponding probability in the environmental prediction result;
[0022] If within the preset time period, the probability that the environment prediction result is a parking lot environment is greater than or equal to a first threshold, then it is determined that the environment prediction result is a parking lot environment and is continuous.
[0023] In some of the embodiments, after activating the automatic parking function, the method further includes:
[0024] When the environment recognition result changes from a parking lot environment to a non-parking lot environment, it is determined whether to maintain activation of the automatic parking function based on the positioning information and the probability that the environment prediction result is a parking lot environment.
[0025] In some embodiments, judging whether to maintain activation of the automatic parking function according to the positioning information and the probability that the environment prediction result is a parking lot environment includes:
[0026] Recording the vehicle positioning information when the environment recognition result changes;
[0027] Based on the positioning information, if within a preset distance, the probability that the environment prediction result is restored to a parking lot environment is greater than or equal to a second threshold, it is determined to maintain activation of the automatic parking function.
[0028] In a second aspect, an automatic parking activation device is provided in this embodiment, including:
[0029] An acquisition module is used to acquire the environment image around the vehicle, the vehicle motion state information and the positioning information;
[0030] An environment recognition module, used to determine a current environment recognition result according to the environment image;
[0031] An environment prediction module, used to predict the environment changes within a preset time period in the future based on the environment recognition result, the vehicle motion state information, the positioning information and the pre-trained world model to obtain an environment prediction result;
[0032] The activation module is used to activate the automatic parking function when it is confirmed that the environment recognition result and the environment prediction result are both parking lot environments and the environment prediction result is continuous.
[0033] In a third aspect, an intelligent driving system is provided in this embodiment, including: a vehicle-mounted camera, a sensor, and a server; the vehicle-mounted camera and the sensor are respectively connected to the server;
[0034] The vehicle-mounted camera and the sensor are used to obtain environmental images around the vehicle, vehicle motion status information and positioning information;
[0035] The server is used to execute the automatic parking activation method described in the first aspect above.
[0036] In a fourth aspect, an electronic device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the automatic parking activation method described in the first aspect is implemented.
[0037] In a fifth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the automatic parking activation method described in the first aspect is implemented.
[0038] Compared with the related art, the automatic parking activation method, device, electronic device and intelligent driving system provided in this embodiment obtain the environmental image around the vehicle, the vehicle motion state information and the positioning information; determine the current environmental recognition result according to the environmental image; predict the environmental changes in the future preset time period based on the environmental recognition result, the vehicle motion state information, the positioning information and the pre-trained world model to obtain the environmental prediction result; when it is confirmed that the environmental recognition result and the environmental prediction result are both parking lot environments, and the environmental prediction result is continuous, the automatic parking function is activated accordingly. Through this embodiment, the world model can be used to predict future environmental changes, and the accuracy and robustness of automatic parking activation in complex and changeable environments can be improved by combining the environmental prediction results and the environmental recognition results, thereby solving the problem of poor automatic parking activation effect in complex and changeable environments.
[0039] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0041] Figure 1 is a hardware structure block diagram of a terminal of an automatic parking activation method in an embodiment;
[0042] Figure 2 is a flow chart of an automatic parking activation method according to an embodiment;
[0043] Figure 3 is a flow chart of an automatic parking activation method according to another embodiment;
[0044] Figure 4 is a block diagram of an automatic parking activation device in one embodiment;
[0045] Figure 5 It is a schematic diagram of the structure of an intelligent driving system in an embodiment.
[0046] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 10, acquisition module; 20, environment recognition module; 30, environment prediction module; 40, activation module. DETAILED DESCRIPTION
[0047] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.
[0048] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.
[0049] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a vehicle terminal, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of the automatic parking activation method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.
[0050] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the automatic parking activation method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0051] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.
[0052] As a driving assistance technology in intelligent driving, automatic parking technology can realize automatic parking of the vehicle and even memorize parking operations through the vehicle's autonomous perception and control system. Automatic parking is a function that detects the vehicle's surrounding environment information and effective parking space through sensors distributed around the vehicle, plans the parking path, controls the vehicle's steering and acceleration and deceleration, and enables the vehicle to complete parking operations semi-automatically or automatically. Memory parking is to use parking (surround view camera) or some driving sensors (front view camera) to draw a partial map of memory parking before using smart parking in and smart parking out, and learn and establish the route map of smart parking in and smart parking out in advance. After the map is built, the user needs to use the established map to drive the entire smart parking in, low-speed cruising and smart parking out control.
[0053] Automatic parking technology has been widely used in automobiles. In actual application scenarios, how to determine whether the automatic parking function can be activated to further perform mapping or cruise parking functions is one of the important issues.
[0054] Most existing solutions are based on GPS (Global Positioning System) signals to determine whether the vehicle is in an underground parking lot to determine whether the automatic parking function can be activated. However, this method has unstable signals and is only applicable to the single environment of an underground parking lot. It cannot achieve accurate activation effects in complex and changing environments.
[0055] In view of the above problems, an automatic parking activation method is provided in this embodiment. Figure 2 is a flow chart of the automatic parking activation method in this embodiment. Figure 2 As shown, the method comprises the following steps:
[0056] Step S210, obtaining an environmental image around the vehicle, vehicle motion state information, and positioning information.
[0057] Specifically, through the perception system of the vehicle, for example, based on the vehicle camera, radar, GPS positioning system or other sensor equipment, the environmental image around the vehicle, the vehicle motion state information and positioning information can be collected. Among them, the environmental image around the vehicle includes but is not limited to the front, rear, left, right, left front and right front images of the vehicle, which are collected by the vehicle camera set at the corresponding position of the vehicle body. The vehicle motion state information includes but is not limited to the vehicle speed, acceleration and steering angle, etc., and the positioning information includes but is not limited to the position and posture of the vehicle in the world coordinate system, which can be collected by sensors such as GPS positioning system and inertial posture sensor.
[0058] Step S220, determining the current environment recognition result according to the environment image.
[0059] Specifically, an environment recognition model can be obtained by model training based on deep learning, wherein real images of parking environments (including underground parking lots and ground parking lots) and non-parking lots are used as training data, and an environment model is obtained based on deep learning training. In this embodiment, for each frame of environment image of each video clip, for each frame, multiple vehicle camera images are obtained. If any one of the multiple vehicle camera images can determine that the vehicle is in an underground parking lot or a ground parking lot, the frame is marked as an underground parking lot or a ground parking lot; otherwise, the frame is marked as a non-parking lot.
[0060] Considering that the real parking environment is complex and changeable, the recognition effect of the environment recognition model may not be ideal for scenes not covered by the training data. Furthermore, a large amount of diverse virtual environment sample data can be generated based on real environment images for use as training data.
[0061] By training a complete environment recognition model, the scene of the environment image is recognized to obtain the environment recognition result. Exemplarily, the environment recognition result can be divided into ground parking lot, underground parking lot and non-parking lot, and the ground parking lot and underground parking lot are both parking lot environments.
[0062] Step S230, based on the environment recognition result, the vehicle motion state information, the positioning information and the pre-trained world model, predict the environment changes within a preset time period in the future to obtain an environment prediction result.
[0063] Specifically, a world model that can simulate the real environment can be pre-built and trained based on large-scale real-world data. The world model learns and understands environmental changes and predicts the changing trend of the future world state under a given sequence.
[0064] By integrating the environment recognition results, vehicle motion state information and positioning information through the world model, the environmental changes within a preset time period in the future are predicted. In this embodiment, the predicted environmental changes may include the possible motion trajectory of the vehicle, changes in the vehicle's surrounding environment, etc., from which the environmental prediction results within the preset time period are determined. The environmental prediction results include the predicted environment type and the corresponding probability.
[0065] Step S240, when it is confirmed that the environment recognition result and the environment prediction result are both parking lot environments and the environment prediction result is continuous, the automatic parking function is activated accordingly.
[0066] Specifically, considering the uncertainty of the prediction of the world model, post-processing logic is performed based on the environment recognition results and the environment prediction results to more accurately determine whether to activate the automatic parking function. In the post-processing logic, based on the environment recognition results, it is determined whether the current environment is in a parking lot environment (including underground parking lots and ground parking lots), and the probability of the environment prediction results and the preset first threshold are further used to make a continuity judgment on the environment prediction results within the preset time period. The environment prediction result is a parking lot environment and the probability of remaining in the parking lot environment within the preset time period meets the first threshold. It is continuous when it is confirmed that the environment recognition result and the environment prediction result are both parking lot environments, and the environment prediction result is continuous, it is determined that the activation conditions are met, and the automatic parking function is activated accordingly.
[0067] In the above decision-making process, the uncertainty estimation of the world model output is taken into account. When the uncertainty is high, the conservatism of the post-processing logic judgment is increased, and a higher probability threshold (first threshold) or a longer observation time (preset time period) can be set.
[0068] It should be noted that the automatic parking activation method in this embodiment is also applicable to the activation of intelligent driving scenarios such as memory parking, and the above method can be constructed as a prediction result based on an end-to-end large model to directly output whether the automatic parking function can be activated. For safety reasons, the entire parking process requires the user to monitor the operating status of the system and, if necessary, brake or steer the vehicle to take over the vehicle.
[0069] Through the above steps, the world model can be used to integrate multi-source data of environmental recognition results, vehicle motion status information and positioning information to predict future environmental changes, and then post-processing logic judgment can be performed based on the environmental prediction results and environmental recognition results. This is more comprehensive and accurate than the activation judgment that relies solely on a single information source in related technologies. The prediction also improves the accuracy and robustness of automatic activation in complex and changing environments, solving the problem of poor activation effect of automatic parking in complex and changing environments.
[0070] In some of the embodiments, determining the current environment recognition result according to the environment image in the above step S220 can be achieved by the following steps:
[0071] Based on real environment images, a variety of virtual environment sample data are generated; a time series classification model is constructed based on the Transformer network, and the time series classification model is trained using virtual environment sample data to obtain an environment recognition model; the environment image is input into the environment recognition model, and the environment recognition result is output.
[0072] Specifically, in order to improve the comprehensiveness of the training data's coverage of complex and changeable parking lot environments, a large amount of diverse virtual environment sample data is generated based on real environment images for use as training data. Exemplarily, a pre-trained world model can be used to generate various complex environments. The virtual environment sample data includes underground parking lots, ground parking lots, and non-parking lot environments. These sample data include changes in different lighting conditions, weather conditions, parking lot layouts, etc., and can be generated by adjusting the environment image parameters (for example, adding noise) and adjusting the world model parameters. Optionally, 1 million virtual environment sample data can be generated, each sample data containing a 5-second video clip with a frame rate of 30hz. Since the scene type is known when the world model generates the data, the corresponding scene category (underground parking lot, ground parking lot, or non-parking lot) can be automatically annotated for each frame of the virtual environment sample data.
[0073] The specific construction and pre-training steps of the above world model are as follows:
[0074] 1. Collect and organize existing open source data sets;
[0075] 2. Develop an automatic labeling process to process unlabeled vehicle camera data;
[0076] 3. Train the initial model using a small amount of high-quality, manually annotated data;
[0077] 4. Apply semi-supervised and self-supervised learning techniques to expand the effective training data set;
[0078] 5. Use data mining methods to select difficult samples for manual labeling;
[0079] 6. Continue to train and optimize the world model based on the above steps.
[0080] A temporal classification model is constructed based on the Transformer network. The model structure can use SwinTransformer (a moving window transformer) as the backbone to extract the features of each input frame image, use the temporal attention mechanism to fuse the features of the current frame with the previous 3 frames, and finally output the classification results through the fully connected layer.
[0081] The generated virtual environment sample data is used to train the time series classification model to obtain the environment recognition model. The cross entropy loss function can be used to train using the Adam optimizer. During the training process, real data is used regularly for verification to ensure the generalization ability of the model in real scenarios. In addition, based on the verification results of real data, the environment recognition model and the world model are continuously optimized to improve the authenticity of the virtual data and the recognition accuracy of the model.
[0082] By training a complete environment recognition model, the scene of the environment image is recognized to obtain the environment recognition result.
[0083] By generating a large amount of diverse virtual environment sample data in this embodiment, the data collection cost is greatly reduced, and the comprehensiveness of scene coverage is improved to improve the accuracy of current environment recognition. In addition, by continuously optimizing the environment recognition model, it can continuously adapt to new environments and scenes, and has strong adaptive capabilities.
[0084] In some of the embodiments, the above step S230 predicts environmental changes within a preset time period in the future based on the environmental recognition result, the vehicle motion state information, the positioning information and the pre-trained world model to obtain the environmental prediction result, which can be achieved by the following steps:
[0085] The environment recognition results, vehicle motion state information and positioning information are multimodally fused to obtain fusion features; based on the world model and the fusion features, the complete state of the current environment is predicted; based on the complete state of the current environment and the vehicle motion state information, the change of the environment state within a preset time period is predicted; based on the change of the environment state, the environment prediction result for each time step within the preset time period is determined.
[0086] Specifically, the world model used in this embodiment can be jointly trained with the world model in the above embodiments. The world model can respectively realize functions such as future prediction and generation of virtual environment sample data. In some cases, a world model can also be trained separately in this embodiment for predicting future changes in environmental states. There is no specific limitation on this.
[0087] The environmental recognition results, vehicle motion state information and positioning information are fused to obtain fusion features as the input of the world model. Based on the fusion features of the current multiple frames of continuous input, the world model predicts the complete state of the current environment. The complete state of the current environment includes but is not limited to the specific location of the vehicle in the parking lot, the distribution of surrounding objects, etc. Further, based on the complete state of the current environment and the vehicle motion state information, the world model uses time series to predict the changes in the environmental state in a preset time period (such as 5 seconds) in the future. The changes in the environmental state include the possible movement trajectory of the vehicle, the changes in the environment around the vehicle, etc. Exemplarily, the predicted changes in the environmental state can be a video of the environment. According to the changes in the environmental state, the environmental prediction results for each time step in the preset time period can be determined. The environmental prediction results include the predicted environmental type and the corresponding probability.
[0088] In addition, historical environmental prediction results can be superimposed on the input of the world model to avoid the impact of instantaneous fluctuations on the prediction. This can be achieved by maintaining a sliding time window (for example, the last 30 seconds) of historical environmental prediction results.
[0089] By introducing the world model in this embodiment to predict future environmental change states, and outputting the environmental type and corresponding probability of the environmental prediction results, the accuracy and robustness of judgment in complex and changing environments are improved. The uncertainty estimation of the world model can also be used for subsequent decision-making processes.
[0090] In some of the embodiments, the continuity determination of the environmental prediction result in the above step S240 can be implemented by the following steps:
[0091] The continuity of the environmental prediction result is judged based on the predicted environment type and the corresponding probability in the environmental prediction result; if within the preset time period, the probability that the environmental prediction result is a parking lot environment is greater than or equal to a first threshold, the environmental prediction result is judged to be a parking lot environment and is continuous.
[0092] Specifically, considering the uncertainty of the prediction of the world model, post-processing logic is performed based on the environment recognition result and the environment prediction result to more accurately determine whether to activate the automatic parking function. In the post-processing logic, the change of the environment state predicted by the world model within the preset time period is analyzed, and the continuity of the environment prediction result is judged according to the predicted environment type and the corresponding probability in the environment prediction result. If the environment prediction result is a parking lot environment, and the probability of maintaining the parking lot environment within the preset time period is greater than or equal to the first threshold, it is a parking lot environment and has continuity.
[0093] Exemplarily, the first threshold can be 80%, the preset time period is 5 seconds, and, in the above decision-making process, the uncertainty estimate of the world model output is taken into account. When the uncertainty is high, the conservatism of the post-processing logic judgment is increased. A higher probability threshold (first threshold) or a longer observation time (preset time period) can be set. There is no specific restriction on the first threshold and the preset time period.
[0094] When it is confirmed that the environment prediction result is a parking lot environment and is continuous, and the current environment recognition result is also a parking lot environment, it is determined that the activation conditions are met and the automatic parking function is activated accordingly.
[0095] By using the environmental prediction results predicted by the world model in this embodiment to perform continuity judgment on the post-processing logic, the stability of the decision can be improved, and complex scenarios can be better handled, thereby improving the activation effect of automatic parking in complex and changing environments.
[0096] In some of the embodiments, after activating the automatic parking function, the method further includes:
[0097] When the environment recognition result changes from a parking lot environment to a non-parking lot environment, a determination is made as to whether to maintain activation of the automatic parking function based on the positioning information and the probability that the environment prediction result is a parking lot environment.
[0098] Among them, the positioning information of the vehicle when the environment recognition result jumps is recorded; based on the positioning information, if within a preset distance, the probability that the environment prediction result is restored to a parking lot environment is greater than or equal to a second threshold, it is determined to maintain the activation of the automatic parking function.
[0099] Specifically, in multi-storey parking environments such as underground parking lots and stereo garages, when vehicles need to park in parking spaces on different floors, cross-floor parking may occur. Due to the complex environmental characteristics of the scene during the cross-floor process, it may be misidentified as a non-parking lot environment, and the current environmental recognition result may jump, thereby affecting the activation of the automatic parking function. Therefore, after the activation conditions are met and the automatic parking function is activated, the post-processing logic also provides processing and judgment of state jumps. When the environmental recognition result jumps from a parking lot environment to a non-parking lot environment, the predictive ability of the world model is used to determine whether to maintain the state of satisfaction of the activation conditions based on the vehicle's positioning information at the time of the jump and the predicted environment type and corresponding probability in the environmental prediction result, that is, to determine whether to maintain the activation of the automatic parking function.
[0100] The positioning information of the vehicle at the moment of the jump is recorded, and then the vehicle continues to cruise. The real-time positioning information of the vehicle is obtained during the cruising process, wherein the position of the center of the rear axle of the vehicle in the world coordinate system can be used as the positioning information. If within the preset distance of the positioning information, the probability of restoring to the parking environment in the environmental prediction result of the world model is greater than or equal to the second threshold, the activation condition is maintained and the automatic parking function is activated. Otherwise, the activation condition is not satisfied and the activation of the automatic parking function is terminated. Exemplarily, the preset distance can be 20 meters, the second threshold can be 70%, and, in the above decision-making process, considering the uncertainty estimate of the world model output, when the uncertainty is high, the conservatism of the post-processing logic judgment is increased, and a higher probability threshold (second threshold) or a longer observation distance (preset distance) can be set, and there is no specific restriction on the second threshold and the preset distance.
[0101] In this embodiment, the predictive ability of the world model is utilized to achieve continuity in activating the automatic parking function in complex and changeable environments such as cross-layers, avoiding discontinuity and instability in activation due to jumps in environmental recognition results, thereby improving the effect of activation in complex and changeable environments.
[0102] The present embodiment is described and illustrated below through preferred embodiments.
[0103] Figure 3 is a flow chart of the automatic parking activation method in this embodiment. Figure 3 As shown, the method comprises the following steps:
[0104] Step S310, obtaining an environmental image around the vehicle, vehicle motion state information, and positioning information.
[0105] Step S320, based on the pre-trained world model, generate a variety of virtual environment sample data; build a time series classification model based on the Transformer network, use the virtual environment sample data to train the time series classification model, and obtain an environment recognition model.
[0106] Step S330: outputting the environment recognition result based on the environment recognition model and the environment image.
[0107] Step S340, multi-modally fuse the environment recognition result, the vehicle motion state information and the positioning information to obtain a fusion feature.
[0108] Step S350, input the fusion features into the world model to predict the complete state of the current environment; then combine with the vehicle motion state information to predict the change of the environment state within a preset time period.
[0109] Step S360, determining the environmental prediction result for each time step within a preset time period according to the change in the environmental state; the environmental prediction result includes the predicted environmental type and the corresponding probability.
[0110] Step S370, when it is confirmed that the environment recognition result is a parking lot environment, and the probability that the environment prediction result is a parking lot environment within a preset time period is greater than or equal to a first threshold, it is determined that the activation condition is met, and the automatic parking function is activated accordingly.
[0111] Step S380, when the environment recognition result jumps from a parking lot environment to a non-parking lot environment, the positioning information of the vehicle when the jump occurs is recorded; if within the preset distance of the positioning information, the probability that the environment prediction result returns to a parking lot environment is greater than or equal to a second threshold, it is determined that the activation condition is maintained and the activation of the automatic parking function is maintained.
[0112] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0113] By introducing the world model in this embodiment, the future environment is predicted, which improves the accuracy and robustness of activating the automatic parking function in complex and changeable environments. Post-processing logic judgment is performed based on the environmental prediction results and the environmental recognition results, which further realizes continuous activation in complex environments such as cross-layers. It can also generate a large amount of diverse virtual scene data to train the environmental recognition model, greatly reducing the data collection cost, while improving the comprehensiveness of scene coverage, improving the accuracy and continuity of activation in complex and changeable environments, and comprehensively improving the activation effect.
[0114] In this embodiment, an automatic parking activation device is also provided, which is used to implement the above embodiments and preferred implementations, and will not be repeated for what has been described. The terms "module", "unit", "subunit", etc. used below can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0115] Figure 4 is a structural block diagram of the automatic parking activation device in this embodiment, such as Figure 4 As shown, the device comprises:
[0116] The acquisition module 10 is used to acquire the environment image around the vehicle, the vehicle motion state information and the positioning information;
[0117] The environment recognition module 20 is used to determine the current environment recognition result according to the environment image;
[0118] The environment prediction module 30 is used to predict the environment changes within a preset time period in the future based on the environment recognition result, the vehicle motion state information, the positioning information and the pre-trained world model to obtain the environment prediction result;
[0119] The activation module 40 is used to activate the automatic parking function when it is confirmed that the environment recognition result and the environment prediction result are both parking lot environments and the environment prediction result is continuous.
[0120] Through the device provided in this embodiment, it is possible to use a world model to integrate multi-source data of environmental recognition results, vehicle motion state information, and positioning information to predict future environmental changes, and then perform post-processing logical judgment based on the environmental prediction results and environmental recognition results. This is more comprehensive and accurate than the activation judgment that relies solely on a single information source in related technologies, and the accuracy and robustness of automatic activation in complex and changing environments are improved through prediction, solving the problem of poor activation effect of automatic parking in complex and changing environments.
[0121] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.
[0122] In this embodiment, an intelligent driving system is provided. Figure 5 Schematic diagram of the structure of the intelligent driving system in this embodiment. Figure 5As shown, the intelligent driving system includes: a vehicle-mounted camera, a sensor and a server; the vehicle-mounted camera and the sensor are connected to the server respectively; wherein,
[0123] The vehicle-mounted camera and sensor are used to obtain environmental images around the vehicle, vehicle motion status information and positioning information; the server is used to execute the automatic parking activation method provided by any embodiment.
[0124] Specifically, the environmental images around the vehicle include but are not limited to the images of the front, rear, left, right, left front and right front of the vehicle, which are collected by the on-board cameras set at the corresponding positions of the vehicle body. The vehicle motion state information includes but is not limited to the vehicle speed, acceleration and steering angle, etc. The positioning information includes but is not limited to the position and attitude of the vehicle in the world coordinate system, which can be collected by sensors such as GPS positioning system and inertial attitude sensor.
[0125] The system can be used to activate automatic parking or memory parking functions in intelligent driving scenarios, and has a good activation effect.
[0126] This embodiment also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0127] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0128] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.
[0129] In addition, in combination with the automatic parking activation method provided in the above embodiments, a storage medium may be provided in this embodiment to implement the automatic parking activation method. The storage medium stores a computer program, and when the computer program is executed by a processor, any one of the automatic parking activation methods in the above embodiments is implemented.
[0130] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.
[0131] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.
[0132] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.
[0133] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.
[0134] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.
Claims
1. An automatic parking activation method, characterized in that: include: Acquire the environment image around the vehicle, vehicle motion status information and positioning information; Determining a current environment recognition result according to the environment image; Based on the environment recognition result, the vehicle motion state information, the positioning information and the pre-trained world model, predict the environment changes within a preset time period in the future to obtain an environment prediction result; When it is confirmed that both the environment recognition result and the environment prediction result are parking lot environments and the environment prediction result is continuous, the automatic parking function is activated accordingly.
2. The automatic parking activation method according to claim 1, characterized in that: The determining the current environment recognition result according to the environment image includes: Generate a variety of virtual environment sample data based on real environment images; Building a time series classification model based on a Transformer network, and training the time series classification model using the virtual environment sample data to obtain an environment recognition model; The environment image is input into the environment recognition model, and the environment recognition result is output.
3. The automatic parking activation method according to any one of claims 1 or 2, characterized in that: The step of predicting environmental changes within a preset time period in the future based on the environmental recognition result, the vehicle motion state information, the positioning information, and the pre-trained world model to obtain an environmental prediction result includes: Performing multimodal fusion on the environment recognition result, the vehicle motion state information and the positioning information to obtain a fusion feature; Predicting a complete state of the current environment based on the world model and the fusion features; Predicting changes in the state of the environment within the preset time period based on the complete state of the current environment and the vehicle motion state information; According to the environmental state change, the environmental prediction result for each time step within the preset time period is determined.
4. The automatic parking activation method according to claim 1, characterized in that: The continuity judgment of the environmental prediction result includes: Performing continuity judgment on the environmental prediction result according to the predicted environmental type and the corresponding probability in the environmental prediction result; If within the preset time period, the probability that the environment prediction result is a parking lot environment is greater than or equal to a first threshold, then it is determined that the environment prediction result is a parking lot environment and is continuous.
5. The automatic parking activation method according to claim 1, characterized in that: After the automatic parking function is activated, the method further includes: When the environment recognition result changes from a parking lot environment to a non-parking lot environment, it is determined whether to maintain activation of the automatic parking function based on the positioning information and the probability that the environment prediction result is a parking lot environment.
6. The automatic parking activation method according to claim 5, characterized in that: The determining whether to maintain activation of the automatic parking function according to the positioning information and the probability that the environment prediction result is a parking lot environment includes: Recording the vehicle positioning information when the environment recognition result changes; Based on the positioning information, if within a preset distance, the probability that the environment prediction result is restored to a parking lot environment is greater than or equal to a second threshold, it is determined to maintain activation of the automatic parking function.
7. An automatic parking activation device, characterized in that: include: An acquisition module is used to acquire the environment image around the vehicle, the vehicle motion state information and the positioning information; An environment recognition module, used to determine a current environment recognition result according to the environment image; An environment prediction module, used to predict the environment changes within a preset time period in the future based on the environment recognition result, the vehicle motion state information, the positioning information and the pre-trained world model to obtain an environment prediction result; The activation module is used to activate the automatic parking function when it is confirmed that the environment recognition result and the environment prediction result are both parking lot environments and the environment prediction result is continuous.
8. An intelligent driving system, characterized in that: include: A vehicle-mounted camera, a sensor and a server; the vehicle-mounted camera and the sensor are connected to the server respectively; The vehicle-mounted camera and the sensor are used to obtain environmental images around the vehicle, vehicle motion status information and positioning information; The server is used to execute the automatic parking activation method according to any one of claims 1 to 6.
9. An electronic device, comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the automatic parking activation method according to any one of claims 1 to 6.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the automatic parking activation method according to any one of claims 1 to 6 are implemented.