Parking assisting method and device, vehicle, storage medium and degree product

Abstract

The invention discloses a parking assisting method and device, a vehicle, a storage medium and a degree product, and aims to improve the parking efficiency. The method comprises the steps that a display device is controlled to display a parking area, and the parking area is generated based on the surrounding environment of a vehicle; controlling a display device to display a first parking position based on the first instruction; and marking the position of a first obstacle, wherein the first obstacle comprises an obstacle interacting with the first parking position. By the adoption of the method, the user can accurately know the position of the obstacle affecting parking, then the user can be assisted to avoid the obstacle and select the parking position where the obstacle does not exist as soon as possible, the number of times of selecting the parking position by the user is reduced, the parking efficiency is effectively improved, and then the parking experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of parking technology, and in particular to a parking assistance method, device, vehicle, storage medium, and degree product. Background Art

[0002] Parking refers to a technology that drives a vehicle into a parking space under low-speed driving scenarios, including fully automatic parking, semi-automatic parking and manual parking.

[0003] Compared to semi-automatic and manual parking, fully automated parking requires only user input via the vehicle's display, mobile phone, or remote control. The vehicle automatically parks in a pre-planned or user-designated parking space, eliminating the need for manual operation or even the need for the user to be in the vehicle. Fully automated parking offers a more intelligent and convenient parking experience and is becoming a standard feature in vehicles.

[0004] However, current mainstream fully automated parking solutions suffer from low parking efficiency. Especially in scenarios where users are required to specify a parking space, they often have to go through multiple selections before finally finding a suitable spot. This significantly reduces parking efficiency and hinders the user's parking experience. Therefore, improving parking efficiency is a pressing technical challenge in the automated parking field. Summary of the Invention

[0005] The present application provides a parking assistance method, device, vehicle, storage medium, and program product to improve parking efficiency.

[0006] In a first aspect, the present application provides a parking assistance method, which includes: controlling a display device to display a parking area, where the parking area is generated based on the surrounding environment of the vehicle; controlling the display device to display a first parking position based on a first instruction; and marking the position of a first obstacle, where the first obstacle includes an obstacle that interacts with the first parking position.

[0007] By using the above method, by marking obstacles that interact with the first parking position, the user can accurately know the location of the obstacle that affects parking, and then assist the user to avoid the obstacle and select a parking position without the obstacle as soon as possible, reducing the number of times the user selects a parking position, effectively improving parking efficiency, and thus improving the user's parking experience.

[0008] In a possible design, the obstacles that interact with the first parking position include obstacles within the first parking position and / or obstacles outside the first parking position that will interact with the first parking position.

[0009] The above design can mark static obstacles and / or dynamic obstacles that affect the current parking situation. By marking static obstacles, users can be assisted in quickly avoiding obstacles in the first parking position. By issuing early warnings for dynamic obstacles, users can be assisted in avoiding obstacles that may appear in the first parking position in the future. This prevents users from being interrupted by dynamic obstacles in the middle of parking and having to choose a parking position again, thereby maintaining efficient parking efficiency.

[0010] In one possible design, the first obstacle is displayed outside the first parking position, and the first obstacle is entirely or partially located within the first parking position. Whether the first obstacle is entirely or partially located within the first parking position is determined by the perception results of the vehicle through the on-board sensors.

[0011] The above design can also mark obstacles that are not displayed in the first parking position due to image distortion or other reasons. This allows the user to accurately know the actual location of the obstacle even if the user cannot see the obstacle in the parking area or sees the wrong obstacle location, thereby helping the user to select a parking space as quickly as possible.

[0012] In a possible design, the first parking position may be a parking position automatically generated by the vehicle, and / or the first parking position may be a parking position determined by a user.

[0013] By adopting the above design, the parking assistance method can be compatible with different parking space selection functions in vehicles, or vehicles with different parking space selection functions, thereby improving the versatility of the parking assistance method.

[0014] In one possible design, marking the position of the first obstacle includes: performing environmental perception based on multiple sensors, and marking the first parking position as occupied by the obstacle.

[0015] The above design can intuitively present the occupancy status of the obstacle itself to the user, allowing the user to accurately know the location of the obstacle.

[0016] In a further possible design, the position of the first obstacle is marked, including at least one of the following three marking contents:

[0017] Annotation content 1: Mark the position of the first obstacle closest to the center point of the first parking space, so that the user can quickly locate the point where the first obstacle intrudes the deepest into the first parking space and understand the extent to which the first obstacle occupies the first parking space;

[0018] The second annotation content is to mark the outline of the first obstacle so that the user can accurately know the area occupied by the first obstacle in the first parking position, so that the user can quickly select an available parking position outside the area;

[0019] The third annotation content is to mark the center position of the first obstacle so that the user can quickly locate the center point of the first obstacle, and to facilitate the user to select a parking position in a radial manner based on the center point.

[0020] In a possible design, marking the position of the first obstacle includes: the first parking position includes multiple sub-areas, and obstacle occupancy is marked for each sub-area.

[0021] The above design allows the sub-area to which the obstacle belongs to be marked. Even if the actual obstacle position deviates from the detected obstacle position due to some reasons (such as detection error), the actual obstacle location can be included in the sub-area to which it belongs and marked synchronously, increasing the probability that the user can adjust to a possible parking position based on the marking.

[0022] In one possible design, marking the position of the first obstacle includes: when the first parking position is partially outside the parking area and the first obstacle occupies the part outside the parking area, marking the edge of the parking area covered by the first parking position as an obstacle occupancy mark.

[0023] With the above design, the user can know the positional relationship between the obstacles outside the parking area and the first parking position, thereby helping the user to avoid the location of the obstacles outside the parking area and select a parking position as quickly as possible.

[0024] In one possible design, marking the position of the first obstacle includes: identifying the first obstacle, and if the first obstacle has an impact on parking, marking the first obstacle as occupied; if the first obstacle has no impact on parking, marking the first obstacle as unoccupied, or not marking the first obstacle.

[0025] The above design can distinguish and mark obstacles that affect parking and those that do not, making it easier for users to understand the type of obstacle so that they can respond more specifically.

[0026] In a further possible design, the first obstacle is considered to have an impact on parking when it meets at least one of the following conditions: the height of the first obstacle exceeds a set height, the maximum distance that the first obstacle intrudes into the first parking position is greater than a set distance, and the first obstacle is not in a preset whitelist.

[0027] The above design covers all kinds of obstacles that affect parking in actual scenarios by identifying obstacles that are high enough to scratch the vehicle chassis, obstacles that intrude deeply into the parking space, and obstacles outside the set whitelist as obstacles that affect parking.

[0028] In a further possible design, if the first obstacle has no effect on parking, a first prompt message may be issued, which is used to prompt the user to choose whether to ignore the first obstacle. Afterwards, the vehicle is controlled to park in the first parking position based on a second instruction, and the second instruction is an instruction to ignore the first obstacle.

[0029] Optionally, the first prompt information may be a voice prompt, a display prompt, a flashing light prompt, a vibration prompt, etc. For example, the display device may be controlled to display text information, prompting the user to click a corresponding button on the display device, or to respond to a corresponding instruction by voice.

[0030] With the above design, in scenarios where the impact of obstacles on parking is negligible, additional prompt information can be issued to facilitate users to quickly make parking choices and assist users in completing parking as quickly as possible.

[0031] A further possible design is to use at least two different marking methods: occupied marking, unoccupied marking, or marking different degrees of occupied marking. For example, obstacles with no impact on parking can be marked green, obstacles with a significant impact on parking can be marked red, and obstacles with uncertain impact on parking can be marked yellow.

[0032] The above design distinguishes and labels obstacles of different categories, and can more intuitively present the category of the current obstacle to the user, making it easier for the user to respond quickly to the current obstacle in a targeted manner.

[0033] In a possible design, the first obstacle may include a dynamic obstacle and a static obstacle, and the dynamic obstacle and the static obstacle are marked in different ways.

[0034] The above design distinguishes and labels dynamic obstacles from static obstacles, allowing users to more intuitively understand the types of obstacles currently affecting parking.

[0035] In one possible design, the annotation includes at least one of a human machine interface (HMI) annotation, a voice prompt annotation, a direction prompt sound annotation, a light annotation, and a projection annotation. Optionally, the HMI annotation includes at least one of the following: a graphic annotation, a text annotation, a color annotation, and a pattern annotation.

[0036] The above design supports multiple marking methods, which can realize diversified options of assisted parking and meet the personalized needs of users.

[0037] In a further possible design, the direction warning sound marking includes: issuing a warning sound at a corresponding direction relative to the driver in the vehicle according to the position of the first obstacle relative to the first parking position.

[0038] With the above design, a prompt sound can be emitted at the position of the obstacle relative to the vehicle, drawing the user's attention to the pronunciation location, allowing the user to more intuitively grasp the position where the obstacle will affect parking, thereby improving the user's parking experience.

[0039] In one possible design, the method further includes: providing first recommendation information based on the position of the first obstacle and the surrounding environment of the vehicle, where the first recommendation information is used to indicate a recommended parking space adjustment direction and / or a recommended parking space adjustment position.

[0040] Optionally, the first recommendation information may be a voice recommendation, a display recommendation, a headlight flashing recommendation, etc. For example, the display device may be controlled to display the drag position or drag direction, so that the user can adjust the parking position.

[0041] The above design can assist users in quickly adjusting their parking positions and speeding up parking by sending additional recommendation information to users.

[0042] In one possible design, the method further includes: marking an edge position of the first obstacle; and / or marking a critical safety position, where the critical safety position is a position at a preset safety distance from the edge position of the first obstacle.

[0043] The above design allows users to more intuitively see the edge position or critical safety position of obstacles, thereby facilitating users to quickly determine the safe area and speed up the selection of a suitable parking location.

[0044] In one possible design, the method further includes: projecting a first parking position in the surroundings of the vehicle, and marking a first obstacle in the projected first parking position.

[0045] The above design can provide an early warning of obstacles in the first parking position. If the obstacle is a dynamic obstacle, a signal can be given to the dynamic obstacle to make it leave the first parking position as soon as possible, assisting the user to complete parking.

[0046] In a second aspect, the present application provides a parking assistance device, which can be a control device inside the vehicle or a control device outside the vehicle. The control device inside the vehicle may include, but is not limited to: a vehicle control unit (VCU) or a vehicle domain controller (VDC) for realizing whole vehicle control, an intelligent driving domain control unit or a mobile data center (MDC) for realizing intelligent driving or assisted driving, an electronic control unit (ECU) or a motor control unit (MCU) for realizing vehicle component control, or an intelligent brake system (IBS) for realizing vehicle driving mode control. The control device outside the vehicle may be, for example, a cloud server, a terminal device, a road side unit (RSU) or other vehicles. The terminal device can be understood as a device of a user riding in the controlled vehicle, such as, but not limited to, a mobile phone, a tablet computer, a laptop computer, a smart watch, a Bluetooth headset and other wearable devices. These control devices can be connected to relevant equipment in the controlled vehicle through the network, such as various sensors and display devices, to achieve parking assistance operations for the controlled vehicle.

[0047] The parking assistance device includes: a control module for controlling a display device to display a parking area, where the parking area is generated based on the surrounding environment of the vehicle, and controlling the display device to display a first parking position based on a first instruction; and a marking module for marking the position of a first obstacle, where the first obstacle includes an obstacle that interacts with the first parking position.

[0048] In a possible design, the obstacles that interact with the first parking position include obstacles within the first parking position and / or obstacles outside the first parking position that will interact with the first parking position.

[0049] In one possible design, the first obstacle is displayed outside the first parking position, and the first obstacle is entirely or partially located within the first parking position. Whether the first obstacle is entirely or partially located within the first parking position is determined by the perception results of the vehicle through the on-board sensors.

[0050] In a possible design, the first parking position is a parking position automatically generated by the vehicle, and / or the first parking position is a parking position determined by a user.

[0051] In one possible design, the labeling module is specifically used to: perform environmental perception based on multiple sensors and label the first parking position as occupied by an obstacle.

[0052] In a further possible design, the marking module is specifically used to: mark the position of the first obstacle closest to the center point of the first parking position; or, mark the outline of the first obstacle; or, mark the center position of the first obstacle.

[0053] In a possible design, the marking module is specifically configured to: the first parking position includes multiple sub-areas, and mark each sub-area for obstacle occupancy.

[0054] In one possible design, the marking module is specifically configured to: when the first parking position is partially outside the parking area and the first obstacle occupies the portion outside the parking area, mark the edge of the parking area covered by the first parking position as being occupied by the obstacle.

[0055] In one possible design, the marking module is specifically used to: identify the first obstacle, and if the first obstacle affects parking, mark the first obstacle as occupied; if the first obstacle has no effect on parking, mark the first obstacle as unoccupied.

[0056] In a further possible design, the marking module is specifically used to: if it is determined that the first obstacle meets at least one of the following conditions, then it is determined that the first obstacle has an impact on parking: the height of the first obstacle exceeds the set height, the maximum distance that the first obstacle intrudes into the first parking position is greater than the set distance, and the first obstacle is not in the preset whitelist.

[0057] In a further possible design, the marking module is also used to: if the first obstacle has no effect on parking, issue a first prompt message, and the first prompt message is used to prompt the user to choose whether to ignore the first obstacle; the control module is also used to: control the vehicle to park in the first parking position based on a second instruction, and the second instruction is an instruction to ignore the first obstacle.

[0058] In a further possible design, at least two different marking methods are adopted, including occupied marking, unoccupied marking, or occupied marking with different degrees of influence.

[0059] In a possible design, the first obstacle includes a dynamic obstacle and a static obstacle, and the dynamic obstacle and the static obstacle are marked in different ways.

[0060] In one possible design, the annotation includes at least one of a human-machine interface HMI annotation, a voice prompt annotation, a direction prompt sound annotation, a light annotation, and a projection annotation.

[0061] In a further possible design, the HMI annotation includes at least one of the following: graphic annotation, text annotation, color annotation, and pattern annotation.

[0062] In a further possible design, when using direction prompt sound annotation, the annotation module is specifically used to: emit a prompt sound at a corresponding direction relative to the driver in the vehicle according to the position of the first obstacle relative to the first parking position.

[0063] In one possible design, the labeling module is further used to: provide first recommendation information based on the position of the first obstacle and the surrounding environment of the vehicle, where the first recommendation information is used to indicate a recommended parking space adjustment direction and / or a recommended parking space adjustment position.

[0064] In one possible design, the marking module is further used to: mark the edge position of the first obstacle; and / or mark the critical safety position, where the critical safety position is a position at a preset safety distance from the edge position of the first obstacle.

[0065] In one possible design, the marking module is further configured to project the first parking position in the surroundings of the vehicle and mark the first obstacle in the first parking position.

[0066] In a third aspect, the present application provides a parking assistance system that can be integrated into a vehicle. The parking assistance system includes a control device and a display device. The control device is configured to generate a parking area based on the vehicle's surroundings and transmit the generated information to the display device. The display device is configured to display the parking area. The control device is further configured to control the display device to display a first parking position and mark the position of a first obstacle based on a first instruction. The first obstacle includes an obstacle that interacts with the first parking position.

[0067] In one possible design, the display device may include an in-vehicle display screen, such as an instrument panel, a central control screen, or a co-pilot screen, or may include an in-vehicle projection screen, such as a windshield or a skylight.

[0068] In a possible design, the parking assistance system may further include a camera; the camera is used to: capture an image of the environment; and the control device is further used to: generate a parking area based on the image of the environment captured by the camera.

[0069] In one possible design, the parking assistance system may also include on-board sensors, which may include one or more of cameras, radars, and sonars; the on-board sensors are used to: collect environmental information and identify obstacle positions and / or motion trajectories from the environmental information; the control device is used to: determine obstacles that interact with the first parking position based on the obstacle positions and / or motion trajectories identified by various on-board sensors.

[0070] In a possible design, the control device is further configured to control the vehicle to park in the first parking position when the first obstacle is not present.

[0071] In a further possible design, the parking assistance system may also include a mechanical steering gear, a motor controller and a motor, the mechanical steering gear being connected between the steering wheel and the wheels, and the motor being connected between the motor controller and the wheels; the control device being specifically used to: when there is no first obstacle, plan the parking trajectory of the vehicle according to the current position of the vehicle and the first parking position, determine the next turning angle and speed according to the parking trajectory, send the turning angle to the mechanical steering gear, and send the speed to the motor controller; the mechanical steering gear being used to: drive the wheels to rotate a corresponding angle according to the turning angle from the control device; the motor controller being used to: generate a corresponding driving electrical signal according to the speed from the control device and send it to the motor; the motor being used to: rotate according to the received driving electrical signal to drive the wheels to rotate.

[0072] It should be noted that the various designs in the above-mentioned first aspect are also applicable to the third aspect, and will not be repeated here one by one.

[0073] In a fourth aspect, the present application provides a parking assistance device, comprising a processor coupled to a memory, the processor being configured to execute a computer program or instruction stored in the memory, so that the parking assistance device performs a parking assistance method as described in the first aspect or any one of the designs of the first aspect.

[0074] In a fifth aspect, the present application provides a vehicle, comprising a parking assistance system as in the third aspect or any one of the designs of the third aspect, or comprising a unit or module for implementing the parking assistance method as in the first aspect or any one of the designs of the first aspect, for example, may comprise a parking assistance device as in the second aspect or any one of the designs of the second aspect, or may comprise a parking assistance device as in the fourth aspect.

[0075] In a sixth aspect, the present application provides an electronic device connected to a controlled vehicle for communicating with the controlled vehicle to implement the parking assistance method described in the first aspect or any one of the embodiments of the first aspect. The electronic device may include a unit or module for implementing the parking assistance method described in the first aspect or any one of the embodiments of the first aspect, for example, the parking assistance device described in the second aspect or any one of the embodiments of the second aspect, or the parking assistance device described in the fourth aspect.

[0076] In a seventh aspect, the present application provides a computer-readable storage medium, which stores a program or instruction. When the program or instruction is executed, it implements the parking assistance method as in the first aspect or any one of the designs of the first aspect.

[0077] In an eighth aspect, the present application provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer executes the parking assistance method as in the first aspect or any one of the designs of the first aspect.

[0078] The technical effects that can be achieved in the above-mentioned second to eighth aspects can refer to the description of the beneficial effects in the above-mentioned first aspect, and will not be repeated here one by one. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 A schematic diagram illustrating a possible application scenario provided by this application is shown as an example;

[0080] Figure 2 A schematic diagram illustrating an architecture of a parking assistance system provided by the present application is exemplified;

[0081] Figure 3 A schematic diagram exemplarily illustrates a flow chart of a parking assistance method provided by the present application;

[0082] Figure 4a An example of a parking area diagram from a frontal perspective provided by the present application is shown;

[0083] Figure 4b An example of a parking area diagram from a top-down perspective provided by the present application is shown;

[0084] Figure 5a A schematic diagram of a parking area of a custom parking space provided by the present application is exemplified;

[0085] Figure 5b A schematic diagram of a parking area for a tow parking position provided by the present application is exemplified;

[0086] Figure 6a An example of a parking area diagram for automatically generating parking spaces provided by the present application is shown;

[0087] Figure 6b A schematic diagram of a parking area for automatically generating parking spaces provided by the present application is exemplified;

[0088] Figure 7a A schematic diagram illustrating an obstacle center position marking method provided by the present application is shown as an example;

[0089] Figure 7b A schematic diagram illustrating an obstacle outline provided by the present application is exemplarily shown;

[0090] Figure 7c A schematic diagram of marking the farthest intrusion point of an obstacle provided by the present application is exemplified;

[0091] Figure 8 An exemplary diagram illustrating a first parking location area marking method provided by the present application;

[0092] Figure 9a A schematic diagram of marking obstacles with symbols provided by the present application is exemplarily shown;

[0093] Figure 9b A schematic diagram of marking obstacles with text provided by the present application is exemplified;

[0094] Figure 9c A schematic diagram of marking obstacles by color provided by the present application is exemplified;

[0095] Figure 9d A schematic diagram of marking obstacles with patterns provided by the present application is exemplified;

[0096] Figure 9e A schematic diagram of marking obstacles with lines provided by the present application is exemplarily shown;

[0097] Figure 10a A schematic diagram illustrating a scenario in which an obstacle is located outside a parking area provided by the present application is exemplified;

[0098] Figure 10b A schematic diagram illustrating an obstacle outside a parking area provided by the present application is shown as an example;

[0099] Figure 10c Another schematic diagram provided by the present application for marking obstacles outside a parking area is shown as an example;

[0100] Figure 11a A schematic diagram of a parking area showing a recommended dragging position and dragging direction provided by the present application is exemplified;

[0101] Figure 11b A schematic diagram of a parking area provided by the present application with obstacle edge positions and critical safety positions marked is shown as an example;

[0102] Figure 12a A schematic diagram illustrating an example of marking obstacles in a horizontal parking space provided by the present application is shown;

[0103] Figure 12b A schematic diagram illustrating an obstacle marking method in a vertical parking space provided by the present application is shown as an example;

[0104] Figure 12c Another schematic diagram of marking obstacles in a vertical parking space provided by the present application is exemplarily shown;

[0105] Figure 12dA schematic diagram illustrating an obstacle marking method in a parking area provided by the present application is shown as an example;

[0106] Figure 12e Another schematic diagram of marking obstacles in a parking area provided by the present application is exemplarily shown;

[0107] Figure 12f A schematic diagram illustrating another method of marking obstacles in a parking area provided by the present application is shown as an example;

[0108] Figure 12g A schematic diagram illustrating another method of marking obstacles in a parking area provided by the present application is exemplified;

[0109] Figure 12h A schematic diagram illustrating another method of marking obstacles in a parking area provided by the present application is shown as an example;

[0110] Figure 13 A schematic diagram illustrating an architecture of a parking assistance system provided by the present application is exemplified;

[0111] Figure 14 A schematic diagram illustrating the structure of a parking assistance device provided by the present application is shown as an example;

[0112] Figure 15 The structural diagram of another parking assistance device provided by the present application is exemplified. DETAILED DESCRIPTION

[0113] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0114] The following is an explanation of some of the terms used in this application. It should be noted that these explanations are for the purpose of facilitating understanding by those skilled in the art and do not limit the scope of protection claimed in this application.

[0115] 1. Image distortion in parking areas

[0116] When capturing images, the ideal camera position is perpendicular to the shooting plane, ensuring that the image is reproduced in its original geometric proportions. However, in the practical application of autonomous vehicles, due to the limitations of the vehicle structure, the camera's installation position will have a certain preview distance. The camera's horizontal and vertical scanning planes expand in a fan-shaped pattern, and the camera is at a certain angle to the ground. This angle will cause certain distortions in the captured image, mainly radial distortion and tangential distortion.

[0117] Due to these distortions, the displayed image of the parking area may be misaligned or inaccurate. For example, a vehicle's sensors may detect an obstacle at a certain location, but the display may not show the obstacle at that location. This can cause the user to mistakenly select a parking spot where the obstacle is located, causing inconvenience.

[0118] 2. Horizontal parking spaces and vertical parking spaces

[0119] Horizontal parking spaces, also known as parallel parking spaces, occupy a relatively large area and are more convenient for parking, generally suitable for roadside parking. Perpendicular parking spaces save space but require backing up when parking or driving, and are generally suitable for indoor and outdoor parking lots. Generally, within a parking area, if the long side of the space is open, it is considered a horizontal space. If the short side of the space is open, it is considered a perpendicular space.

[0120] The previous text introduced some of the terms involved in this application. The following text introduces the possible application scenarios of this application.

[0121] See also Figure 1 , exemplarily shows a possible application scenario of the present application. This application scenario takes the application of the parking assistance solution to a vehicle as an example, and the vehicle has an automatic parking function. Figure 1 As shown, assume the vehicle to be parked is vehicle A. There is an open area to the right of vehicle A, flanked by greenery. In this parking scenario, if the user has enabled the automatic parking function, vehicle A automatically collects information about its surroundings and generates a parking area based on the vehicle's surroundings. This area is then presented to the user via a display device, and the user waits for the user to select a parking location. If the user's selected parking location is unoccupied, the vehicle plans a driving path based on its current location and the parking location, and automatically parks in that location according to the driving path. Conversely, if the user's selected parking location is occupied, the vehicle instructs the user to reselect a parking location until the user selects an available parking location, at which point the vehicle automatically parks in that available location.

[0122] Exemplarily, the above-mentioned vehicles can be pure electric vehicles (pure electric vehicle / battery electric vehicle, pure EV / battery EV), hybrid electric vehicles (hybrid electric vehicle, HEV), range extended electric vehicles (range extended electric vehicle, REEV), plug-in hybrid electric vehicles (plug-in hybrid electric vehicle, PHEV), or other new energy vehicles (new energy vehicle, NEV), or they can also be fuel vehicles, etc. These vehicles can be used in fields such as unmanned driving, assisted driving, intelligent driving, automatic driving or connected vehicles.

[0123] It should be understood that the above application scenarios are merely examples, and the parking assistance solution provided in this application can also be applied in other possible scenarios, not limited to the ones exemplified above. For example, the parking assistance solution can also be applied to other modes of transportation, such as ships, airplanes, trains, subways, high-speed trains, submarines, rockets, or spacecraft, as an auxiliary control solution for achieving efficient parking of other vehicles, thereby reducing unnecessary parking delays. For another example, the parking assistance solution can also be applied in the medical field, such as in smart wheelchairs with display screens. When obstacles are present in the automatically planned parking location, the location of the obstacles can be displayed to the patient in the wheelchair, assisting the patient in finding a safe parking spot. For another example, the parking assistance solution can also be applied in the smart home field, such as robot vacuums, robot mops, autonomous food delivery robots, or mobile smart appliances. These devices can interact with user terminals to assist users in remotely indicating a safe parking location. For another example, the parking assistance solution can also be applied in smart living scenarios, such as in automatically following suitcases, smart dining chairs, or smart mobility devices, etc. I will not list them all here.

[0124] It should be noted that the application scenarios described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application.

[0125] As described in the background, existing automated parking solutions suffer from low parking efficiency. This problem is particularly pronounced in automated parking solutions that utilize a custom parking feature (also known as custom parking, custom parking space parking, etc.) to select a parking location. The custom parking feature is a commonly used parking space selection feature that allows users to select or drag their vehicle to a desired parking location within a parking area, satisfying their personalized needs. However, as noted in the aforementioned terminology explanation, the displayed image of the parking area often exhibits some distortion, which can cause the image to appear different from the actual environment. For example, an obstacle may be present at a certain location in the actual environment, detected by the vehicle, but not displayed at that location in the parking area, or the displayed location may deviate from the actual location, or the displayed size may deviate from the actual size. Due to these discrepancies, users who select a parking location based solely on the displayed image of the parking area may mistakenly select a location with an obstacle, rendering the currently selected parking location unavailable.

[0126] When a parking spot is unavailable, the current parking solutions offered by the industry only display the unavailable status in the parking area, such as displaying the text "Current location is unavailable." However, from the user's perspective, the currently selected location in the parking area is not blocked by obstacles. Even if the user is informed of the unavailable status, the user does not know the specific location or size of the obstacle. This results in the user still being able to select a location occupied by an obstacle when selecting the next parking spot, and the user may need to repeatedly select multiple times before finally selecting a suitable parking spot. This method of requiring repeated selections obviously reduces the efficiency of automatic parking and is not conducive to improving the user's automatic parking experience.

[0127] In view of this, the present application provides a parking assistance method. When there is an obstacle at the currently selected parking location, by marking the obstacle, the user can accurately know the location of the obstacle, and then assist the user to avoid the obstacle and select a parking location without obstacles as soon as possible, so as to improve the efficiency of automatic parking and enhance the user's parking experience.

[0128] The parking assistance solution proposed in this application is described in detail below with reference to specific drawings.

[0129] In the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0130] To facilitate understanding of this application, a system for implementing a parking assistance function is first provided. Figure 2 , which is a schematic diagram of a possible architecture of the system. Functionally, the architecture may include three major types of systems, namely, a sensor system 210 , a human-computer interaction system 220 and a control system 230 .

[0131] The sensor system 210 may include various on-board sensors for detecting the vehicle's surrounding environment, such as a camera 211 (e.g., a four-way fisheye camera) and a radar 212 (e.g., an ultrasonic radar, such as a 12-way ultrasonic radar). The camera 211 is typically mounted near the license plate frame and license plate light, inside the bumper, on the door, or at the bottom of the rearview mirror, and is primarily responsible for capturing images of the environment surrounding the vehicle. The radar 212 is typically mounted on the roof, near the bumper, on the lights, on the doors, and in the rearview mirror, and is primarily responsible for capturing information about the distance between obstacles around the vehicle and the vehicle. In some embodiments, the sensor system 210 may also include a sonar 213, which is typically mounted at the front or rear of the vehicle and is primarily responsible for capturing the distance between obstacles in front of or behind the vehicle (e.g., vehicles, pedestrians, or other obstacles) and the front or rear of the vehicle to prevent collisions.

[0132] The human-computer interaction system 220, which may also be referred to as an HMI system, may include various devices for realizing human-computer interaction, such as an image processor 221 and a display device 222. The image processor 221 is connected to the camera 211, and is mainly responsible for processing the environmental image captured by the camera 211, and completing operations such as parking space recognition, obstacle recognition, and passable area recognition. In some scenarios, the image processor 221 may also perform anti-distortion processing on the environmental image, such as inverse perspective mapping (IPM). The display device 222 may include any type of screen that can realize a display function, such as an instrument panel, a navigation screen, a central control screen, or a co-pilot screen, or other vehicle-mounted display screens, such as a skylight or side window, or a windshield in a head-up display (HUD).

[0133] Control system 230 may include various devices for controlling the entire vehicle and its components. For example, it may include a control device 231. This control device 231 is primarily responsible for module scheduling, receiving radar data, image processing data, and sonar data, performing data information fusion, determining planning schemes, and generating and outputting planning instructions. Control device 231 may be any control device within the vehicle, including, but not limited to, devices for controlling the entire vehicle, such as a VCU or VDC; devices for intelligent driving or assisted driving, such as an intelligent driving domain control unit or MDC; devices for controlling vehicle components, such as an ECU or MCU; and devices for controlling the vehicle's driving mode, such as an IBS.

[0134] For example, in a specific example, after the vehicle is started, the vehicle's default control device 231 is VCU or VDC. However, when the vehicle starts the automatic parking function, MDC will take over the vehicle control, so the control device 231 becomes MDC. After the automatic parking is completed, VCU or VDC will take over the vehicle again, so that the control device 231 becomes VCU or VDC again. Therefore, in this example, when the user turns on the automatic parking function, the control device 231 can be considered to be MDC. However, it should be understood that in other examples, the automatic parking function can also be encapsulated in other control devices, such as VCU or VDC. In this case, the control device 231 during the automatic parking process can also be VCU or VDC, and this application does not make specific limitations on this.

[0135] It should be understood that the above parking assistance system architecture is only an example. In other examples, the parking assistance system may also include more, fewer, or different components, each component may also include more, fewer, or different parts, and the parts shown or not shown may also be combined or divided in any manner. The division of the units in the figure is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into one physical entity, or they may be physically separated. This application does not make specific limitations on this.

[0136] See also Figure 3 , shows a flow chart of a parking assistance method provided by the present application, which is applicable to a parking assistance device. The parking assistance device may be a control device in a vehicle, such as Figure 2 The control device 231 shown in the figure can also be a control device outside the vehicle, such as a cloud server, terminal device, RSU or other vehicle. Among them, the terminal device can be understood as the device of the user riding in the controlled vehicle, such as but not limited to mobile phones, tablets, laptops, smart watches, Bluetooth headsets and other wearable devices. The control device outside the vehicle can be connected to the relevant components in the controlled vehicle through the network, such as Figure 2 Various types of vehicle-mounted sensors 211 - 213 , an image processor 221 and a display device 222 are shown to implement a parking assistance solution for the controlled vehicle.

[0137] For example, a parking assist device is used Figure 2 As an example, the control device 231 shown in FIG. Figure 3 As shown, the method includes the following steps:

[0138] Step 301 : Control a display device to display a parking area, where the parking area is generated based on the surrounding environment of the vehicle.

[0139] Optionally, during the driving process of the vehicle, if the control device 231 receives an automatic parking request triggered by the user, it can collect environmental information based on various vehicle-mounted sensors 211 to 213 to determine the vehicle information and the surrounding environment information of the vehicle, and then generate a parking area based on this information, and send the parking area to the display device 222 to display it to the user through the display device 222.

[0140] For example, the control device 231 may obtain an environmental image captured by the camera 211 and determine the vehicle's position, shape, and surrounding environment based on the environmental image. It then generates a parking area based on the pre-configured vehicle image and sends it to the display device 222. In some scenarios, before sending the parking area to the display device 222, the control device 231 may also identify obstacle information around the vehicle (such as the distance, location, and shape of the obstacle) based on radar data collected by the radar 212 and sonar data collected by the sonar 213. These obstacles are then added to the generated parking area to provide a more detailed and accurate image of the parking area.

[0141] Furthermore, optionally, the parking area can be divided into multiple perspectives, e.g. Figure 4a and Figure 4b , showing two possible perspectives of parking area diagrams provided by this application, Figure 4a The front view is shown. Figure 4b In one example, after the automatic parking function is turned on, the control device 231 can control the display device 222 to display Figure 4a The parking area in the front view is shown. The parking area shows the environment in front of the vehicle. When you need to switch to another perspective, you can click on a specific location in the parking area to switch to another perspective. For example, click Figure 4a Position the parking area above the Figure 4b The top-down perspective shown in the figure. This top-down perspective not only displays an image of the surrounding environment but also an image of the vehicle. Compared to the front-view perspective, it better reflects the vehicle's position in the surrounding environment. Therefore, the parking area in the top-down perspective is generally used as a reference interface for selecting a parking position. It is understood that actual vehicles may also include other perspectives, such as rear-view, left-view, and right-view perspectives, and this application does not specifically limit these.

[0142] by Figure 4bTaking the top view as an example, it should be noted that the figure shows a complete parking interface, which can be divided into two parts, left and right. The left part is called the control area, and the right part is called the parking area. The control area on the left contains some control buttons. The user can control the display device 222 to display the corresponding content in the parking area on the right by clicking these control buttons. See the following for details. It should be understood that Figures 4a to 4b The parking interface shown is only an example, and this application does not limit the specific presentation form of the parking interface. For example, in another possible example, the parking interface can also be presented as a whole, or can be divided into two parts, upper and lower, or can be divided into at least three parts. For example, in another possible example, the position, function and form of each element in the parking interface can also be different from the above. Figures 4a to 4b There are differences, for example, the function buttons can be replaced with text or icon forms, the position of the function buttons can also change, etc. This application does not make specific restrictions on this.

[0143] Step 302: Control a display device to display a first parking position based on a first instruction.

[0144] Alternatively, the first instruction may be triggered by a user performing a certain operation, or may be automatically generated by the vehicle after the automatic parking feature is enabled. For example, the first instruction may be triggered by a user dragging or selecting a parking area. In this case, the first parking position is the parking position determined by the user, such as the parking position dragged or selected by the user. For another example, the first instruction may be automatically generated by the vehicle after detecting that the automatic parking feature has been enabled. In this case, the first parking position is the parking position automatically generated by the vehicle.

[0145] It should be noted that the first parking position can be the first parking position dragged by the user, or the first selection by the user, or the first parking position automatically generated by the vehicle after the automatic parking function is started. It can also be the parking position dragged again by the user, or reselected by the user, or regenerated by the vehicle after the parking position dragged by the user, or the last selection, or the automatically generated parking position cannot be parked. There is no specific limitation.

[0146] For example, taking the parking position that the user drags for the first time as an example, see Figure 4b In the parking area shown, a button M for a custom parking space function may also be present in the control area on the left. When the control device 231 detects that the button M is clicked by the user, the display device 222 may be controlled to generate and display a virtual parking space in the parking area on the right, such as Figure 5aAs shown. The virtual parking space can be generated around the vehicle, such as any one or multiple directions of the left, right, top and bottom of the vehicle. The virtual parking space itself can be dragged, and a rotation button K can be displayed around it. The user can change the position of the virtual parking space by clicking and dragging the virtual parking space. The user can change the posture of the virtual parking space by clicking and holding the rotation button K and rotating it. The user's dragging and rotating operations trigger the first instruction. The control device 231 controls the display device 222 according to the first instruction to display the virtual parking space at the position after the user's dragging and rotating operations are completed, that is, the first parking position. For example, please refer to Figure 5b , assuming that the control device 231 moves the virtual parking space to the left front of the actual vehicle according to the first instruction triggered by the user's dragging operation and rotation operation, and rotates the virtual parking space clockwise by a certain angle so that it is exactly on the platform, then the position finally occupied by the virtual parking space is the first parking position.

[0147] For example, taking the parking location selected by the user for the first time, see Figure 4b In the parking area shown, there may also be a button N for automatically generating parking spaces in the control area on the left. When the control device 231 detects that the button N is clicked by the user, it can automatically plan multiple parking spaces that can be parked and multiple parking spaces that cannot be parked based on the information of various vehicle-mounted sensors 211 to 213, and can control the display device 222 to display these parking spaces in the parking area on the right, as shown in FIG. Figure 6a As shown. Optionally, different types of parking locations can also be presented to the user in different colors, for example, available parking locations are displayed in green, and unavailable parking locations are displayed in red. The user can trigger the first instruction by clicking on one of the parking locations. The control device 231 determines the parking location clicked by the user as the first parking location according to the first instruction, and can control the display device 222 to fill in the virtual vehicle logo in the first parking location. For example, if the control device 231 detects that the user clicks Figure 6a If the green parking position shown in FIG. 1 is a green parking position, the green parking position may be used as the first parking position, and the display device 222 may be controlled to fill in the virtual vehicle identifier in the green parking position, such as Figure 6b shown.

[0148] For another example, take the first generated parking position of a vehicle as an example, please refer to Figure 6a After the user clicks the button N in the parking area, the first instruction can be automatically triggered. The control device 231 automatically plans multiple parking locations that can be parked and multiple parking locations that cannot be parked based on the information of various vehicle-mounted sensors 211 to 213, and can automatically select a parking location among them as the first parking location according to the first instruction. Then, the display device 222 can be controlled to fill in the virtual vehicle identifier in the first parking location, and the following is obtained: Figure 6bParking areas shown.

[0149] It is understandable that the first instruction can also be triggered by other means, such as but not limited to: user's voice instruction triggering, gesture instruction triggering, brain wave instruction triggering, etc., which will not be listed one by one in this application.

[0150] Step 303: Mark the position of the first obstacle, where the first obstacle includes an obstacle that interacts with the first parking position.

[0151] In one possible implementation, the control device 231 may identify the environment based on various types of vehicle-mounted sensors 211 to 213 to determine the positions of one or more obstacles in the environment surrounding the vehicle body, and then, based on the positional relationship between the positions of the one or more obstacles and the first parking position, screen out obstacles that may affect the vehicle parking in the first parking position from the one or more obstacles, and identify these obstacles as first obstacles.

[0152] Here, obstacles that affect the vehicle's parking in the first parking position can be understood as obstacles that affect parking from the perspective of position and / or movement trend. For example, they may include but are not limited to at least one of the following two types of obstacles:

[0153] The first category is obstacles that interact with the first parking position. For example, obstacles that are fully or partially displayed in the first parking position, such as static obstacles such as plastic bags, cardboard boxes, lawns, stone pillars, curbs, or dynamic obstacles that are fully or partially in the first parking position. Another example is obstacles that are located outside the first parking position but will interact with the first parking position, such as dynamic obstacles that will move into the first parking position within the next 1 minute (or other relatively short time), such as pedestrians, oncoming vehicles, small animals, toy cars, etc.

[0154] The second category is obstacles that are displayed outside the first parking position, but the on-board sensors sense that they are fully or partially located within the first parking position, such as static obstacles and dynamic obstacles that are not displayed within the first parking position due to image distortion and other reasons.

[0155] For example, taking the above two types of obstacles as an example, the control device 231 can obtain the position and movement trajectory of one or more obstacles identified by the camera 211, radar 212 and sonar 213 based on the recognition of the vehicle's surrounding environment by the camera 211, radar 212 and sonar 213. Then, based on the position of the one or more obstacles, it can be determined whether the position of the obstacle is located within the first parking position; based on the position and movement trajectory of the one or more obstacles, it can be determined whether the obstacle will move into the first parking position within a short period of time in the future; and based on the position and movement trajectory of the one or more obstacles, it can be determined whether a certain obstacle is partially or completely located within the first parking position but the parking area shows that the obstacle is located outside the first parking position. Afterwards, the obstacle determined as "yes" can be regarded as the first obstacle.

[0156] When the first obstacle is of the first category, the first obstacle is displayed within the first parking space, or is displayed outside the first parking space but will be moved into the first parking space within a short period of time. When the first obstacle is of the second category, the first obstacle is displayed outside the first parking space, but the vehicle determines through onboard sensors that the first obstacle is fully or partially within the first parking space. The first obstacle can be understood as the actual obstacle in the actual environment corresponding to the first obstacle displayed in the parking area.

[0157] As will be appreciated, if the first obstacle does not exist, indicating that the first parking position is available for parking, control device 231 may send control signals to other control components in control system 230 to drive the other control components to automatically park the vehicle in the first parking position. For example, control device 231 may plan a parking trajectory for the vehicle from its current position to the first parking position based on the first parking position and the vehicle's current position. It may then determine the vehicle's next turning angle and speed based on the parking trajectory. The turning angle may then be sent to a mechanical rotator (e.g., a steering rod) of the vehicle, and the speed may be sent to a motor controller of the vehicle. The mechanical steering device is connected between the vehicle's steering wheel and wheels. Upon receiving the turning angle from control device 231, the mechanical rotator may drive the wheels and steering wheel to rotate to a corresponding angle. The steering wheel rotation may present steering information to the user, while the wheel rotation may drive the vehicle to steer. The motor controller is connected to the vehicle's motor, which is in turn connected to the wheels. After receiving the speed information from control device 231, the motor controller generates a corresponding drive signal and sends it to the motor. The motor rotates at the speed corresponding to the drive signal, driving the wheels and, in turn, moving the vehicle forward or backward. Using this control logic, control device 231 can control the vehicle to gradually park in the first parking position along the planned trajectory.

[0158] Conversely, if the first obstacle exists, it indicates that the first parking location may not be available. In this case, the control device 231 may mark the location of the first obstacle. This marking may be implemented in various ways, including but not limited to at least one of HMI marking, voice prompt marking, direction prompt sound marking, light marking, and projection marking.

[0159] The HMI annotation may refer to controlling the display device 222 to display the first obstacle in the parking area. There are many ways to display the first obstacle, such as but not limited to: graphic annotation, text annotation, color annotation, pattern annotation, etc.

[0160] The voice prompt annotation may be, for example, a voice prompt message issued through the vehicle's speakers (or external speakers if supported) or a voice device such as a voice assistant, to inform the user where the first obstacle is located in the first parking position. These locations may be, for example, the left side of the vehicle, the right side of the vehicle, the front, the middle, or the rear of the vehicle, or a combination thereof, or other iconic locations within the vehicle, without limitation.

[0161] The positional sound indication can be based on the position of the first obstacle relative to the first parking position, and a sound can be emitted at the corresponding position in the vehicle relative to the driver. For example, when the first obstacle is located to the left rear of the first parking position, a vibration or ringing sound can be emitted through the speaker in the left rear seat of the vehicle. This allows the user to intuitively sense the location of the obstacle even when the vehicle is not parked in the first parking position, thereby improving the parking experience.

[0162] Light marking can be achieved by using interior or exterior vehicle lights to mark the location of the first obstacle. For example, when the first obstacle is located in the front left corner of the first parking space, the vehicle's left front turn signal can be flashed. Another example is when the first obstacle is located in the center of the first parking space, the interior ambient light can be flashed. And so on.

[0163] The projection marking can include using in-vehicle or external projection equipment to mark the location of the first obstacle. For example, the in-vehicle head-up display (HUD) can display the first parking position and the position of the first obstacle within the first parking position in a virtual image in front of the windshield. In another example, an onboard projector can display the location of the first obstacle within the vehicle, such as on the left or right window, front seat, rear seat, or other interior space. In another example, the vehicle's exterior lights can be used to project the first parking position onto the vehicle's surroundings and identify the first obstacle within the first parking position. For example, smart headlights can project a parking space at the location corresponding to the first parking position in the real environment, and obstacles such as pedestrians, vehicles, and boulders currently within the area can be identified with red light or other colored light. In this way, if the obstacle is dynamic, this method can provide a signal to the dynamic obstacle, prompting it to move away from the first parking position as quickly as possible, thereby assisting the user in completing parking.

[0164] Of course, there are other ways of marking. For example, you can also use dynamic effects in the interface of the central control screen. For example, vibration or flashing prompts. For example, you can also display prompts on the display screen outside the car. And so on. This application does not make specific restrictions on this.

[0165] In one possible implementation, if the first obstacle includes a dynamic obstacle, the control device 231 may also dynamically update the occupancy status of the first parking space. For example, if the vehicle has not left its current location or has left a short distance, the control device 231 may reacquire the surrounding environment captured in real time by various sensors every 5 seconds (or other relatively short period of time). Based on this surrounding environment, the control device 231 re-determines obstacles within the first parking space or outside the first parking space that may interact with the first parking space, thereby updating the obstacle information displayed in the parking area, promptly notifying the user of the removal of dynamic obstacles, and assisting the user in parking in the first available parking space.

[0166] In one possible implementation, when determining to mark the location of the first obstacle, the control device 231 may use a variety of methods to mark the location. Three possible marking methods are exemplified below. Furthermore, for ease of understanding, the following three marking methods are described using HMI implementation as an example.

[0167] Marking method 1: direct marking.

[0168] In marking method 1, the control device 231 can mark the first parking space as an obstacle occupancy based on the locations of one or more obstacles sensed by various onboard sensors 211-213. In other words, the obstacle occupancy is directly marked at the first parking space. In other words, the obstacle occupancy information is directly marked at the first parking space. This occupancy information can include location, size, or degree of encroachment.

[0169] For example, Figure 5b For example, see the parking area shown. Figure 7a 、 Figure 7b and Figure 7c , assuming that there is an obstacle X in front of the left side of the vehicle, and the user drags the virtual parking space to the location of obstacle X, then:

[0170] For an example, see Figure 7a The control device 231 may control the display device 222 to mark the center position of the obstacle X at the first parking position, such as marking a circle or other shape at the center point of the obstacle X. Optionally, a line or text may be marked and pointed to the center point to display information indicating that there is an obstacle, so that the user can quickly locate the center point.

[0171] For another example, see Figure 7b The control device 231 may control the display device 222 to mark the outline of the obstacle X at the first parking position, so that the user can accurately know the area occupied by the first obstacle in the first parking position, and the user can quickly select an available parking position outside the area;

[0172] For another example, see Figure 7c The control device 231 can control the display device 222 to mark the location of the obstacle X closest to the center point of the first parking position. In other words, the deepest point of the obstacle X's encroachment into the first parking position is marked. For example, a circle or other shape can be marked at the deepest point, and a line and text can be used to introduce it, allowing the user to quickly locate the deepest point and understand the extent to which the first obstacle occupies the first parking position.

[0173] Using the above-mentioned marking method 1, the user can intuitively present the occupancy status of the obstacle itself, allowing the user to accurately know the location of the obstacle. It is understood that in addition to the three marking examples mentioned above, other marking examples can also exist, such as color marking, pattern marking, symbol marking, etc., which are not listed here one by one.

[0174] Marking method 2: area marking.

[0175] In the second marking method, the control device 231 can divide the first parking location into multiple sub-areas and mark each sub-area as occupied by an obstacle. For example, based on the locations of the obstacles sensed by the various onboard sensors 211-213, the control device 231 can determine the sub-area where the obstacle is located from among the multiple sub-areas, and then control the display device 222 to mark the sub-area where the obstacle is located. The number of sub-areas can be any integer, such as 4, 6, 8, etc., and is not specifically limited.

[0176] For example, taking six sub-areas as an example, see Figure 8 Figure (A) illustrates a possible sub-area division method for the first parking location. In this example, the control device 231 may divide the first parking location into six sub-areas and assign each of these six sub-areas a different identification (ID), such as S1, S2, S3, S4, S5, and S6. Sub-areas S1 and S2 are located at the front of the vehicle, sub-areas S3 and S4 are located in the middle of the vehicle, and sub-areas S5 and S6 are located at the rear. For each of these six sub-areas, the control device 231 may first determine whether an obstacle, in whole or in part, is located within the sub-area. If so, the sub-area is determined to be occupied by an obstacle, and the control device 231 may assign a first attribute to the sub-area. Otherwise, the sub-area is determined to be unoccupied by an obstacle, and the control device 231 may assign a second attribute to the sub-area. The control device 231 may then label the sub-areas assigned the first attribute based on the attributes of the six sub-areas, clearly indicating to the user the sub-area occupied by the obstacle in a visually appealing manner.

[0177] Optionally, the first attribute and the second attribute can be indicated in a variety of ways, such as text indication, pattern indication, numerical indication, etc. For example, in one example, the first attribute can be indicated by filling "1" in the sub-area, and the second attribute can be indicated by filling "0" in the sub-area. For example, see Figure 8 In (B1) and (B2), if the obstacle (for example, a cone is shown) only occupies the sub-area S4 of the first parking position, the control device 231 may fill the sub-area S4 with "1" and the remaining sub-areas S1 to S3, S5, and S6 with "0". After the filling is completed, the display device 222 may be controlled to mark the sub-area filled with "1" (i.e., S4). For another example, please refer to Figure 8In (C1) and (C2), if an obstacle (a wheel chock is used as an example in the figure) occupies both sub-areas S1 and S2 of the first parking position, the control device 231 may fill sub-areas S1 and S2 with "1" and may fill the remaining sub-areas S3 to S6 with "0". After the filling is completed, the display device 222 may be controlled to mark the sub-areas filled with "1" (i.e., S1 and S2). For another example, please refer to Figure 8 In (D1) and (D2), if an obstacle (using green plants as an example in the figure) occupies sub-areas S1, S3, and S5 of the first parking position at the same time, the control device 231 may fill sub-areas S1, S3, and S5 with "1" and may fill the remaining sub-areas S2, S4, and S6 with "0". After the filling is completed, the display device 222 may be controlled to mark the sub-areas filled with "1" (i.e., S1, S3, and S5). There are many possible examples, which are not listed here one by one.

[0178] Furthermore, there are many optional ways to mark the sub-regions, such as graphic marking, text marking, color marking, pattern marking, and line marking. Figure 5b Parking areas shown and Figure 8 Taking the obstacle position shown in (B1) as an example, assuming that the user drags the virtual parking space to the location of obstacle X, and the control device 231 determines that obstacle X occupies sub-area S4 in the first parking position, then:

[0179] For an example, see Figure 9a , the control device 231 can control the display device 222 to mark a set symbol on the sub-area S4, and the set symbol can be, for example, a rhombus, square, triangle, trapezoid, circle, ring or other regular or irregular shapes. Optionally, in order to enable the user to focus on the location of the obstacle X more quickly, the marked symbol can also be colored, such as gray as shown in the figure, or other colors. Optionally, in order to enable the user to quickly distinguish the type of obstacle, different types of obstacles can also be marked with different shapes and colors, such as green plants with green triangles, buildings with red circles, wheel chocks with gray rhombuses, cones with blue squares, etc., which are not listed here one by one;

[0180] For another example, see Figure 9b , the control device 231 can control the display device 222 to mark text on the sub-area S4, such as marking text directly at the center point of the sub-area S4, or marking text outside the first parking position, and draw an arrow (or broken line, or other forms) on the sub-area S4 to point to the text. The marked text can be uniform, such as "obstacles". Figure 9bAlternatively, different obstacles may be labeled with different characters, such as the Chinese name of the obstacle, the number of the obstacle, or the English name of the obstacle, etc., which will not be listed here one by one;

[0181] For another example, see Figure 9c , the control device 231 can control the display device 222 to fill the sub-area S4 with color, such as filling the entire sub-area S4 with a set color. The set color can be uniform, such as Figure 9c The color of the obstacle can be gray, or other colors, such as red. Red itself has a highlighting effect, which can help users focus on the location of the obstacle more quickly. Alternatively, different obstacles can be represented by different colors, such as gray for wheel chocks, green for green plants, red for buildings, blue for cones, etc., which will not be listed here one by one;

[0182] For another example, see Figure 9d The control device 231 can control the display device 222 to mark a pattern on the sub-area S4. For example, a pattern that can be used to indicate the type of obstacle can be marked on the sub-area S4. For example, when the obstacle X is a cone barrel, a pattern can be marked as follows: Figure 9d The cone pattern shown. By marking the obstacle type map, users can quickly know what type of obstacle is occupying the parking space, which can help users plan their next parking space. Of course, other patterns can also be marked, and this application does not specifically limit this;

[0183] For another example, see Figure 9e The control device 231 can control the display device 222 to mark a line on the sub-area S4, for example, to draw a line to outline the area occupied by the obstacle. This outline can help the user quickly understand the actual size of the obstacle and facilitate the selection of another parking location.

[0184] By using the second annotation method above, by annotating the sub-region to which the obstacle belongs, even if the actual obstacle location deviates from the detected obstacle location due to some reasons (such as detection error), the actual obstacle location can be included in the corresponding sub-region and annotated simultaneously, thereby increasing the probability that the user can adjust to a suitable parking position based on the annotation. It is understood that in addition to the above-mentioned annotation examples, other annotation examples are possible and are not listed here one by one.

[0185] Marking method three: edge marking.

[0186] In the third marking method, when the first parking position is partially outside the parking area and the first obstacle occupies the portion outside the parking area, the control device 231 may mark the positional relationship of the first obstacle relative to the first parking position, for example, marking the edge of the parking area covered by the first parking position as occupied by the obstacle. There are many ways to implement edge marking. For example, see Figure 10a Assuming that the first parking position is located at the upper left of the parking area, the front portion of the vehicle at the first parking position is located within the parking area, the rear portion of the vehicle at the first parking position is located outside the parking area, and there is an obstacle X outside the parking area, the control device 231 may control the display device 222 to mark the parking area with position information indicating that "obstacle X is located to the left of the current parking position." For example:

[0187] For an example, see Figure 10b The control device 231 can control the display device 222 to mark the boundary between the first parking position and the edge of the parking area with a color, such as black as shown, or a more obvious red, to attract attention. Optionally, to improve the viewing effect, the required marking color can also be marked in the form of light, such as marking the edge with red light to increase the aesthetics;

[0188] For another example, see Figure 10c The control device 231 can control the display device 222 to indicate the position of the obstacle X in the first parking position through arrows and text. For example, the arrow can be pointed to the intersection of the first parking position and the edge of the parking area, and the text "Obstacle on the left" can be used to indicate to the user that the obstacle is on the left side of the first parking position.

[0189] And so on. There are many possible annotation implementation forms, which are not listed here one by one.

[0190] By using the above marking method, users can intuitively know the location of the obstacle outside the parking area relative to the first parking position, thereby helping users to accurately avoid the location of the obstacle and quickly drag the virtual parking space to a position without obstacles, thereby achieving efficient placement of custom parking spaces and improving the adjustment efficiency of custom parking spaces.

[0191] It is understandable that in addition to the above three marking methods, there may be other marking methods, and this application does not make specific limitations on this.

[0192] In a possible implementation, in addition to marking the location of the first obstacle, the control device 231 may also provide first recommendation information based on the location of the first obstacle and the surrounding environment of the vehicle. The first recommendation information is used to indicate the recommended parking adjustment direction and / or recommended parking adjustment position. The first recommendation information may be voice recommendation, display recommendation, headlight flashing recommendation, etc. For example, taking the example of simultaneously recommending the parking adjustment direction and parking adjustment position in a display manner, please refer to Figure 11a The control device 231 can display the drag position and drag direction in the parking area by controlling the display device 222, so that the user can quickly move the virtual parking space to the recommended parking position by referring to the recommended drag direction, thereby accelerating parking.

[0193] In a possible implementation, in addition to marking the position of the first obstacle, the control device 231 may also mark the association between the first obstacle and the available parking position. The association may include, for example, the edge position of the first obstacle and / or a critical safety position, where the critical safety position is a position at a preset safety distance from the edge position of the first obstacle. For example, see Figure 11b The three solid lines within the first parking spot represent the edge of the first obstacle, and the three dashed lines represent the edge of the first obstacle plus a preset safety distance, i.e., the critical safety line. If parking conditions are met, such as if the free area around the first parking spot is large enough for the user to reselect a location farther from the current one, the user can drag the virtual parking space with reference to the critical safety line, ensuring that the virtual parking space does not extend beyond the critical safety line. If parking conditions are not met, such as if the free area around the first parking spot is relatively small, ensuring that the user only reselects a location relatively close to the current one, the user can drag the virtual parking space with reference to the edge of the first obstacle, ensuring that the virtual parking space does not extend beyond the critical safety line. This allows users to more intuitively identify the edge of the obstacle or the critical safety line, making it easier for them to quickly determine a safe area and accelerate the selection of a suitable parking spot.

[0194] In one possible implementation, upon determining the presence of a first obstacle, the control device 231 may further identify the first obstacle before marking the location of the first obstacle in the manner described above. If the first obstacle is identified as an obstacle that affects parking, the first obstacle may be annotated with occupied information, for example, using any of the marking methods 1 through 3 described above. Conversely, if the first obstacle has no effect on parking, the first obstacle may be annotated with unoccupied information. Alternatively, if it is impossible to determine whether the first obstacle affects parking, the first obstacle may be annotated with unknown information.

[0195] Optionally, the non-occupied information can be implemented in a variety of forms. For example, in one example, the non-occupied information can be implemented by not marking it. In this case, when a first obstacle is marked in the parking area, it indicates that the first obstacle is an obstacle that will affect parking. Therefore, the user can directly select a position that can avoid the first obstacle to adjust the parking space. For example, in another example, the non-occupied information can also be marked according to any of the marking methods one to three mentioned above, but it is different from the marking method corresponding to the occupied information. For example, obstacles that do not affect parking are marked in green, and obstacles that do affect parking are marked in red. For another example, obstacles that do not affect parking are marked as dots, and obstacles that do affect parking are marked as circles, and so on.

[0196] Alternatively, unknown information may be labeled using any of the above-mentioned methods 1 through 3, but the labeling methods may differ from those used for occupied and unoccupied information. For example, obstacles that do not affect parking may be labeled green, obstacles that do affect parking may be labeled red, and obstacles whose impact on parking cannot be determined may be labeled yellow. Another example is obstacles that do not affect parking may be labeled as dots, obstacles that do affect parking may be labeled as circles, and obstacles whose impact on parking cannot be determined may be labeled as triangles, and so on.

[0197] Optionally, the occupancy information for obstacles with varying degrees of impact on parking can be labeled differently. For example, obstacles with a significant impact on parking can be labeled red, obstacles with a moderate impact can be labeled gray, and obstacles with a slight impact can be labeled blue. Another example is obstacles with a significant impact can be labeled as pentagons, obstacles with a moderate impact can be labeled as quadrilaterals, and obstacles with a slight impact can be labeled as triangles, and so on.

[0198] Optionally, when the first obstacle is an obstacle that has no effect on parking, the control device 231 may also issue a first prompt message, and the first prompt message is used to prompt the user to choose whether to ignore the first obstacle. Afterwards, if the control device 231 receives a second instruction from the user to ignore the first obstacle, it can control the vehicle to automatically park in the first parking position based on the second instruction. The first prompt message may be one or more of a voice prompt, a display prompt, a flashing headlight prompt, a vibration prompt, etc. For example, the user may be prompted by voice through an on-board speaker or other on-board voice device, or by displaying text in the parking area, or by displaying text on the virtual image of the windshield through the HUD, or by projecting a pattern or text through the headlights, etc., without specific limitation. In addition, the user may trigger the second instruction by clicking a button, by voice, by gesture, or by other interactive methods, etc., without specific limitation.

[0199] Optionally, whether the first obstacle has an impact on parking is identified. There are many optional identification methods, and several examples are given below to illustrate.

[0200] One possible identification method can be through a neural network model. For example, a neural network model can be pre-trained based on known types of obstacles that may affect parking and types that do not. The first obstacle can then be identified using the trained neural network model. This method offers high accuracy.

[0201] Another possible identification method is to pre-extract first features for various types of obstacles that have no impact on parking and second features for various types of obstacles that do impact on parking. A first obstacle is then matched against both the first and second features. If the first obstacle possesses the first feature, it is determined to be an obstacle that has no impact on parking. If the first obstacle possesses the second feature, it is determined to be an obstacle that has an impact on parking. If the first obstacle possesses neither the first nor the second feature, it is determined to be an obstacle that cannot be determined to have an impact on parking. The first feature can include at least one of the following: the obstacle's height is less than or equal to a set height, the maximum distance the obstacle intrudes into the parking space is less than or equal to a set distance, or other features. The second feature can include at least one of the following: the obstacle's height is greater than a set height, the maximum distance the obstacle intrudes into the parking space is greater than a set distance, or other features.

[0202] Optionally, the aforementioned set height can be set based on the vehicle's chassis height. For example, it can be set to a value slightly lower than the chassis height. Thus, when the height of the first obstacle is less than or equal to the set height, it means that the first obstacle is lower than the vehicle's chassis height. Even if the vehicle passes over the obstacle, it is highly unlikely to scratch the chassis. Therefore, the first obstacle can be considered an obstacle that does not affect parking. Of course, the set height can also be set to other values, and there is no specific limitation.

[0203] Optionally, the aforementioned set distance can be set based on the distance between the wheels and the edge of the vehicle body. For example, it can be set to be slightly smaller than the distance between the wheels and the edge of the vehicle body. Thus, when the maximum distance that the first obstacle intrudes into the parking position exceeds the set distance, it means that the first obstacle has penetrated deeply into the position where the wheels are located and is likely to block the wheels from moving. Therefore, the first obstacle can be considered an obstacle that affects parking. Of course, the set distance can also be set to other values, and there is no specific limitation.

[0204] Another possible identification method is to configure a preset whitelist based on various types of obstacles that are known to have no effect on parking, and / or configure a preset blacklist based on various types of obstacles that are known to have an effect on parking. When the type of the first obstacle belongs to the preset whitelist, the first obstacle can be determined to be an obstacle that has no effect on parking. When the type of the first obstacle belongs to the preset blacklist, the first obstacle can be determined to be an obstacle that has an effect on parking. When the first obstacle belongs to neither the preset whitelist nor the preset blacklist, the first obstacle can be determined to be an obstacle that cannot be identified as to whether it affects parking.

[0205] Optionally, the preset whitelist can contain identification information, such as name or type, for obstacles that have minimal impact on parking. These obstacles may include soft materials such as plastic bags and cardboard boxes, or other obstacles with negligible impact. Conversely, the preset blacklist can contain identification information, such as name or type, for obstacles that have a significant impact on parking. These obstacles may include hard materials such as lawns, stone piers, curbs, or wheel chocks, or other obstacles with significant impact.

[0206] It is understandable that whether the first obstacle affects parking can be determined by the above-mentioned single identification method, or the above-mentioned multiple identification methods can be combined to comprehensively determine whether the first obstacle affects parking. Alternatively, other identification methods can be used, and this application does not make specific limitations on this.

[0207] In the above implementation, by distinguishing and marking obstacles that affect parking and those that do not, users can understand the type of the current obstacle so that they can respond to different types of obstacles more specifically and assist users in quickly adjusting parking positions.

[0208] In one possible implementation, the control device 231 may also use different labeling methods for static obstacles and dynamic obstacles. For example, static obstacles may be labeled red, while dynamic obstacles may be labeled yellow. Another example is that static obstacles may be labeled with text, while dynamic obstacles may be labeled with graphics. Another example is that static obstacles may be labeled with triangles, while dynamic obstacles may be labeled with squares, and so on. By distinguishing between static and dynamic obstacles, different display effects can be presented to the user, improving the user's parking viewing experience.

[0209] It is understandable that the above content only exemplifies several possible implementation methods of marking obstacles. This application does not limit the marking of obstacles in the parking area to these methods. Any marking method that can prompt the user of the location of the obstacle through an external effect is within the scope of protection of this application, and this application does not make specific limitations on this.

[0210] To further illustrate the parking assistance method in this application, the following describes several possible application scenarios and their specific implementations. Each of the following application scenarios uses color marking as an example, with the first parking position divided into six zones.

[0211] Application Scenario 1

[0212] See also Figure 12a , showing a schematic diagram of marking obstacles in a horizontal parking space provided by the present application.

[0213] like Figure 12a As shown, assuming that the user drags the virtual parking space to Figure 12a In the first parking position shown in (A), there is a cone in the actual scene. In this case, the control device 231 can determine that there is an obstacle located in the area S4 of the first parking position based on the detection results of various vehicle-mounted sensors 211 to 213. Area S4 is occupied by the obstacle. Therefore, the control device 231 can fill "1" in area S4 and fill "0" in other areas, such as Figure 12a As shown in (B). Figure 12a As shown in (C), the control device 231 can mark the area S4 filled with "1" with a color, and not mark other areas with a color. The marked color can be Figure 12a The gray shown in (C) can also be other colors, such as red, without specific limitation.

[0214] Application Scenario 2

[0215] See also Figure 12b , showing a schematic diagram of marking obstacles in a vertical parking space provided by the present application.

[0216] like Figure 12b As shown, assuming that the user drags the virtual parking space to Figure 12b In the first parking position shown in (A), there is a wheel chock in the actual scene. In this case, the control device 231 can determine that there is an obstacle across the area S1 and area S2 of the first parking position based on the detection results of various vehicle-mounted sensors 211-213. The obstacle prevents the parking of the wheels. Therefore, area S1 and area S2 are occupied by the obstacle. The control device 231 can fill "1" in area S1 and area S2 and fill "0" in other areas, such as Figure 12b As shown in (B). Figure 12b As shown in (C), the control device 231 can mark the area S1 and area S2 filled with "1" with different colors, for example Figure 12b Gray shown in (C).

[0217] Application Scenario 3

[0218] See also Figure 12c , showing another schematic diagram of marking obstacles in a vertical parking space provided by the present application.

[0219] like Figure 12c As shown in (A), it is assumed that the vehicle automatically generates four parking spaces, which are arranged side by side from top to bottom, and the user selects the second parking space at the top for parking. In the actual scene, there is a cone in the area between the vehicle and the second parking space. In this case, the control device 231 can determine that there are obstacles in areas S3 and S4 of the second parking space based on the detection results of various on-board sensors 211-213. Areas S3 and S4 are occupied by obstacles. Therefore, the control device 231 can fill "1" in areas S3 and S4 and fill "0" in other areas, as shown in FIG. Figure 12c As shown in (B). Afterwards, the control device 231 can mark the colors of the area S3 and area S4 filled with "1", such as Figure 12c Gray shown in (C).

[0220] Application Scenario 4

[0221] See also Figure 12d , showing a schematic diagram of marking obstacles in a parking area provided by the present application.

[0222] like Figure 12d As shown, assume that there is an obstacle X at the lower right corner of the vehicle's current position, and the user drags the virtual parking space to the right of the vehicle's current position during automatic parking, so that the lower left area of the dragged parking space is blocked by obstacle X. In this case, the control device 231 determines that the lower left area of the dragged parking space is occupied by the obstacle based on the detection results of the various on-board sensors 211-213. Therefore, the control device 231 can mark the lower left area of the dragged parking space with a color, such as gray as shown. Optionally, to improve the aesthetics of the parking area, the control device 231 can also set a gradient effect on the edges of the colored area, such as gradually transitioning from gray as shown to white in other areas, to enhance the user's viewing experience.

[0223] In an example, Figure 12d As shown, to more clearly indicate the location of the obstacle, the control device 231 may also mark a predetermined shape in the lower left area at the actual location of the obstacle X. This predetermined shape may be a square, circle, trapezoid, cone, or other regular or irregular shape. The illustration uses a square containing a circle as an example. Optionally, the predetermined shape may be filled with a darker color than other locations, such as dark gray in the illustration, to draw the user's attention.

[0224] In an example, Figure 12dAs shown, in order to help users quickly understand the unparkable status of the current parking location, text can also be displayed above the parking area (or other locations). For example, Figure 12d Taking "Obstacle blocked, please adjust the target parking space" as an example, the text can also be any other text content that can indicate the unparking status, and this application does not make specific limitations on this.

[0225] Application Scenario 5

[0226] See also Figure 12e , showing another schematic diagram of marking obstacles in a parking area provided by the present application.

[0227] like Figure 12e As shown, assume that there is an obstacle X located slightly to the lower right of the vehicle's current position. During automatic parking, the user drags the virtual parking space to the right of the vehicle's current position, causing the boundary between the lower left and center left regions of the dragged parking space to be blocked by obstacle X. In this case, the control device 231 determines, based on the detection results of the various onboard sensors 211-213, that the lower left and center left regions of the dragged parking space are occupied by the obstacle. Therefore, the control device 231 may color the lower left and center left regions of the dragged parking space, such as gray as shown. Optionally, a predetermined shape may be marked at the boundary between the lower left and center left regions, and the predetermined shape may be filled with a darker color than other locations to attract the user's attention.

[0228] Application Scenario 6

[0229] See also Figure 12f , showing another schematic diagram of marking obstacles in a parking area provided by the present application.

[0230] like Figure 12f As shown, suppose there is a flower bed to the left of the vehicle's current position and another vehicle to the left front. During automatic parking, the user drags the virtual parking space to the left of the vehicle's current position, causing the entire vehicle body in the dragged parking position to be blocked by the flower bed and other vehicles. In this case, the control device 231 determines, based on the detection results of the various on-board sensors 211-213, that the entire area of the dragged parking position is occupied by obstacles. Therefore, the control device 231 may mark the entire area of the dragged parking position with a color, such as gray as shown. Optionally, a set shape may be marked at the location of the flower bed in the entire area, and the set shape may be filled with a darker color than other locations to attract the user's attention.

[0231] Application Scenario 7

[0232] See also Figure 12g , showing another schematic diagram of marking obstacles in a parking area provided by the present application.

[0233] like Figure 12g As shown, assume that there is an obstacle X located further to the lower right of the vehicle's current position. During automatic parking, the user drags the virtual parking space to the lower right of the vehicle's current position, placing the front half of the vehicle within the parking area and the rear half outside. The rear half is obstructed by obstacle X, but because the obstructed portion is outside the parking area, the obstacle is not displayed within the parking area. In this case, the control device 231 may color the edge of the parking area obstructed by the obstacle, such as gray as shown. Alternatively, to highlight the obstacle's location outside the parking area, the edge may be illuminated with light, such as gray as shown, or other colors, such as red, with the obstacle as the luminous point.

[0234] Application Scenario 8

[0235] Assume the vehicle is in Figure 1 For parking scenarios shown, please refer to Figure 12h , shows another schematic diagram of marking obstacles in a parking area provided by the present application, and the diagram takes an example where there is only a relatively small space on the right side of the vehicle's current position.

[0236] First, suppose that after the user starts the automatic parking function, the display device 222 shows the following Figure 12h In the parking interface shown in (A), there is a button M for customizing parking space function at the lower right corner of the control area on the left side of the parking interface. If the user clicks the button M, a default virtual parking space will appear in the parking area on the right side of the parking interface. The virtual parking space is displayed near the vehicle, for example Figure 12h On the right side of the vehicle shown in (A), there is also a rotation button K displayed above the virtual parking space. Figure 12h In (B), assuming that the user selects a parking position using the custom parking space function, the user can click and hold the virtual parking space and drag it to the free area in the upper right corner, click and hold the rotation button K and rotate it 90° counterclockwise so that the virtual parking space fits exactly into the free area. Optionally, there will be an adsorption button V in the upper right corner of the control area. After the user drags the virtual parking space to the parking position, he can also click the adsorption button V to automatically adsorb the virtual parking space to the parking position. Further, combined with Figure 12hIn (B) and (C), in the actual parking scene, green plants have invaded the left area of the current parking position. However, due to image distortion of the parking area (such as wide IPM distortion), the user mistakenly believes that the currently selected parking position is not blocked by green plants. In this case, the control device 231 determines that the left side of the current parking position is occupied by an obstacle based on the detection results of various on-board sensors 211-213. Therefore, the control device 231 can mark the left area of the current parking position with a color, such as marking Figure 12h At the same time, the upper position of the parking area can also display the unavailable status of the current parking position through text, such as Figure 12h The message "Obstacle blocking the target parking space, please adjust the target parking space" shown in (D) can optionally provide a text prompt of the location of the obstacle, such as "Obstacle blocking the left side of the target parking space, please adjust the target parking space", or other text descriptions, so that the user can pay attention to the result more quickly and assist in changing the parking position.

[0237] According to the above content, by displaying the location of obstacles in the parking area, the user can be intuitively prompted that the currently selected parking location is blocked by an obstacle, so that the user can accurately know the positional relationship between the obstacle and the currently selected parking location, and then assist the user to avoid the obstacle and quickly select a parking location where there are no obstacles, thereby improving the efficiency of the user in selecting a parking location and enhancing the user's parking experience.

[0238] It should be noted that the above content is introduced using the example of a control device in a vehicle executing a parking assistance method, but the parking assistance method can also be executed by other devices or components, such as other components in the vehicle, including but not limited to: on-board terminals, on-board controllers, on-board modules, on-board modules, on-board components, on-board chips, on-board units, on-board radars or on-board cameras and other sensors, or devices or components outside the vehicle, such as cloud servers, terminal devices, RSUs or other vehicles. The controlled vehicle can display obstacles at the current parking position to the user by interacting with these devices or components to assist the user in quickly selecting a parking position where there are no obstacles.

[0239] Furthermore, this parking assistance solution can be extended to any device or system requiring efficient parking. For example, it can be applied to any movable device with parking capabilities, including but not limited to ships, airplanes, high-speed trains, trains, helicopters, lawn mowers, and so on. Alternatively, it can be applied to mobile robots. For example, when an obstacle is detected in the robot's automatically planned parking location, relevant information about the obstacle can be provided to the user, helping the user quickly select a suitable parking location. And so on.

[0240] Furthermore, with the evolution of system architecture and the emergence of new scenarios, the parking assistance method provided in this application is also applicable to similar technical problems, and this application does not make any specific limitations on this.

[0241] Based on the parking assistance method described above, this application can also provide a parking assistance system. Figure 13 , showing a possible architecture diagram of the parking assistance system. The parking assistance system 1300 can be integrated into a vehicle. The parking assistance system 1300 may include a control device 231 and a display device 222 connected to the control device 231. The control device 231 may be any device in the vehicle that can realize the control function, such as an MDC, a VCU or a VDC. The display device 222 may be an on-board display screen, such as an instrument panel, a central control screen, a co-pilot screen, etc., or a projection screen, such as a windshield, a side window or a skylight in a HUD, etc. The control device 231 is used to: generate a parking area based on the surrounding environment of the vehicle and send it to the display device 222; the display device 222 is used to: display the parking area; the control device 231 is also used to: control the display device 222 to display a first parking position based on a first instruction, and mark the position of a first obstacle, where the first obstacle includes an obstacle that interacts with the first parking position.

[0242] In one possible implementation, see Figure 13 Parking assistance system 1300 may also include a sensor system 210, which includes a camera for capturing environmental images. Control device 231 is connected to the camera and, upon detecting that the vehicle has activated the automatic parking function, acquires the environmental image captured by the camera, generates a parking area based on the environmental image, and transmits the image to display device 222.

[0243] For further possible implementations, see Figure 13 Sensor system 210 may also include radar and / or sonar, which are used to collect environmental information and identify the location and / or movement trajectory of obstacles from the environmental information. Control device 231 is connected to the radar and / or sonar and is used to determine obstacles that interact with the first parking position based on the obstacle location and / or movement trajectory identified by the radar and / or sonar.

[0244] In one possible implementation, see Figure 13 The control device 231 is further used to control the vehicle to park in the first parking position when there is no first obstacle.

[0245] For further possible implementations, see Figure 13The parking assistance system 1300 may also include a mechanical steering device 241, a motor controller 242, and a motor 243. The mechanical steering device 241 is connected between the vehicle's steering wheel 251 and wheels 252, and the motor 243 is connected between the motor controller 242 and the wheels 252. The control device 231 is specifically configured to, when the first obstacle is absent, plan a parking trajectory for the vehicle based on the vehicle's current position and the first parking position, determine the next turning angle and speed based on the parking trajectory, and transmit the turning angle to the mechanical steering device 241 and the speed to the motor controller 242. The mechanical steering device 241 is configured to rotate the steering wheel 251 and wheels 252 to corresponding angles based on the turning angle from the control device 231. The rotation of the steering wheel 251 can present steering information to the user, while the rotation of the wheels 252 can cause the vehicle to steer. The motor controller 242 is configured to generate a corresponding drive electrical signal based on the speed from the control device 231 and transmit it to the motor 243. The motor 243 is used to rotate according to the driving electrical signal from the motor controller 242 to drive the wheels 252 to rotate, thereby realizing forward or backward movement of the vehicle.

[0246] It should be noted that the functions of the various components in the parking assistance system 1300, the concepts involved and related to the technical solution provided by this application, the explanation and detailed description and other steps, please refer to the description of these contents in the aforementioned method embodiment, and will not be repeated here.

[0247] Based on the parking assistance method described above, the present application may also provide a parking assistance device, which may be used to execute the above parking assistance method. For related features, please refer to the above method embodiment and will not be repeated here.

[0248] In one possible implementation, see Figure 14 , showing a possible structural diagram of a parking assistance device. The parking assistance device 1400 can be a chip or circuit, such as a chip or circuit that can be set in the vehicle, or a chip or circuit set in an electronic device outside the vehicle. The parking assistance device 1400 can correspond to the control device in the above method, such as the control device 231. The parking assistance device 1400 can implement the above Figure 3 The steps performed by the control device 231 in the method shown above may be implemented as follows Figure 13 The steps performed by the control device 231 in the system are shown.

[0249] like Figure 14As shown, the parking assistance device 1400 may include a control module 1410 and a labeling module 1420. When the parking assistance device 1400 is in operation, the control module 1410 is configured to control the display device to display a parking area, which is generated based on the surrounding environment of the vehicle, and to control the display device to display a first parking position based on a first instruction. The labeling module 1420 is configured to label the position of a first obstacle, which includes obstacles that interact with the first parking position.

[0250] For the concepts, explanations, detailed descriptions, and other steps involved in the parking assistance device 1400 and related to the technical solutions provided in the embodiments of the present application, please refer to the descriptions of these contents in the aforementioned method or other embodiments, and will not be repeated here.

[0251] It should be understood that the above division of units within parking assistance device 1400 is merely a division of logical functions. In actual implementation, these units may be fully or partially integrated into a single physical entity, or physically separated. This application does not impose any specific limitations on this. The functions of each unit within parking assistance device 1400 can be found in the corresponding method embodiments and will not be further elaborated here.

[0252] In one possible implementation, see Figure 15 , shows another possible structural diagram of a parking assistance device. The parking assistance device 1500 can be a chip or a chip system. Optionally, the chip system can be composed of a chip, or can include a chip and other discrete devices. Figure 15 As shown, the parking assistance device 1500 may include at least one processor 1510 and a memory 1520. The at least one processor 1510 is coupled to the memory 1520, which may be located within or outside the parking assistance device 1500. The memory 1520 stores the computer programs or instructions necessary to implement any of the aforementioned method embodiments. The at least one processor 1510 executes the computer programs or instructions stored in the memory 1520 to implement the parking assistance method in any of the aforementioned method embodiments.

[0253] The parking assistance device 1500 may also include a communication interface 1530, through which the parking assistance device 1500 can exchange information with other devices. The communication interface 1530 may be a circuit, a bus, a transceiver, or any other device capable of exchanging information, or may be referred to as a signal transceiver unit. When the parking assistance device 1500 is a chip-type device or circuit, the communication interface 1530 in the parking assistance device 1500 may also be an input / output circuit capable of inputting (or receiving) and outputting (or transmitting) data. The at least one processor 1510 may be an integrated processor, microprocessor, or integrated circuit, and may determine output data based on input data.

[0254] When parking assistance device 1500 is a control device in a vehicle, at least one processor 1510 can access computer programs or instructions stored in memory 1520 to execute the steps of any of the above method embodiments. For example, at least one processor 1510 can communicate with a display device in the vehicle via communication interface 1530 to control the display device to display a parking area, and control the display device to display a first parking position based on a first instruction, where the parking area is generated based on the vehicle's surrounding environment. Furthermore, at least one processor 1510 can communicate with various onboard sensors in the vehicle via communication interface 1530 to mark the position of a first obstacle, where the first obstacle includes an obstacle that interacts with the first parking position.

[0255] When parking assistance device 1500 is a control device external to the vehicle, such as an electronic device external to the vehicle, at least one processor 1510 can access a computer program or instruction stored in memory 1520 to execute the steps of any of the above method embodiments. For example, at least one processor 1510 can communicate with the controlled vehicle via communication interface 1530, causing a display device of the controlled vehicle to display a parking area generated based on the vehicle's surroundings, control the display device to display a first parking position based on a first instruction, and mark the position of a first obstacle, where the first obstacle includes an obstacle that interacts with the first parking position.

[0256] The processor 1510 can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and can implement or execute the methods, steps, and logic block diagrams disclosed herein. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed herein can be directly implemented and executed by a hardware processor, or by a combination of hardware and software modules within the processor.

[0257] The above-mentioned memory 1520 can be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or a volatile memory (volatile memory), such as a random-access memory (RAM). The memory can also be any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this. The memory 1520 in this application can also be a circuit or any other device that can implement a storage function, for storing computer programs, computer programs or instructions and / or data.

[0258] For the concepts, explanations, detailed descriptions and other steps involved in the parking assistance device 1500 and related to the technical solutions provided in the embodiments of the present application, please refer to the descriptions of these contents in the aforementioned method or other embodiments, which are not repeated here.

[0259] Based on the parking assistance method described above, the present application may also provide a vehicle, which may include a unit or module for implementing the above parking assistance method, such as the above Figure 14 or Figure 15 The parking assistance device shown, or may include the above Figure 13 The parking assistance system shown will not be repeated here.

[0260] For example, the vehicle may be a car, truck, motorcycle, bus, recreational vehicle, amusement park vehicle, construction equipment, tram, toy car, golf cart, train, etc., and this application does not impose any particular limitation. In addition, the vehicle may be a new energy vehicle, including an electric vehicle, such as a two-wheel drive electric vehicle or a four-wheel drive electric vehicle, or a fuel vehicle, and this application does not impose any particular limitation.

[0261] Based on the parking assistance method described above, the present application may also provide an electronic device, which may include a unit or module for implementing the above parking assistance method, such as the above Figure 14 or Figure 15 The electronic device is connected to the controlled vehicle and implements the above parking assistance method by communicating with the controlled vehicle.

[0262] Exemplarily, the electronic device may be a cloud server, a terminal device, an RSU or other vehicles, etc., and this application does not make any special limitation.

[0263] Based on the parking assistance method described above, the present application further provides a computer-readable storage medium, which stores a program or instruction. When the program or instruction is executed on the parking assistance device, the parking assistance device executes the above-described method. Figure 3 The method shown.

[0264] Based on the parking assistance method described above, the present application may also provide a computer program product, which includes: computer program code, which, when executed on a computer, enables the computer to execute the above Figure 3 The method shown.

[0265] In this application, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. "At least one" or similar expressions refer to any combination of these, including any combination of single or plural. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple. In addition, in this application, the word "optionally" or "exemplarily" is used to indicate an example, illustration or description. Any embodiment or design scheme described as "example" or "optional" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Alternatively, it can be understood that the use of the word "example" or "optional" is intended to present concepts in a specific way and does not constitute a limitation on this application.

[0266] In addition, the "connection" in this application can be understood as an electrical connection, and the connection between two electrical components can be a direct or indirect connection between the two electrical components. For example, the connection between A and B can be a direct connection between A and B, or an indirect connection between A and B through one or more other electrical components. For example, A and B can be connected, or A and C can be directly connected, and C and B can be directly connected, and A and B are connected through C. In some scenarios, "connection" can also be understood as coupling, such as electromagnetic coupling between two inductors. In short, the connection between A and B enables the transmission of electrical energy and signals between A and B.

[0267] It will be appreciated that the various numerical numbers involved in this application are merely for the purpose of describing the distinctions made, and are not intended to limit the scope of the embodiments of the present application. The size of the sequence numbers of the above-mentioned processes does not imply the order of execution, and the order of execution of each process should be determined by its function and inherent logic. Terms such as "first", "second", and similar expressions are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, comprising a series of steps or units. Methods, systems, products, or devices are not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products, or devices.

Claims

1. A parking assistance method, characterized in that: The method comprises: controlling a display device to display a parking area, wherein the parking area is generated based on a surrounding environment of the vehicle; controlling the display device to display a first parking position based on a first instruction; The position of a first obstacle is marked, where the first obstacle includes an obstacle that interacts with the first parking position.

2. The method according to claim 1, wherein The obstacles that interact with the first parking position include obstacles within the first parking position and / or obstacles outside the first parking position that will interact with the first parking position.

3. The method according to claim 1, wherein The first obstacle is displayed outside the first parking position, and the first obstacle is entirely or partially located within the first parking position. Whether the first obstacle is entirely or partially located within the first parking position is determined by the perception result of the vehicle through the on-board sensor.

4. The method according to any one of claims 1 to 3, characterized in that The first parking position is a parking position automatically generated by the vehicle, and / or the first parking position is a parking position determined by a user.

5. The method according to any one of claims 1 to 4, characterized in that The marking the position of the first obstacle includes: The first parking position is marked as occupied by an obstacle based on environmental perception performed by multiple sensors.

6. The method according to any one of claims 1 to 5, characterized in that The marking the position of the first obstacle includes: Marking the position of the first obstacle closest to the center point of the first parking position; or, marking the outline of the first obstacle; or, The center position of the first obstacle is marked.

7. The method according to any one of claims 1 to 5, characterized in that The marking the position of the first obstacle includes: The first parking position includes a plurality of sub-areas, and each sub-area is marked with an obstacle occupancy mark.

8. The method according to any one of claims 1 to 5, characterized in that The marking the position of the first obstacle includes: The first parking position is partially located outside the parking area, and the first obstacle occupies a portion outside the parking area. The edge of the parking area covered by the first parking position is marked as occupied by the obstacle.

9. The method according to any one of claims 1 to 8, characterized in that The marking the position of the first obstacle includes: The first obstacle is identified, and if the first obstacle has an impact on parking, the first obstacle is marked as occupied; if the first obstacle has no impact on parking, the first obstacle is marked as unoccupied.

10. The method according to claim 9, wherein The first obstacle has an impact on parking, including: The first obstacle satisfies at least one of the following conditions: a height of the first obstacle exceeds a set height, a maximum distance that the first obstacle intrudes into the first parking position is greater than a set distance, and the first obstacle is not in a preset whitelist.

11. The method according to claim 9 or 10, wherein: If the first obstacle has no effect on parking, the method further includes: issuing a first prompt message, where the first prompt message is used to prompt the user to choose whether to ignore the first obstacle; The vehicle is controlled to park in the first parking position based on a second instruction, where the second instruction is an instruction to ignore the first obstacle.

12. The method according to any one of claims 9 to 11, characterized in that The method further comprises: At least two different marking methods are used: occupied marking, unoccupied marking, or occupied marking with different degrees of influence.

13. The method according to any one of claims 1 to 12, characterized in that The first obstacles include dynamic obstacles and static obstacles, and the dynamic obstacles and the static obstacles are marked in different ways.

14. The method according to any one of claims 1 to 13, characterized in that The annotation includes at least one of a human-machine interface HMI annotation, a voice prompt annotation, a direction prompt sound annotation, a light annotation, and a projection annotation.

15. The method according to claim 14, wherein The HMI annotation includes at least one of the following: graphic annotation, text annotation, color annotation, and pattern annotation.

16. The method according to claim 15, wherein The direction prompt sound annotation includes: According to the position of the first obstacle relative to the first parking position, a prompt sound is emitted at a corresponding position relative to the driver in the vehicle.

17. The method according to any one of claims 1 to 16, characterized in that The method further comprises: Based on the position of the first obstacle and the surrounding environment of the vehicle, first recommendation information is provided, where the first recommendation information is used to indicate a recommended parking space adjustment direction and / or a recommended parking space adjustment position.

18. The method according to any one of claims 1 to 17, characterized in that The method further comprises: Marking the edge position of the first obstacle; and / or, A critical safety position is marked, where the critical safety position is a position that is a preset safety distance from an edge position of the first obstacle.

19. The method according to any one of claims 1 to 18, characterized in that The method further comprises: The first parking position is projected into the surroundings of the vehicle, and the first obstacle is marked in the projected first parking position.

20. A parking assistance device, characterized in that: include: a control module configured to control a display device to display a parking area, the parking area being generated based on a surrounding environment of the vehicle, and to control the display device to display a first parking position based on a first instruction; The marking module is used to mark the position of the first obstacle, where the first obstacle includes an obstacle that interacts with the first parking position.

21. The device according to claim 20, characterized in that The obstacles that interact with the first parking position include obstacles within the first parking position and / or obstacles outside the first parking position that will interact with the first parking position.

22. The device according to claim 20 or 21, characterized in that The first obstacle is displayed outside the first parking position, and the first obstacle is entirely or partially located within the first parking position. Whether the first obstacle is entirely or partially located within the first parking position is determined by the perception result of the vehicle through the on-board sensor.

23. The device according to any one of claims 20 to 22, characterized in that The annotation module is specifically used for: The first parking position is marked as occupied by an obstacle based on environmental perception performed by multiple sensors.

24. The device according to any one of claims 20 to 23, characterized in that The annotation module is specifically used for: The first parking position includes a plurality of sub-areas, and each sub-area is marked with an obstacle occupancy mark.

25. The device according to any one of claims 20 to 23, characterized in that The annotation module is specifically used for: The first parking position is partially located outside the parking area, and the first obstacle occupies a portion outside the parking area. The edge of the parking area covered by the first parking position is marked as occupied by the obstacle.

26. The device according to any one of claims 20 to 25, characterized in that The annotation module is specifically used for: The first obstacle is identified, and if the first obstacle has an impact on parking, the first obstacle is marked as occupied; if the first obstacle has no impact on parking, the first obstacle is marked as unoccupied.

27. The device according to any one of claims 20 to 26, characterized in that The annotation includes at least one of a human-machine interface HMI annotation, a voice prompt annotation, a direction prompt sound annotation, a light annotation, and a projection annotation.

28. The device according to any one of claims 20 to 27, characterized in that The annotation module is also used for: Based on the position of the first obstacle and the surrounding environment of the vehicle, first recommendation information is provided, where the first recommendation information is used to indicate a recommended parking space adjustment direction and / or parking space adjustment position.

29. The device according to any one of claims 20 to 28, characterized in that The annotation module is also used for: Marking the edge position of the first obstacle; and / or, A critical safety position is marked, where the critical safety position is a position that is a preset safety distance from an edge position of the first obstacle.

30. A parking assistance device, characterized in that: comprising a processor coupled to a memory; The processor is configured to execute the computer program or instructions stored in the memory, so as to enable the parking assistance device to perform the method according to any one of claims 1 to 19.

31. A vehicle, characterized in that: The method comprises the parking assistance device according to any one of claims 20 to 30.

32. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program or instruction, and when the program or instruction is executed, the method according to any one of claims 1 to 19 is implemented.

33. A computer program product, characterized in that The computer program product comprises a computer program code, which, when run on a computer, causes the computer to perform the method according to any one of claims 1 to 19.

Images