Parking assistance method and apparatus, and vehicle, storage medium and program product

Abstract

A parking assistance method and apparatus, and a vehicle, a storage medium and a program product, which are used for improving the parking efficiency. The method comprises: controlling a display apparatus (222) to display a parking area, wherein the parking area is generated on the basis of surroundings of a vehicle; on the basis of a first instruction, controlling the display apparatus (222) to display a first parking position; and marking the position of a first obstacle, wherein the first obstacle includes an obstacle interacting with the first parking position. The method can enable a user to accurately learn of the position of an obstacle that affects parking, and thus can assist the user in avoiding the obstacle and promptly selecting an available parking position free of the obstacle, thereby reducing the number of instances of the user selecting a parking position and effectively improving the parking efficiency, and thus improving the parking experience of the user.

Description

Parking assistance method, device, vehicle, storage medium and program product

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on February 20, 2024, with application number 202410190053.8 and application name "A parking assistance method, device, vehicle, storage medium and degree product", all of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of parking technology, and in particular to a parking assistance method, device, vehicle, storage medium, and program product. Background Art

[0004] Parking refers to a technology that drives a vehicle into a parking space under low-speed driving scenarios, including fully automatic parking, semi-automatic parking and manual parking.

[0005] Compared to semi-automatic and manual parking, fully automated parking requires only user input via the vehicle's display, mobile phone, or remote control. The vehicle automatically parks in a pre-planned or user-designated parking space, eliminating the need for manual operation or even the need for the user to be in the vehicle. Fully automated parking offers a more intelligent and convenient parking experience and is becoming a standard feature in vehicles.

[0006] However, current mainstream fully automated parking solutions suffer from low parking efficiency. Especially in scenarios where users are required to specify a parking space, they often have to go through multiple selections before finally finding a suitable spot. This significantly reduces parking efficiency and hinders the user's parking experience. Therefore, improving parking efficiency is a pressing technical challenge in the automated parking field. Summary of the Invention

[0007] The present application provides a parking assistance method, device, vehicle, storage medium, and program product to improve parking efficiency.

[0008] In a first aspect, the present application provides a parking assistance method, which includes: controlling a display device to display a parking area, where the parking area is generated based on the surrounding environment of the vehicle; controlling the display device to display a first parking position based on a first instruction; and marking the position of a first obstacle, where the first obstacle includes an obstacle that interacts with the first parking position.

[0009] By using the above method, by marking obstacles that interact with the first parking position, the user can accurately know the location of the obstacle that affects parking, and then assist the user to avoid the obstacle and select a parking position without the obstacle as soon as possible, reducing the number of times the user selects a parking position, effectively improving parking efficiency, and thus improving the user's parking experience.

[0010] In a possible design, the obstacles that interact with the first parking position include obstacles within the first parking position and / or obstacles outside the first parking position that will interact with the first parking position.

[0011] The above design can mark static obstacles and / or dynamic obstacles that affect the current parking situation. By marking static obstacles, users can be assisted in quickly avoiding obstacles in the first parking position. By issuing early warnings for dynamic obstacles, users can be assisted in avoiding obstacles that may appear in the first parking position in the future. This prevents users from being interrupted by dynamic obstacles in the middle of parking and having to choose a parking position again, thereby maintaining efficient parking efficiency.

[0012] In one possible design, the first obstacle is displayed outside the first parking position, and the first obstacle is entirely or partially located within the first parking position. Whether the first obstacle is entirely or partially located within the first parking position is determined by the perception results of the vehicle through the on-board sensors.

[0013] The above design can also mark obstacles that are not displayed in the first parking position due to image distortion or other reasons. This allows the user to accurately know the actual location of the obstacle even if the user cannot see the obstacle in the parking area or sees the wrong obstacle location, thereby helping the user to select a parking space as quickly as possible.

[0014] In a possible design, the first parking position may be a parking position automatically generated by the vehicle, and / or the first parking position may be a parking position determined by a user.

[0015] By adopting the above design, the parking assistance method can be compatible with different parking space selection functions in vehicles, or vehicles with different parking space selection functions, thereby improving the versatility of the parking assistance method.

[0016] In one possible design, marking the position of the first obstacle includes: performing environmental perception based on multiple sensors, and marking the first parking position as occupied by the obstacle.

[0017] The above design can intuitively present the occupancy status of the obstacle itself to the user, allowing the user to accurately know the location of the obstacle.

[0018] In a further possible design, the position of the first obstacle is marked, including at least one of the following three marking contents:

[0019] Annotation content 1: Mark the position of the first obstacle closest to the center point of the first parking space, so that the user can quickly locate the point where the first obstacle intrudes the deepest into the first parking space and understand the extent to which the first obstacle occupies the first parking space;

[0020] The second annotation content is to mark the outline of the first obstacle so that the user can accurately know the area occupied by the first obstacle in the first parking position, so that the user can quickly select an available parking position outside the area;

[0021] The third annotation content is to mark the center position of the first obstacle so that the user can quickly locate the center point of the first obstacle, and to facilitate the user to select a parking position in a radial manner based on the center point.

[0022] In a possible design, marking the position of the first obstacle includes: the first parking position includes multiple sub-areas, and obstacle occupancy is marked for each sub-area.

[0023] The above design allows the sub-area to which the obstacle belongs to be marked. Even if the actual obstacle position deviates from the detected obstacle position due to some reasons (such as detection error), the actual obstacle location can be included in the sub-area to which it belongs and marked synchronously, increasing the probability that the user can adjust to a possible parking position based on the marking.

[0024] In one possible design, marking the position of the first obstacle includes: when the first parking position is partially outside the parking area and the first obstacle occupies the part outside the parking area, marking the edge of the parking area covered by the first parking position as an obstacle occupancy mark.

[0025] With the above design, the user can know the positional relationship between the obstacles outside the parking area and the first parking position, thereby helping the user to avoid the location of the obstacles outside the parking area and select a parking position as quickly as possible.

[0026] In one possible design, marking the position of the first obstacle includes: identifying the first obstacle, and if the first obstacle has an impact on parking, marking the first obstacle as occupied; if the first obstacle has no impact on parking, marking the first obstacle as unoccupied, or not marking the first obstacle.

[0027] The above design can distinguish and mark obstacles that affect parking and those that do not, making it easier for users to understand the type of obstacle so that they can respond more specifically.

[0028] In a further possible design, the first obstacle is considered to have an impact on parking when it meets at least one of the following conditions: the height of the first obstacle exceeds a set height, the maximum distance that the first obstacle intrudes into the first parking position is greater than a set distance, and the first obstacle is not in a preset whitelist.

[0029] The above design covers all kinds of obstacles that affect parking in actual scenarios by identifying obstacles that are high enough to scratch the vehicle chassis, obstacles that intrude deeply into the parking space, and obstacles outside the set whitelist as obstacles that affect parking.

[0030] In a further possible design, if the first obstacle has no effect on parking, a first prompt message may be issued, which is used to prompt the user to choose whether to ignore the first obstacle. Afterwards, the vehicle is controlled to park in the first parking position based on a second instruction, and the second instruction is an instruction to ignore the first obstacle.

[0031] Optionally, the first prompt information may be a voice prompt, a display prompt, a flashing light prompt, a vibration prompt, etc. For example, the display device may be controlled to display text information, prompting the user to click a corresponding button on the display device, or to respond to a corresponding instruction by voice.

[0032] With the above design, in scenarios where the impact of obstacles on parking is negligible, additional prompt information can be issued to facilitate users to quickly make parking choices and assist users in completing parking as quickly as possible.

[0033] A further possible design is to use at least two different marking methods: occupied marking, unoccupied marking, or marking different degrees of occupied marking. For example, obstacles with no impact on parking can be marked green, obstacles with a significant impact on parking can be marked red, and obstacles with uncertain impact on parking can be marked yellow.

[0034] The above design distinguishes and labels obstacles of different categories, and can more intuitively present the category of the current obstacle to the user, making it easier for the user to respond quickly to the current obstacle in a targeted manner.

[0035] In a possible design, the first obstacle may include a dynamic obstacle and a static obstacle, and the dynamic obstacle and the static obstacle are marked in different ways.

[0036] The above design distinguishes and labels dynamic obstacles from static obstacles, allowing users to more intuitively understand the types of obstacles currently affecting parking.

[0037] In one possible design, the annotation includes at least one of a human machine interface (HMI) annotation, a voice prompt annotation, a direction prompt sound annotation, a light annotation, and a projection annotation. Optionally, the HMI annotation includes at least one of the following: a graphic annotation, a text annotation, a color annotation, and a pattern annotation.

[0038] The above design supports multiple marking methods, which can realize diversified options of assisted parking and meet the personalized needs of users.

[0039] In a further possible design, the direction warning sound marking includes: issuing a warning sound at a corresponding direction relative to the driver in the vehicle according to the position of the first obstacle relative to the first parking position.

[0040] With the above design, a prompt sound can be emitted at the position of the obstacle relative to the vehicle, drawing the user's attention to the pronunciation location, allowing the user to more intuitively grasp the position where the obstacle will affect parking, thereby improving the user's parking experience.

[0041] In one possible design, the method further includes: providing first recommendation information based on the position of the first obstacle and the surrounding environment of the vehicle, where the first recommendation information is used to indicate a recommended parking space adjustment direction and / or a recommended parking space adjustment position.

[0042] Optionally, the first recommendation information may be a voice recommendation, a display recommendation, a headlight flashing recommendation, etc. For example, the display device may be controlled to display the drag position or drag direction, so that the user can adjust the parking position.

[0043] The above design can assist users in quickly adjusting their parking positions and speeding up parking by sending additional recommendation information to users.

[0044] In one possible design, the method further includes: marking an edge position of the first obstacle; and / or marking a critical safety position, where the critical safety position is a position at a preset safety distance from the edge position of the first obstacle.

[0045] The above design allows users to more intuitively see the edge position or critical safety position of obstacles, thereby facilitating users to quickly determine the safe area and speed up the selection of a suitable parking location.

[0046] In one possible design, the method further includes: projecting a first parking position in the surroundings of the vehicle, and marking a first obstacle in the projected first parking position.

[0047] The above design can provide an early warning of obstacles in the first parking position. If the obstacle is a dynamic obstacle, a signal can be given to the dynamic obstacle to make it leave the first parking position as soon as possible, assisting the user to complete parking.

[0048] In a second aspect, the present application provides a parking assistance device, which can be a control device inside the vehicle or a control device outside the vehicle. The control device inside the vehicle may include, but is not limited to: a vehicle control unit (VCU) or a vehicle domain controller (VDC) for realizing vehicle control, an intelligent driving domain control unit or a mobile data center (MDC) for realizing intelligent driving or assisted driving, an electronic control unit (ECU) or a motor control unit (MCU) for realizing vehicle component control, or an intelligent brake system (IBS) for realizing vehicle driving mode control. The control device outside the vehicle may be, for example, a cloud server, a terminal device, a road side unit (RSU) or other vehicles. The terminal device can be understood as a device of a user riding in the controlled vehicle, such as, but not limited to, a mobile phone, a tablet computer, a laptop computer, a smart watch, a Bluetooth headset and other wearable devices. These control devices can be connected to relevant equipment in the controlled vehicle through the network, such as various sensors and display devices, to achieve parking assistance operations for the controlled vehicle.

[0049] The parking assistance device includes: a control module for controlling a display device to display a parking area, where the parking area is generated based on the surrounding environment of the vehicle, and controlling the display device to display a first parking position based on a first instruction; and a marking module for marking the position of a first obstacle, where the first obstacle includes an obstacle that interacts with the first parking position.

[0050] In a possible design, the obstacles that interact with the first parking position include obstacles within the first parking position and / or obstacles outside the first parking position that will interact with the first parking position.

[0051] In one possible design, the first obstacle is displayed outside the first parking position, and the first obstacle is entirely or partially located within the first parking position. Whether the first obstacle is entirely or partially located within the first parking position is determined by the perception results of the vehicle through the on-board sensors.

[0052] In a possible design, the first parking position is a parking position automatically generated by the vehicle, and / or the first parking position is a parking position determined by a user.

[0053] In one possible design, the labeling module is specifically used to: perform environmental perception based on multiple sensors and label the first parking position as occupied by an obstacle.

[0054] In a further possible design, the marking module is specifically used to: mark the position of the first obstacle closest to the center point of the first parking position; or, mark the outline of the first obstacle; or, mark the center position of the first obstacle.

[0055] In a possible design, the marking module is specifically configured to: the first parking position includes multiple sub-areas, and mark each sub-area for obstacle occupancy.

[0056] In one possible design, the marking module is specifically configured to: when the first parking position is partially outside the parking area and the first obstacle occupies the portion outside the parking area, mark the edge of the parking area covered by the first parking position as being occupied by the obstacle.

[0057] In one possible design, the marking module is specifically used to: identify the first obstacle, and if the first obstacle affects parking, mark the first obstacle as occupied; if the first obstacle has no effect on parking, mark the first obstacle as unoccupied.

[0058] In a further possible design, the marking module is specifically used to: if it is determined that the first obstacle meets at least one of the following conditions, then it is determined that the first obstacle has an impact on parking: the height of the first obstacle exceeds the set height, the maximum distance that the first obstacle intrudes into the first parking position is greater than the set distance, and the first obstacle is not in the preset whitelist.

[0059] In a further possible design, the marking module is also used to: if the first obstacle has no effect on parking, issue a first prompt message, and the first prompt message is used to prompt the user to choose whether to ignore the first obstacle; the control module is also used to: control the vehicle to park in the first parking position based on a second instruction, and the second instruction is an instruction to ignore the first obstacle.

[0060] In a further possible design, at least two different marking methods are adopted, including occupied marking, unoccupied marking, or occupied marking with different degrees of influence.

[0061] In a possible design, the first obstacle includes a dynamic obstacle and a static obstacle, and the dynamic obstacle and the static obstacle are marked in different ways.

[0062] In one possible design, the annotation includes at least one of a human-machine interface HMI annotation, a voice prompt annotation, a direction prompt sound annotation, a light annotation, and a projection annotation.

[0063] In a further possible design, the HMI annotation includes at least one of the following: graphic annotation, text annotation, color annotation, and pattern annotation.

[0064] In a further possible design, when using direction prompt sound annotation, the annotation module is specifically used to: emit a prompt sound at a corresponding direction relative to the driver in the vehicle according to the position of the first obstacle relative to the first parking position.

[0065] In one possible design, the labeling module is further used to: provide first recommendation information based on the position of the first obstacle and the surrounding environment of the vehicle, where the first recommendation information is used to indicate a recommended parking space adjustment direction and / or a recommended parking space adjustment position.

[0066] In one possible design, the marking module is further used to: mark the edge position of the first obstacle; and / or mark the critical safety position, where the critical safety position is a position at a preset safety distance from the edge position of the first obstacle.

[0067] In one possible design, the marking module is further configured to project the first parking position in the surroundings of the vehicle and mark the first obstacle in the first parking position.

[0068] In a third aspect, the present application provides a parking assistance system that can be integrated into a vehicle. The parking assistance system includes a control device and a display device. The control device is configured to generate a parking area based on the vehicle's surroundings and transmit the generated information to the display device. The display device is configured to display the parking area. The control device is further configured to control the display device to display a first parking position and mark the position of a first obstacle based on a first instruction. The first obstacle includes an obstacle that interacts with the first parking position.

[0069] In one possible design, the display device may include an in-vehicle display screen, such as an instrument panel, a central control screen, or a co-pilot screen, or may include an in-vehicle projection screen, such as a windshield or a skylight.

[0070] In a possible design, the parking assistance system may further include a camera; the camera is used to: capture an image of the environment; and the control device is further used to: generate a parking area based on the image of the environment captured by the camera.

[0071] In one possible design, the parking assistance system may also include on-board sensors, which may include one or more of cameras, radars, and sonars; the on-board sensors are used to: collect environmental information and identify obstacle positions and / or motion trajectories from the environmental information; the control device is used to: determine obstacles that interact with the first parking position based on the obstacle positions and / or motion trajectories identified by various on-board sensors.

[0072] In a possible design, the control device is further configured to control the vehicle to park in the first parking position when the first obstacle is not present.

[0073] In a further possible design, the parking assistance system may also include a mechanical steering gear, a motor controller and a motor, the mechanical steering gear being connected between the steering wheel and the wheels, and the motor being connected between the motor controller and the wheels; the control device being specifically used to: when there is no first obstacle, plan the parking trajectory of the vehicle according to the current position of the vehicle and the first parking position, determine the next turning angle and speed according to the parking trajectory, send the turning angle to the mechanical steering gear, and send the speed to the motor controller; the mechanical steering gear being used to: drive the wheels to rotate a corresponding angle according to the turning angle from the control device; the motor controller being used to: generate a corresponding driving electrical signal according to the speed from the control device and send it to the motor; the motor being used to: rotate according to the received driving electrical signal to drive the wheels to rotate.

[0074] It should be noted that the various designs in the above-mentioned first aspect are also applicable to the third aspect, and will not be repeated here one by one.

[0075] In a fourth aspect, the present application provides a parking assistance device, comprising a processor coupled to a memory, the processor being configured to execute a computer program or instruction stored in the memory, so that the parking assistance device performs a parking assistance method as described in the first aspect or any one of the designs of the first aspect.

[0076] In a fifth aspect, the present application provides a vehicle, comprising a parking assistance system as in the third aspect or any one of the designs of the third aspect, or comprising a unit or module for implementing the parking assistance method as in the first aspect or any one of the designs of the first aspect, for example, may comprise a parking assistance device as in the second aspect or any one of the designs of the second aspect, or may comprise a parking assistance device as in the fourth aspect.

[0077] In a sixth aspect, the present application provides an electronic device connected to a controlled vehicle for communicating with the controlled vehicle to implement the parking assistance method described in the first aspect or any one of the embodiments of the first aspect. The electronic device may include a unit or module for implementing the parking assistance method described in the first aspect or any one of the embodiments of the first aspect, for example, the parking assistance device described in the second aspect or any one of the embodiments of the second aspect, or the parking assistance device described in the fourth aspect.

[0078] In a seventh aspect, the present application provides a computer-readable storage medium, which stores a program or instruction. When the program or instruction is executed, it implements the parking assistance method as in the first aspect or any one of the designs of the first aspect.

[0079] In an eighth aspect, the present application provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer executes the parking assistance method as in the first aspect or any one of the designs of the first aspect.

[0080] The technical effects that can be achieved in the above-mentioned second to eighth aspects can refer to the description of the beneficial effects in the above-mentioned first aspect, and will not be repeated here one by one. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] FIG1 exemplarily shows a schematic diagram of a possible application scenario provided by the present application;

[0082] FIG2 exemplarily shows a schematic diagram of the architecture of a parking assistance system provided by the present application;

[0083] FIG3 exemplarily shows a flow chart of a parking assistance method provided by the present application;

[0084] FIG4a exemplarily shows a parking area diagram from a frontal perspective provided by the present application;

[0085] FIG4 b exemplarily shows a parking area diagram from a top-down perspective provided by the present application;

[0086] FIG5a exemplarily shows a parking area diagram of a custom parking space provided in the present application;

[0087] FIG5 b exemplarily shows a parking area diagram of a drag parking position provided by the present application;

[0088] FIG6a exemplarily shows a schematic diagram of a parking area for automatically generating parking spaces provided by the present application;

[0089] FIG6 b exemplarily shows a schematic diagram of a parking area for selecting automatic generation of parking spaces provided by the present application;

[0090] FIG7a exemplarily shows a schematic diagram of marking the center position of an obstacle provided by the present application;

[0091] FIG7b exemplarily shows a schematic diagram of marking the outline of an obstacle provided by the present application;

[0092] FIG7c exemplarily shows a schematic diagram of marking the farthest intrusion point of an obstacle provided by the present application;

[0093] FIG8 exemplarily shows a diagram of an area marking method for a first parking position provided by the present application;

[0094] FIG9a exemplarily shows a schematic diagram of marking obstacles with symbols provided by the present application;

[0095] FIG9b exemplarily shows a schematic diagram of marking obstacles with text provided by the present application;

[0096] FIG9c exemplarily shows a schematic diagram of marking obstacles by color provided by the present application;

[0097] FIG9d exemplarily shows a schematic diagram of marking obstacles with patterns provided by the present application;

[0098] FIG9e exemplarily shows a schematic diagram of marking obstacles with lines provided by the present application;

[0099] FIG10a exemplarily shows a schematic diagram of a scenario provided by the present application in which an obstacle is located outside a parking area;

[0100] FIG10b exemplarily shows a schematic diagram of marking obstacles outside a parking area provided by the present application;

[0101] FIG10c exemplarily shows another schematic diagram of marking obstacles outside a parking area provided by the present application;

[0102] FIG11a exemplarily shows a parking area diagram of a recommended dragging position and dragging direction provided by the present application;

[0103] FIG11b exemplarily shows a schematic diagram of a parking area provided by the present application, with obstacle edge positions and critical safety positions marked;

[0104] FIG12a exemplarily shows a schematic diagram of marking obstacles in a horizontal parking space provided by the present application;

[0105] FIG12b exemplarily shows a schematic diagram of marking obstacles in a vertical parking space provided by the present application;

[0106] FIG12c exemplarily shows another schematic diagram of marking obstacles in a vertical parking space provided by the present application;

[0107] FIG12d exemplarily shows a schematic diagram of marking obstacles in a parking area provided by the present application;

[0108] FIG12e exemplarily shows another schematic diagram of marking obstacles in a parking area provided by the present application;

[0109] FIG12f exemplarily shows another schematic diagram of marking obstacles in a parking area provided by the present application;

[0110] FIG12g exemplarily shows another schematic diagram of marking obstacles in a parking area provided by the present application;

[0111] FIG12h exemplarily shows another schematic diagram of marking obstacles in a parking area provided by the present application;

[0112] FIG13 exemplarily shows a schematic diagram of the architecture of a parking assistance system provided by the present application;

[0113] FIG14 exemplarily shows a schematic structural diagram of a parking assistance device provided by the present application;

[0114] FIG15 exemplarily shows a schematic structural diagram of another parking assistance device provided in the present application. DETAILED DESCRIPTION

[0115] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0116] The following is an explanation of some of the terms used in this application. It should be noted that these explanations are for the purpose of facilitating understanding by those skilled in the art and do not limit the scope of protection claimed in this application.

[0117] 1. Image distortion in parking areas

[0118] When capturing images, the ideal camera position is perpendicular to the shooting plane, ensuring that the image is reproduced in its original geometric proportions. However, in the practical application of autonomous vehicles, due to the limitations of the vehicle structure, the camera's installation position will have a certain preview distance. The camera's horizontal and vertical scanning planes expand in a fan-shaped pattern, and the camera is at a certain angle to the ground. This angle will cause certain distortions in the captured image, mainly radial distortion and tangential distortion.

[0119] Due to these distortions, the displayed image of the parking area may be misaligned or inaccurate. For example, a vehicle's sensors may detect an obstacle at a certain location, but the display may not show the obstacle at that location. This can cause the user to mistakenly select a parking spot where the obstacle is located, causing inconvenience.

[0120] 2. Horizontal parking spaces and vertical parking spaces

[0121] Horizontal parking spaces, also known as parallel parking spaces, occupy a relatively large area and are more convenient for parking, generally suitable for roadside parking. Perpendicular parking spaces save space but require backing up when parking or driving, and are generally suitable for indoor and outdoor parking lots. Generally, within a parking area, if the long side of the space is open, it is considered a horizontal space. If the short side of the space is open, it is considered a perpendicular space.

[0122] The previous text introduced some of the terms involved in this application. The following text introduces the possible application scenarios of this application.

[0123] Please refer to Figure 1, which illustrates a possible application scenario of the present application. This application scenario takes the application of a parking assistance solution to a vehicle as an example, and the vehicle has an automatic parking function. As shown in Figure 1, assume that the vehicle to be parked is vehicle A. There is an empty area on the right side of vehicle A, and both sides of the empty area are covered with green plants. In the current parking scenario, if the user turns on the automatic parking function, vehicle A can automatically collect the vehicle's surrounding environment and generate a parking area based on the vehicle's surrounding environment. The parking area is then presented to the user through a display device and waits for the user to select a parking location. Afterwards, if the parking location selected by the user is not occupied, the vehicle can plan a driving path based on the current location and the parking location, and can automatically park in the parking location according to the driving path. Conversely, if the parking location selected by the user is occupied, the vehicle can instruct the user to reselect a parking location until the user selects an available parking location, and then control the vehicle to automatically park in the available parking location.

[0124] Exemplarily, the above-mentioned vehicles can be pure electric vehicles (pure electric vehicle / battery electric vehicle, pure EV / battery EV), hybrid electric vehicles (hybrid electric vehicle, HEV), range extended electric vehicles (range extended electric vehicle, REEV), plug-in hybrid electric vehicles (plug-in hybrid electric vehicle, PHEV), or other new energy vehicles (new energy vehicle, NEV), or they can also be fuel vehicles, etc. These vehicles can be used in fields such as unmanned driving, assisted driving, intelligent driving, automatic driving or connected vehicles.

[0125] It should be understood that the above application scenarios are merely examples, and the parking assistance solution provided in this application can also be applied in other possible scenarios, not limited to the ones exemplified above. For example, the parking assistance solution can also be applied to other modes of transportation, such as ships, airplanes, trains, subways, high-speed trains, submarines, rockets, or spacecraft, as an auxiliary control solution for achieving efficient parking of other vehicles, thereby reducing unnecessary parking delays. For another example, the parking assistance solution can also be applied in the medical field, such as in smart wheelchairs with display screens. When obstacles are present in the automatically planned parking location, the location of the obstacles can be displayed to the patient in the wheelchair, assisting the patient in finding a safe parking spot. For another example, the parking assistance solution can also be applied in the smart home field, such as robot vacuums, robot mops, autonomous food delivery robots, or mobile smart appliances. These devices can interact with user terminals to assist users in remotely indicating a safe parking location. For another example, the parking assistance solution can also be applied in smart living scenarios, such as in automatically following suitcases, smart dining chairs, or smart mobility devices, etc. I will not list them all here.

[0126] It should be noted that the application scenarios described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application.

[0127] As described in the background, existing automated parking solutions suffer from low parking efficiency. This problem is particularly pronounced in automated parking solutions that utilize a custom parking feature (also known as custom parking, custom parking space parking, etc.) to select a parking location. The custom parking feature is a commonly used parking space selection feature that allows users to select or drag their vehicle to a desired parking location within a parking area, satisfying their personalized needs. However, as noted in the aforementioned terminology explanation, the displayed image of the parking area often exhibits some distortion, which can cause the image to appear different from the actual environment. For example, an obstacle may be present at a certain location in the actual environment, detected by the vehicle, but not displayed at that location in the parking area, or the displayed location may deviate from the actual location, or the displayed size may deviate from the actual size. Due to these discrepancies, users who select a parking location based solely on the displayed image of the parking area may mistakenly select a location with an obstacle, rendering the currently selected parking location unavailable.

[0128] When a parking spot is unavailable, the current parking solutions offered by the industry only display the unavailable status in the parking area, such as displaying the text "Current location is unavailable." However, from the user's perspective, the currently selected location in the parking area is not blocked by obstacles. Even if the user is informed of the unavailable status, the user does not know the specific location or size of the obstacle. This results in the user still being able to select a location occupied by an obstacle when selecting the next parking spot, and the user may need to repeatedly select multiple times before finally selecting a suitable parking spot. This method of requiring repeated selections obviously reduces the efficiency of automatic parking and is not conducive to improving the user's automatic parking experience.

[0129] In view of this, the present application provides a parking assistance method. When there is an obstacle at the currently selected parking location, by marking the obstacle, the user can accurately know the location of the obstacle, and then assist the user to avoid the obstacle and select a parking location without obstacles as soon as possible, so as to improve the efficiency of automatic parking and enhance the user's parking experience.

[0130] The parking assistance solution proposed in this application is described in detail below with reference to specific drawings.

[0131] In the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0132] To facilitate understanding of this application, a system for implementing a parking assistance function is first provided. FIG2 shows a schematic diagram of a possible architecture of this system. Functionally, this architecture may include three major systems: a sensor system 210 , a human-computer interaction system 220 , and a control system 230 .

[0133] The sensor system 210 may include various on-board sensors for detecting the vehicle's surrounding environment, such as a camera 211 (e.g., a four-way fisheye camera) and a radar 212 (e.g., an ultrasonic radar, such as a 12-way ultrasonic radar). The camera 211 is typically mounted near the license plate frame and license plate light, inside the bumper, on the door, or at the bottom of the rearview mirror, and is primarily responsible for capturing images of the environment surrounding the vehicle. The radar 212 is typically mounted on the roof, near the bumper, on the lights, on the doors, and in the rearview mirror, and is primarily responsible for capturing information about the distance between obstacles around the vehicle and the vehicle. In some embodiments, the sensor system 210 may also include a sonar 213, which is typically mounted at the front or rear of the vehicle and is primarily responsible for capturing the distance between obstacles in front of or behind the vehicle (e.g., vehicles, pedestrians, or other obstacles) and the front or rear of the vehicle to prevent collisions.

[0134] The human-computer interaction system 220, which may also be referred to as an HMI system, may include various devices for realizing human-computer interaction, such as an image processor 221 and a display device 222. The image processor 221 is connected to the camera 211 and is mainly responsible for processing the environmental image captured by the camera 211 to complete operations such as parking space recognition, obstacle recognition, and passable area recognition. In some scenarios, the image processor 221 may also perform anti-distortion processing on the environmental image, such as inverse perspective mapping (IPM). The display device 222 may include any type of screen that can realize a display function, such as an instrument panel, a navigation screen, a central control screen, or a co-pilot screen, or other vehicle-mounted display screens, such as a skylight or side window, or a windshield in a head-up display (HUD).

[0135] Control system 230 may include various devices for controlling the entire vehicle and its components. For example, it may include a control device 231. This control device 231 is primarily responsible for module scheduling, receiving radar data, image processing data, and sonar data, performing data information fusion, determining planning schemes, and generating and outputting planning instructions. Control device 231 may be any control device within the vehicle, including, but not limited to, devices for controlling the entire vehicle, such as a VCU or VDC; devices for intelligent driving or assisted driving, such as an intelligent driving domain control unit or MDC; devices for controlling vehicle components, such as an ECU or MCU; and devices for controlling the vehicle's driving mode, such as an IBS.

[0136] For example, in a specific example, after the vehicle is started, the vehicle's default control device 231 is VCU or VDC. However, when the vehicle starts the automatic parking function, MDC will take over the vehicle control, so the control device 231 becomes MDC. After the automatic parking is completed, VCU or VDC will take over the vehicle again, so that the control device 231 becomes VCU or VDC again. Therefore, in this example, when the user turns on the automatic parking function, the control device 231 can be considered to be MDC. However, it should be understood that in other examples, the automatic parking function can also be encapsulated in other control devices, such as VCU or VDC. In this case, the control device 231 during the automatic parking process can also be VCU or VDC, and this application does not make specific limitations on this.

[0137] It should be understood that the above parking assistance system architecture is only an example. In other examples, the parking assistance system may also include more, fewer, or different components, each component may also include more, fewer, or different parts, and the parts shown or not shown may also be combined or divided in any manner. The division of the units in the figure is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into one physical entity, or they may be physically separated. This application does not make specific limitations on this.

[0138] Please refer to Figure 3, which shows a flow chart of a parking assistance method provided by the present application, which is applicable to a parking assistance device. The parking assistance device can be a control device in the vehicle, such as the control device 231 shown in Figure 2. It can also be a control device outside the vehicle, such as a cloud server, terminal device, RSU or other vehicle. Among them, the terminal device can be understood as a device of a user riding in the controlled vehicle, such as but not limited to a mobile phone, tablet computer, laptop computer, smart watch, Bluetooth headset and other wearable devices. The control device outside the vehicle can be connected to the relevant components in the controlled vehicle through the network, such as the various types of vehicle-mounted sensors 211 to 213, image processor 221 and display device 222 shown in Figure 2, to implement a parking assistance solution for the controlled vehicle.

[0139] For example, taking the parking assistance device as the control device 231 shown in FIG2 as an example, as shown in FIG3 , the method includes the following steps:

[0140] Step 301 : Control a display device to display a parking area, where the parking area is generated based on the surrounding environment of the vehicle.

[0141] Optionally, during the driving process of the vehicle, if the control device 231 receives an automatic parking request triggered by the user, it can collect environmental information based on various vehicle-mounted sensors 211 to 213 to determine the vehicle information and the surrounding environment information of the vehicle, and then generate a parking area based on this information, and send the parking area to the display device 222 to display it to the user through the display device 222.

[0142] For example, the control device 231 may obtain an environmental image captured by the camera 211 and determine the vehicle's position, shape, and surrounding environment based on the environmental image. It then generates a parking area based on the pre-configured vehicle image and sends it to the display device 222. In some scenarios, before sending the parking area to the display device 222, the control device 231 may also identify obstacle information around the vehicle (such as the distance, location, and shape of the obstacle) based on radar data collected by the radar 212 and sonar data collected by the sonar 213. These obstacles are then added to the generated parking area to provide a more detailed and accurate image of the parking area.

[0143] Furthermore, the parking area can optionally be divided into multiple perspectives. For example, see Figures 4a and 4b, which illustrate two possible perspectives of the parking area provided herein. Figure 4a shows a front view, and Figure 4b shows a top view. In one example, after the automatic parking function is enabled, the control device 231 may control the display device 222 to display the parking area in the front view shown in Figure 4a by default. This parking area presents an image of the environment in front of the vehicle. To switch to another perspective, the user can click a specific location within the parking area to switch to another perspective. For example, clicking above the parking area shown in Figure 4a switches to the top view shown in Figure 4b. In addition to presenting an image of the environment, this top view may also display an image of the vehicle itself. Compared to the front view, this view better reflects the vehicle's position within the surrounding environment. Therefore, the top view parking area is typically used as a reference interface for selecting a parking location. It is understood that actual vehicles may also include other perspectives, such as rear, left, and right views, and this application does not specifically limit this.

[0144] Taking the top-down perspective shown in Figure 4b as an example, it should be noted that the figure shows a complete parking interface, which can be divided into two parts, left and right. The left part is called the control area, and the right part is called the parking area. The left control area contains several control buttons. By clicking these control buttons, the user controls the display device 222 to display corresponding content in the right parking area. See the following description for details. It should be understood that the parking interface shown in Figures 4a and 4b is merely an example, and this application does not limit the specific presentation format of the parking interface. For example, in another possible example, the parking interface can be presented as a single entity, or divided into upper and lower parts, or at least three parts. For another example, the position, function, and form of the various elements in the parking interface may differ from those in Figures 4a and 4b. For example, the function buttons may be replaced with text or icons, and the position of the function buttons may be changed, etc. This application does not specifically limit this.

[0145] Step 302: Control a display device to display a first parking position based on a first instruction.

[0146] Alternatively, the first instruction may be triggered by a user performing a certain operation, or may be automatically generated by the vehicle after the automatic parking feature is enabled. For example, the first instruction may be triggered by a user dragging or selecting a parking area. In this case, the first parking position is the parking position determined by the user, such as the parking position dragged or selected by the user. For another example, the first instruction may be automatically generated by the vehicle after detecting that the automatic parking feature has been enabled. In this case, the first parking position is the parking position automatically generated by the vehicle.

[0147] It should be noted that the first parking position can be the first parking position dragged by the user, or the first selection by the user, or the first parking position automatically generated by the vehicle after the automatic parking function is started. It can also be the parking position dragged again by the user, or reselected by the user, or regenerated by the vehicle after the parking position dragged by the user, or the last selection, or the automatically generated parking position cannot be parked. There is no specific limitation.

[0148] For example, taking the parking location that the user first drags as an example, refer to the parking area shown in Figure 4b. A button M for a custom parking space function may also be located in the left control area. When the control device 231 detects that the user has clicked this button M, it may control the display device 222 to generate and display a virtual parking space in the right parking area, as shown in Figure 5a. The virtual parking space can be generated around the vehicle, such as in any one or more of the vehicle's left, right, top, or bottom positions. The virtual parking space itself can be dragged, and a rotation button K may be displayed around it. The user can change the position of the virtual parking space by clicking and holding the virtual parking space and dragging it. The user can change the position of the virtual parking space by clicking and holding the rotation button K and rotating it. The user's dragging and rotating operations trigger a first instruction. Based on this first instruction, the control device 231 controls the display device 222 to display the virtual parking space at the position where the user's dragging and rotating operations are completed, i.e., the first parking location. For example, referring to FIG5b , it is assumed that the control device 231 moves the virtual parking space to the left front of the actual vehicle according to the first instruction triggered by the user's dragging and rotating operations, and rotates the virtual parking space clockwise by a certain angle so that it is exactly on the platform. Then, the position finally occupied by the virtual parking space is the first parking position.

[0149] For another example, taking the user's first selected parking location as an example, refer to the parking area shown in FIG4b . The left control area may also contain a button N for automatically generating parking spaces. When the control device 231 detects that the user has clicked button N, it can automatically map multiple available and unavailable parking locations based on information from various onboard sensors 211-213, and control the display device 222 to display these parking locations in the right parking area, as shown in FIG6a . Optionally, different parking locations can be presented to the user using different colors, such as green for available parking locations and red for unavailable parking locations. The user can trigger a first instruction by clicking on a parking location. Based on the first instruction, the control device 231 determines the parking location clicked by the user as the first parking location and controls the display device 222 to populate the first parking location with a virtual vehicle identifier. For example, assuming that the control device 231 detects that the user clicks the green parking position shown in Figure 6a, the green parking position can be used as the first parking position, and the display device 222 can be controlled to fill the virtual vehicle logo in the green parking position, as shown in Figure 6b.

[0150] For another example, taking the parking position generated for the vehicle for the first time as an example, please refer to Figure 6a. After the user clicks the button N in the parking area, the first instruction can be automatically triggered. The control device 231 automatically plans multiple parking positions that can be parked and multiple parking positions that cannot be parked based on the information of various vehicle-mounted sensors 211~213, and can automatically select a parking position among them as the first parking position according to the first instruction, and then control the display device 222 to fill in the virtual vehicle logo in the first parking position, thereby obtaining the parking area shown in Figure 6b.

[0151] It is understandable that the first instruction can also be triggered by other means, such as but not limited to: user's voice instruction triggering, gesture instruction triggering, brain wave instruction triggering, etc., which will not be listed one by one in this application.

[0152] Step 303: Mark the position of the first obstacle, where the first obstacle includes an obstacle that interacts with the first parking position.

[0153] In one possible implementation, the control device 231 may identify the environment based on various types of vehicle-mounted sensors 211 to 213 to determine the positions of one or more obstacles in the environment surrounding the vehicle body, and then, based on the positional relationship between the positions of the one or more obstacles and the first parking position, screen out obstacles that may affect the vehicle parking in the first parking position from the one or more obstacles, and identify these obstacles as first obstacles.

[0154] Here, obstacles that affect the vehicle's parking in the first parking position can be understood as obstacles that affect parking from the perspective of position and / or movement trend. For example, they may include but are not limited to at least one of the following two types of obstacles:

[0155] The first category is obstacles that interact with the first parking position. For example, obstacles that are fully or partially displayed in the first parking position, such as static obstacles such as plastic bags, cardboard boxes, lawns, stone pillars, curbs, or dynamic obstacles that are fully or partially in the first parking position. Another example is obstacles that are located outside the first parking position but will interact with the first parking position, such as dynamic obstacles that will move into the first parking position within the next 1 minute (or other relatively short time), such as pedestrians, oncoming vehicles, small animals, toy cars, etc.

[0156] The second category is obstacles that are displayed outside the first parking position, but the on-board sensors sense that they are fully or partially located within the first parking position, such as static obstacles and dynamic obstacles that are not displayed within the first parking position due to image distortion and other reasons.

[0157] For example, taking the above two types of obstacles as an example, the control device 231 can obtain the position and movement trajectory of one or more obstacles identified by the camera 211, radar 212 and sonar 213 based on the recognition of the vehicle's surrounding environment by the camera 211, radar 212 and sonar 213. Then, based on the position of the one or more obstacles, it can be determined whether the position of the obstacle is located within the first parking position; based on the position and movement trajectory of the one or more obstacles, it can be determined whether the obstacle will move into the first parking position within a short period of time in the future; and based on the position and movement trajectory of the one or more obstacles, it can be determined whether a certain obstacle is partially or completely located within the first parking position but the parking area shows that the obstacle is located outside the first parking position. Afterwards, the obstacle determined as "yes" can be regarded as the first obstacle.

[0158] When the first obstacle is of the first category, the first obstacle is displayed within the first parking space, or is displayed outside the first parking space but will be moved into the first parking space within a short period of time. When the first obstacle is of the second category, the first obstacle is displayed outside the first parking space, but the vehicle determines through onboard sensors that the first obstacle is fully or partially within the first parking space. The first obstacle can be understood as the actual obstacle in the actual environment corresponding to the first obstacle displayed in the parking area.

[0159] As will be appreciated, if the first obstacle does not exist, indicating that the first parking position is available for parking, control device 231 may send control signals to other control components in control system 230 to drive the other control components to automatically park the vehicle in the first parking position. For example, control device 231 may plan a parking trajectory for the vehicle from its current position to the first parking position based on the first parking position and the vehicle's current position. It may then determine the vehicle's next turning angle and speed based on the parking trajectory. The turning angle may then be sent to a mechanical rotator (e.g., a steering rod) of the vehicle, and the speed may be sent to a motor controller of the vehicle. The mechanical steering device is connected between the vehicle's steering wheel and wheels. Upon receiving the turning angle from control device 231, the mechanical rotator may drive the wheels and steering wheel to rotate to a corresponding angle. The steering wheel rotation may present steering information to the user, while the wheel rotation may drive the vehicle to steer. The motor controller is connected to the vehicle's motor, which is in turn connected to the wheels. After receiving the speed information from control device 231, the motor controller generates a corresponding drive signal and sends it to the motor. The motor rotates at the speed corresponding to the drive signal, driving the wheels and, in turn, moving the vehicle forward or backward. Using this control logic, control device 231 can control the vehicle to gradually park in the first parking position along the planned trajectory.

[0160] Conversely, if the first obstacle exists, it indicates that the first parking location may not be available. In this case, the control device 231 may mark the location of the first obstacle. This marking may be implemented in various ways, including but not limited to at least one of HMI marking, voice prompt marking, direction prompt sound marking, light marking, and projection marking.

[0161] The HMI annotation may refer to controlling the display device 222 to display the first obstacle in the parking area. There are many ways to display the first obstacle, such as but not limited to: graphic annotation, text annotation, color annotation, pattern annotation, etc.

[0162] The voice prompt annotation may be, for example, a voice prompt message issued through the vehicle's speakers (or external speakers if supported) or a voice device such as a voice assistant, to inform the user where the first obstacle is located in the first parking position. These locations may be, for example, the left side of the vehicle, the right side of the vehicle, the front, the middle, or the rear of the vehicle, or a combination thereof, or other iconic locations within the vehicle, without limitation.

[0163] The positional sound indication can be based on the position of the first obstacle relative to the first parking position, and a sound can be emitted at the corresponding position in the vehicle relative to the driver. For example, when the first obstacle is located to the left rear of the first parking position, a vibration or ringing sound can be emitted through the speaker in the left rear seat of the vehicle. This allows the user to intuitively sense the location of the obstacle even when the vehicle is not parked in the first parking position, thereby improving the parking experience.

[0164] Light marking can be achieved by using interior or exterior vehicle lights to mark the location of the first obstacle. For example, when the first obstacle is located in the front left corner of the first parking space, the vehicle's left front turn signal can be flashed. Another example is when the first obstacle is located in the center of the first parking space, the interior ambient light can be flashed. And so on.

[0165] The projection marking can include using in-vehicle or external projection equipment to mark the location of the first obstacle. For example, the in-vehicle head-up display (HUD) can display the first parking position and the position of the first obstacle within the first parking position in a virtual image in front of the windshield. In another example, an onboard projector can display the location of the first obstacle within the vehicle, such as on the left or right window, front seat, rear seat, or other interior space. In another example, the vehicle's exterior lights can be used to project the first parking position onto the vehicle's surroundings and identify the first obstacle within the first parking position. For example, smart headlights can project a parking space at the location corresponding to the first parking position in the real environment, and obstacles such as pedestrians, vehicles, and boulders currently within the area can be identified with red light or other colored light. In this way, if the obstacle is dynamic, this method can provide a signal to the dynamic obstacle, prompting it to move away from the first parking position as quickly as possible, thereby assisting the user in completing parking.

[0166] Of course, there are other ways of marking. For example, you can also use dynamic effects in the interface of the central control screen. For example, vibration or flashing prompts. For example, you can also display prompts on the display screen outside the car. And so on. This application does not make specific restrictions on this.

[0167] In one possible implementation, if the first obstacle includes a dynamic obstacle, the control device 231 may also dynamically update the occupancy status of the first parking space. For example, if the vehicle has not left its current location or has left a short distance, the control device 231 may reacquire the surrounding environment captured in real time by various sensors every 5 seconds (or other relatively short period of time). Based on this surrounding environment, the control device 231 re-determines obstacles within the first parking space or outside the first parking space that may interact with the first parking space, thereby updating the obstacle information displayed in the parking area, promptly notifying the user of the removal of dynamic obstacles, and assisting the user in parking in the first available parking space.

[0168] In one possible implementation, when determining to mark the location of the first obstacle, the control device 231 may use a variety of methods to mark the location. Three possible marking methods are exemplified below. Furthermore, for ease of understanding, the following three marking methods are described using HMI implementation as an example.

[0169] Marking method 1: direct marking.

[0170] In marking method 1, the control device 231 can mark the first parking space as an obstacle occupancy based on the locations of one or more obstacles sensed by various onboard sensors 211-213. In other words, the obstacle occupancy is directly marked at the first parking space. In other words, the obstacle occupancy information is directly marked at the first parking space. This occupancy information can include location, size, or degree of encroachment.

[0171] For example, taking the parking area shown in FIG5b as an example, refer to FIG7a, FIG7b and FIG7c. Assume that there is an obstacle X in front of the left side of the vehicle, and the user drags the virtual parking space to the location of obstacle X. Then:

[0172] In one example, referring to FIG. 7 a , the control device 231 may control the display device 222 to mark the center position of the obstacle X at the first parking position, such as by marking a circle or other shape at the center point of the obstacle X. Optionally, a line or text may be marked, and the center point may be pointed to, to display information indicating that there is an obstacle, so that the user can quickly locate the center point.

[0173] In another example, referring to FIG. 7 b , the control device 231 may control the display device 222 to mark the outline of the obstacle X at the first parking position, so that the user can accurately know the area occupied by the first obstacle at the first parking position, and the user can quickly select an available parking position outside of the area.

[0174] In another example, referring to FIG. 7c , the control device 231 can control the display device 222 to mark the location of the obstacle X closest to the center point of the first parking space. In other words, the deepest point of the obstacle X's encroachment into the first parking space can be marked. For example, a circle or other shape can be marked at the deepest point, and a line and text can be used to highlight the deepest point, allowing the user to quickly locate the deepest point and understand the extent to which the first obstacle encroaches on the first parking space.

[0175] Using the above-mentioned marking method 1, the user can intuitively present the occupancy status of the obstacle itself, allowing the user to accurately know the location of the obstacle. It is understood that in addition to the three marking examples mentioned above, other marking examples can also exist, such as color marking, pattern marking, symbol marking, etc., which are not listed here one by one.

[0176] Marking method 2: area marking.

[0177] In the second marking method, the control device 231 can divide the first parking location into multiple sub-areas and mark each sub-area as occupied by an obstacle. For example, based on the locations of the obstacles sensed by the various onboard sensors 211-213, the control device 231 can determine the sub-area where the obstacle is located from among the multiple sub-areas, and then control the display device 222 to mark the sub-area where the obstacle is located. The number of sub-areas can be any integer, such as 4, 6, 8, etc., and is not specifically limited.

[0178] For example, taking six sub-areas as an example, please refer to (A) in Figure 8, which shows a possible sub-area division method for the first parking position. In this example, the control device 231 can divide the first parking position into six sub-areas and assign different identifications (IDs) to each of the six sub-areas, such as S1, S2, S3, S4, S5, and S6. Sub-areas S1 and S2 are located at the front of the vehicle, sub-areas S3 and S4 are located in the middle of the vehicle, and sub-areas S5 and S6 are located at the rear of the vehicle. For any of the six sub-areas, the control device 231 can first determine whether the entire obstacle or part of the obstacle is located in the sub-area. If so, it can be determined that the sub-area is occupied by an obstacle, and the control device 231 can assign a first attribute to the sub-area. Otherwise, it can be determined that the sub-area is not occupied by an obstacle, and the control device 231 can assign a second attribute to the sub-area. Then, the control device 231 may mark the sub-regions assigned with the first attribute according to the attributes of the six sub-regions, and clearly point out to the user the sub-regions occupied by the obstacles in a visible manner.

[0179] Optionally, the first attribute and the second attribute can be indicated in a variety of ways, such as text indication, pattern indication, digital indication, etc. For example, in one example, the first attribute can be indicated by filling "1" in the sub-area, and the second attribute can be indicated by filling "0" in the sub-area. For example, please refer to (B1) and (B2) in Figure 8. If the obstacle (the cone barrel is shown as an example) only occupies the sub-area S4 of the first parking position, the control device 231 can fill "1" in the sub-area S4 and can fill "0" in the remaining sub-areas S1 to S3 and S5 and S6. After the filling is completed, the display device 222 can be controlled to mark the sub-area filled with "1" (i.e., S4). For another example, referring to (C1) and (C2) in FIG8 , if an obstacle (a wheel chock is shown as an example) simultaneously occupies sub-areas S1 and S2 of the first parking position, the control device 231 may fill sub-areas S1 and S2 with "1" and may fill the remaining sub-areas S3 to S6 with "0". Once the filling is complete, the display device 222 may be controlled to mark the sub-areas filled with "1" (i.e., S1 and S2). For another example, referring to (D1) and (D2) in FIG8 , if an obstacle (a plant is shown as an example) simultaneously occupies sub-areas S1, S3, and S5 of the first parking position, the control device 231 may fill sub-areas S1, S3, and S5 with "1" and may fill the remaining sub-areas S2, S4, and S6 with "0". Once the filling is complete, the display device 222 may be controlled to mark the sub-areas filled with "1" (i.e., S1, S3, and S5). There are many more possible examples, which are not listed here.

[0180] Furthermore, optionally, there are many forms of implementing the annotation of the sub-areas, such as graphic annotation, text annotation, color annotation, pattern annotation, and line annotation, etc. For ease of understanding, taking the parking area shown in FIG5b and the obstacle position shown in (B1) in FIG8 as an example, assuming that the user just drags the virtual parking space to the position where the obstacle X is located, and the control device 231 determines that the obstacle X occupies the sub-area S4 in the first parking position, then:

[0181] In one example, referring to FIG9a , the control device 231 may control the display device 222 to mark a set symbol on the sub-area S4. The set symbol may be, for example, a rhombus, square, triangle, trapezoid, circle, ring, or other regular or irregular shapes. Optionally, in order to enable the user to focus on the location of the obstacle X more quickly, the marked symbol may also be colored, such as gray as shown in the figure, or other colors. Optionally, in order to enable the user to quickly distinguish the type of obstacle, different types of obstacles may also be marked with different shapes and colors, such as green plants with green triangles, buildings with red circles, wheel chocks with gray rhombuses, cones with blue squares, etc., which are not listed here one by one;

[0182] In another example, referring to FIG9b , the control device 231 may control the display device 222 to mark text on the sub-area S4, such as marking text directly at the center point of the sub-area S4, or marking text outside the first parking position, and drawing an arrow (or broken line, or other form) on the sub-area S4 to point to the text. The marked text may be uniform, such as "obstacles", as shown in FIG9b . Alternatively, different obstacles may be marked with different text, such as the Chinese name of the obstacle, the number of the obstacle, the English name of the obstacle, etc., which are not listed here one by one;

[0183] In another example, please refer to Figure 9c. The control device 231 can control the display device 222 to fill the sub-area S4 with color, such as filling the entire sub-area S4 with a set color. The set color can be uniform, such as the gray shown in Figure 9c, or other colors, such as a more obvious color, such as red. Red itself has a highlighting effect, which can enable users to focus on the location of the obstacle more quickly. Alternatively, different obstacles can correspond to different colors, such as gray for wheel chocks, green for green plants, red for buildings, blue for cones, etc., which will not be listed here one by one;

[0184] In another example, referring to Figure 9d, the control device 231 can control the display device 222 to mark a pattern on sub-area S4. For example, a pattern that can be used to indicate the type of obstacle can be marked on sub-area S4. For example, when obstacle X is a cone, a cone pattern as shown in Figure 9d can be marked. By marking the obstacle type pattern, the user can quickly know what type of obstacle occupies the parking space, which can facilitate the user to plan the next parking space. Of course, other patterns can also be marked, and this application does not specifically limit this.

[0185] In another example, referring to FIG9e , the control device 231 may control the display device 222 to mark a line on the sub-area S4, for example, to draw a line outlining the area occupied by the obstacle. This outline allows the user to quickly understand the actual size of the obstacle, making it easier for the user to select another parking location.

[0186] By using the second annotation method above, by annotating the sub-region to which the obstacle belongs, even if the actual obstacle location deviates from the detected obstacle location due to some reasons (such as detection error), the actual obstacle location can be included in the corresponding sub-region and annotated simultaneously, thereby increasing the probability that the user can adjust to a suitable parking position based on the annotation. It is understood that in addition to the above-mentioned annotation examples, other annotation examples are possible and are not listed here one by one.

[0187] Marking method three: edge marking.

[0188] In the third marking method, when the first parking position is partially outside the parking area and the first obstacle occupies the portion outside the parking area, the control device 231 may mark the positional relationship of the first obstacle relative to the first parking position, such as marking the edge of the parking area covered by the first parking position as occupied by the obstacle. There are many ways to implement marking the edge. For example, referring to FIG10a, assuming that the first parking position is located in the upper left of the parking area, the front part of the first parking position is located in the parking area, the rear part of the first parking position is located outside the parking area, and there is an obstacle X in the portion outside the parking area, then the control device 231 may control the display device 222 to mark the position information indicating that "obstacle X is located to the left of the current parking position" in the parking area. For example:

[0189] In one example, referring to FIG10b , the control device 231 can control the display device 222 to mark the boundary between the first parking position and the edge of the parking area with a color, such as black as shown, or a more obvious red, to attract attention. Optionally, to improve the viewing effect, the desired color can also be marked in the form of light, such as red light at the edge to increase the aesthetics.

[0190] In another example, referring to FIG. 10c , the control device 231 may control the display device 222 to indicate the position of the obstacle X in the first parking position through arrows and text. For example, an arrow may be pointed at the intersection of the first parking position and the edge of the parking area, and text "Obstacle on the left" may be used to indicate to the user that the obstacle is on the left side of the first parking position.

[0191] And so on. There are many possible annotation implementation forms, which are not listed here one by one.

[0192] By using the above marking method, users can intuitively know the location of the obstacle outside the parking area relative to the first parking position, thereby helping users to accurately avoid the location of the obstacle and quickly drag the virtual parking space to a position without obstacles, thereby achieving efficient placement of custom parking spaces and improving the adjustment efficiency of custom parking spaces.

[0193] It is understandable that in addition to the above three marking methods, there may be other marking methods, and this application does not make specific limitations on this.

[0194] In one possible implementation, in addition to marking the position of the first obstacle, the control device 231 may also provide first recommendation information based on the position of the first obstacle and the surrounding environment of the vehicle. The first recommendation information is used to indicate the recommended parking adjustment direction and / or the recommended parking adjustment position. The first recommendation information may be a voice recommendation, a display recommendation, a headlight flashing recommendation, etc. For example, taking the display method of recommending both the parking adjustment direction and the parking adjustment position as an example, refer to FIG11a. The control device 231 may control the display device 222 to display the drag position and the drag direction in the parking area, so that the user can quickly move the virtual parking space to the recommended parking position with reference to the recommended drag direction, thereby accelerating parking.

[0195] In one possible implementation, in addition to marking the position of the first obstacle, the control device 231 may also mark the relationship between the first obstacle and the available parking position. This relationship may, for example, include the edge position of the first obstacle and / or a critical safety position, which is a position at a preset safety distance from the edge position of the first obstacle. For example, referring to FIG11b , the three solid lines within the first parking position represent the edge position lines of the first obstacle, and the three dashed lines represent the position line consisting of the edge position of the first obstacle plus the preset safety distance, i.e., the critical safety position line. If parking conditions are met, such as if the free area around the first parking position is sufficiently large for the user to reselect a position farther from the current parking position, the user may drag the virtual parking space with reference to the critical safety position line to ensure that the virtual parking space does not extend beyond the range defined by the critical safety position line. If parking conditions are unavailable, for example, if the area around the first parking spot is relatively small, allowing the user to select a new spot closer to the current one, the user can drag the virtual parking space relative to the edge of the first obstacle, ensuring that the virtual parking space does not extend beyond the edge. This allows the user to more intuitively identify the edge of the obstacle or the critical safety position, allowing the user to quickly identify a safe area and select a suitable parking spot more quickly.

[0196] In one possible implementation, upon determining the presence of a first obstacle, the control device 231 may further identify the first obstacle before marking the location of the first obstacle in the manner described above. If the first obstacle is identified as an obstacle that affects parking, the first obstacle may be annotated with occupied information, for example, using any of the marking methods 1 through 3 described above. Conversely, if the first obstacle has no effect on parking, the first obstacle may be annotated with unoccupied information. Alternatively, if it is impossible to determine whether the first obstacle affects parking, the first obstacle may be annotated with unknown information.

[0197] Optionally, the non-occupied information can be implemented in a variety of forms. For example, in one example, the non-occupied information can be implemented by not marking it. In this case, when a first obstacle is marked in the parking area, it indicates that the first obstacle is an obstacle that will affect parking. Therefore, the user can directly select a position that can avoid the first obstacle to adjust the parking space. For example, in another example, the non-occupied information can also be marked according to any of the marking methods one to three mentioned above, but it is different from the marking method corresponding to the occupied information. For example, obstacles that do not affect parking are marked in green, and obstacles that do affect parking are marked in red. For another example, obstacles that do not affect parking are marked as dots, and obstacles that do affect parking are marked as circles, and so on.

[0198] Alternatively, unknown information may be labeled using any of the above-mentioned methods 1 through 3, but the labeling methods may differ from those used for occupied and unoccupied information. For example, obstacles that do not affect parking may be labeled green, obstacles that do affect parking may be labeled red, and obstacles whose impact on parking cannot be determined may be labeled yellow. Another example is obstacles that do not affect parking may be labeled as dots, obstacles that do affect parking may be labeled as circles, and obstacles whose impact on parking cannot be determined may be labeled as triangles, and so on.

[0199] Optionally, the occupancy information for obstacles with varying degrees of impact on parking can be labeled differently. For example, obstacles with a significant impact on parking can be labeled red, obstacles with a moderate impact can be labeled gray, and obstacles with a slight impact can be labeled blue. Another example is obstacles with a significant impact can be labeled as pentagons, obstacles with a moderate impact can be labeled as quadrilaterals, and obstacles with a slight impact can be labeled as triangles, and so on.

[0200] Optionally, when the first obstacle is an obstacle that has no effect on parking, the control device 231 may also issue a first prompt message, and the first prompt message is used to prompt the user to choose whether to ignore the first obstacle. Afterwards, if the control device 231 receives a second instruction from the user to ignore the first obstacle, it can control the vehicle to automatically park in the first parking position based on the second instruction. The first prompt message may be one or more of a voice prompt, a display prompt, a flashing headlight prompt, a vibration prompt, etc. For example, the user may be prompted by voice through an on-board speaker or other on-board voice device, or by displaying text in the parking area, or by displaying text on the virtual image of the windshield through the HUD, or by projecting a pattern or text through the headlights, etc., without specific limitation. In addition, the user may trigger the second instruction by clicking a button, by voice, by gesture, or by other interactive methods, etc., without specific limitation.

[0201] Optionally, whether the first obstacle has an impact on parking is identified. There are many optional identification methods, and several examples are given below to illustrate.

[0202] One possible identification method can be through a neural network model. For example, a neural network model can be pre-trained based on known types of obstacles that may affect parking and types that do not. The first obstacle can then be identified using the trained neural network model. This method offers high accuracy.

[0203] Another possible identification method is to pre-extract first features for various types of obstacles that have no impact on parking and second features for various types of obstacles that do impact on parking. A first obstacle is then matched against both the first and second features. If the first obstacle possesses the first feature, it is determined to be an obstacle that has no impact on parking. If the first obstacle possesses the second feature, it is determined to be an obstacle that has an impact on parking. If the first obstacle possesses neither the first nor the second feature, it is determined to be an obstacle that cannot be determined to have an impact on parking. The first feature can include at least one of the following: the obstacle's height is less than or equal to a set height, the maximum distance the obstacle intrudes into the parking space is less than or equal to a set distance, or other features. The second feature can include at least one of the following: the obstacle's height is greater than a set height, the maximum distance the obstacle intrudes into the parking space is greater than a set distance, or other features.

[0204] Optionally, the aforementioned set height can be set based on the vehicle's chassis height. For example, it can be set to a value slightly lower than the chassis height. Thus, when the height of the first obstacle is less than or equal to the set height, it means that the first obstacle is lower than the vehicle's chassis height. Even if the vehicle passes over the obstacle, it is highly unlikely to scratch the chassis. Therefore, the first obstacle can be considered an obstacle that does not affect parking. Of course, the set height can also be set to other values, and there is no specific limitation.

[0205] Optionally, the aforementioned set distance can be set based on the distance between the wheels and the edge of the vehicle body. For example, it can be set to be slightly smaller than the distance between the wheels and the edge of the vehicle body. Thus, when the maximum distance that the first obstacle intrudes into the parking position exceeds the set distance, it means that the first obstacle has penetrated deeply into the position where the wheels are located and is likely to block the wheels from moving. Therefore, the first obstacle can be considered an obstacle that affects parking. Of course, the set distance can also be set to other values, and there is no specific limitation.

[0206] Another possible identification method is to configure a preset whitelist based on various types of obstacles that are known to have no effect on parking, and / or configure a preset blacklist based on various types of obstacles that are known to have an effect on parking. When the type of the first obstacle belongs to the preset whitelist, the first obstacle can be determined to be an obstacle that has no effect on parking. When the type of the first obstacle belongs to the preset blacklist, the first obstacle can be determined to be an obstacle that has an effect on parking. When the first obstacle belongs to neither the preset whitelist nor the preset blacklist, the first obstacle can be determined to be an obstacle that cannot be identified as to whether it affects parking.

[0207] Optionally, the preset whitelist can contain identification information, such as name or type, for obstacles that have minimal impact on parking. These obstacles may include soft materials such as plastic bags and cardboard boxes, or other obstacles with negligible impact. Conversely, the preset blacklist can contain identification information, such as name or type, for obstacles that have a significant impact on parking. These obstacles may include hard materials such as lawns, stone piers, curbs, or wheel chocks, or other obstacles with significant impact.

[0208] It is understandable that whether the first obstacle affects parking can be determined by the above-mentioned single identification method, or the above-mentioned multiple identification methods can be combined to comprehensively determine whether the first obstacle affects parking. Alternatively, other identification methods can be used, and this application does not make specific limitations on this.

[0209] In the above implementation, by distinguishing and marking obstacles that affect parking and those that do not, users can understand the type of the current obstacle so that they can respond to different types of obstacles more specifically and assist users in quickly adjusting parking positions.

[0210] In one possible implementation, the control device 231 may also use different labeling methods for static obstacles and dynamic obstacles. For example, static obstacles may be labeled red, while dynamic obstacles may be labeled yellow. Another example is that static obstacles may be labeled with text, while dynamic obstacles may be labeled with graphics. Another example is that static obstacles may be labeled with triangles, while dynamic obstacles may be labeled with squares, and so on. By distinguishing between static and dynamic obstacles, different display effects can be presented to the user, improving the user's parking viewing experience.

[0211] It is understandable that the above content only exemplifies several possible implementation methods of marking obstacles. This application does not limit the marking of obstacles in the parking area to these methods. Any marking method that can prompt the user of the location of the obstacle through an external effect is within the scope of protection of this application, and this application does not make specific limitations on this.

[0212] To further illustrate the parking assistance method in this application, the following describes several possible application scenarios and their specific implementations. Each of the following application scenarios uses color marking as an example, with the first parking position divided into six zones.

[0213] Application Scenario 1

[0214] Please refer to FIG12 a , which shows a schematic diagram of marking obstacles in a horizontal parking space provided by the present application.

[0215] As shown in Figure 12a, assuming that the user drags the virtual parking space to the first parking position shown in Figure 12a (A), there is a cone barrel at the first parking position in the actual scene. In this case, the control device 231 can determine that there is an obstacle located in area S4 of the first parking position based on the detection results of various on-board sensors 211 to 213. Area S4 is occupied by the obstacle. Therefore, the control device 231 can fill "1" in area S4 and fill "0" in other areas, as shown in Figure 12a (B). Afterwards, as shown in Figure 12a (C), the control device 231 can mark the color of area S4 filled with "1" and not mark the color of other areas. The marked color can be gray as shown in Figure 12a (C), or other colors, such as red, without specific limitation.

[0216] Application Scenario 2

[0217] Please refer to FIG12 b , which shows a schematic diagram of marking obstacles in a vertical parking space provided by the present application.

[0218] As shown in FIG12b , assuming that a user drags a virtual parking space to the first parking position shown in FIG12b (A), which has a wheel chock in the actual scene, the control device 231 can determine, based on the detection results of various on-board sensors 211 to 213, that an obstacle spans regions S1 and S2 of the first parking position, preventing the wheels from parking. Therefore, regions S1 and S2 are occupied by the obstacle, and the control device 231 can fill regions S1 and S2 with "1" and fill other regions with "0", as shown in FIG12b (B). Subsequently, as shown in FIG12b (C), the control device 231 can color the regions S1 and S2 filled with "1", such as gray as shown in FIG12b (C).

[0219] Application Scenario 3

[0220] Please refer to FIG. 12 c , which shows another schematic diagram of marking obstacles in a vertical parking space provided by the present application.

[0221] As shown in Figure 12c (A), assume that the vehicle automatically generates four parking spaces, arranged side by side from top to bottom, and the user selects the second parking space at the top for parking. In the actual scene, there are cones in the area between the vehicle and the second parking space. In this case, based on the detection results of various onboard sensors 211-213, the control device 231 can determine that there are obstacles in areas S3 and S4 of the second parking space. Areas S3 and S4 are occupied by obstacles. Therefore, the control device 231 can fill areas S3 and S4 with "1" and fill other areas with "0", as shown in Figure 12c (B). The control device 231 can then color the areas S3 and S4 filled with "1", such as gray as shown in Figure 12c (C).

[0222] Application Scenario 4

[0223] Please refer to FIG. 12 d , which shows a schematic diagram of marking obstacles in a parking area provided by the present application.

[0224] As shown in FIG12d , assume that there is an obstacle X at the lower right corner of the vehicle's current position, and the user drags the virtual parking space to the right of the vehicle's current position during automatic parking, causing the lower left area of ​​the dragged parking space to be blocked by obstacle X. In this case, the control device 231 determines, based on the detection results of the various on-board sensors 211 to 213, that the lower left area of ​​the dragged parking space is occupied by the obstacle. Therefore, the control device 231 may mark the lower left area of ​​the dragged parking space with a color, such as gray as shown. Optionally, to enhance the aesthetics of the parking area, the control device 231 may also set a gradient effect on the edges of the colored area, such as gradually transitioning from gray as shown to white in other areas, to enhance the user's viewing experience.

[0225] In one example, as shown in Figure 12d, to more clearly indicate the location of the obstacle, the control device 231 may also mark a predetermined shape in the lower left area at the actual location of the obstacle X. This predetermined shape may be a square, circle, trapezoid, cone, or other regular or irregular shape. The illustration uses a square containing a circle as an example. Optionally, the predetermined shape may be filled with a darker color than other locations, such as dark gray in the illustration, to draw the user's attention.

[0226] In one example, as shown in FIG12d , text may be displayed above the parking area (or elsewhere) to help the user quickly understand that the current parking space is unavailable. For example, FIG12d uses the text "Obstacle blocked, please adjust target parking space" as an example, but the text may also be any other text that indicates an unavailable parking space, and this application does not specifically limit this.

[0227] Application Scenario 5

[0228] Please refer to FIG. 12e , which shows another schematic diagram of marking obstacles in a parking area provided by the present application.

[0229] As shown in FIG12e , assume that an obstacle X exists to the lower right of the vehicle's current position. During automatic parking, the user drags the virtual parking space to the right of the vehicle's current position, resulting in the lower left and center left boundaries of the dragged parking space being blocked by obstacle X. In this scenario, the control device 231 determines, based on the detection results of the various onboard sensors 211 to 213 , that the lower left and center left regions of the dragged parking space are occupied by the obstacle. Therefore, the control device 231 may color the lower left and center left regions of the dragged parking space, such as gray as shown. Optionally, a predetermined shape may be marked at the boundary of the lower left and center left regions, and the predetermined shape may be filled with a darker color than other areas to draw the user's attention.

[0230] Application Scenario 6

[0231] Please refer to FIG. 12 f , which shows another schematic diagram of marking obstacles in a parking area provided by the present application.

[0232] As shown in Figure 12f, assume that there is a flower bed to the left of the vehicle's current position and another vehicle to the left in front. During automatic parking, the user drags the virtual parking space to the left of the vehicle's current position, causing the entire vehicle body in the dragged parking position to be blocked by the flower bed and other vehicles. In this case, the control device 231, based on the detection results of the various on-board sensors 211-213, determines that the entire area of ​​the dragged parking position is occupied by obstacles. Therefore, the control device 231 may mark the entire area of ​​the dragged parking position with a color, such as gray as shown. Optionally, a predetermined shape may be marked at the location of the flower bed in the entire area, and the predetermined shape may be filled with a darker color than other locations to attract the user's attention.

[0233] Application Scenario 7

[0234] Please refer to FIG. 12g , which shows another schematic diagram of marking obstacles in a parking area provided by the present application.

[0235] As shown in Figure 12g, assume that there is an obstacle X located further to the lower right of the vehicle's current position. During automatic parking, the user drags the virtual parking space to the lower right of the vehicle's current position, so that the front half of the vehicle is located within the parking area, while the rear half is located outside the parking area. The rear half is blocked by obstacle X, but because the blocked portion is outside the parking area, the obstacle is not displayed within the parking area. In this case, the control device 231 may color the edge of the parking area blocked by the obstacle, such as gray as shown. Optionally, to highlight the obstacle's location outside the parking area, the edge may be illuminated with light, such as gray as shown, or other colors, such as red, with the obstacle as the luminous point.

[0236] Application Scenario 8

[0237] Assuming that the vehicle is in the parking scene shown in Figure 1, please refer to Figure 12h, which shows another schematic diagram provided by the present application for marking obstacles in the parking area. The diagram takes the example of a relatively small space on the right side of the vehicle's current position.

[0238] First, assume that after a user activates the automatic parking function, the display device 222 displays the parking interface shown in FIG12h (A). A button M for the custom parking function is located in the lower right corner of the control area on the left side of the parking interface. If the user clicks this button M, a default virtual parking space appears in the parking area on the right side of the parking interface. This virtual parking space is displayed near the vehicle, such as to the right of the vehicle as shown in FIG12h (A). A rotation button K is also displayed above the virtual parking space. Next, referring to FIG12h (B), assume that the user selects a parking space using the custom parking function. The user can click and hold the virtual parking space and drag it to the free area in the upper right corner. Then, they can click and hold the rotation button K and rotate it 90° counterclockwise so that the virtual parking space fits neatly into the free area. Optionally, there is also a snap button V in the upper right corner of the control area. After dragging the virtual parking space to the parking location, the user can click this snap button V to automatically snap the virtual parking space to the parking location. Furthermore, referring to (B) and (C) in FIG12h , in an actual parking scenario, greenery has already encroached upon the left side of the current parking location. However, due to image distortion of the parking area (e.g., wide IPM distortion), the user mistakenly believes that the currently selected parking location is not obstructed by greenery. In this case, the control device 231 determines, based on the detection results of the various onboard sensors 211-213, that the left side of the current parking location is occupied by an obstacle. Therefore, the control device 231 may color the left side of the current parking location, such as gray as shown in (D) in FIG12h , or a more obvious color, such as red. Simultaneously, the upper portion of the parking area may also display text indicating that the current parking location is unavailable, such as "Obstacle blocked, please adjust target parking space" as shown in (D) in FIG12h . Optionally, a textual indication of the location of the obstruction may be provided, such as "Obstacle blocked to the left of target parking space, please adjust target parking space," or other textual descriptions, to help the user more quickly notice the result and assist in changing the parking location.

[0239] According to the above content, by displaying the location of obstacles in the parking area, the user can be intuitively prompted that the currently selected parking location is blocked by an obstacle, so that the user can accurately know the positional relationship between the obstacle and the currently selected parking location, and then assist the user to avoid the obstacle and quickly select a parking location where there are no obstacles, thereby improving the efficiency of the user in selecting a parking location and enhancing the user's parking experience.

[0240] It should be noted that the above content is introduced using the example of a control device in a vehicle executing a parking assistance method, but the parking assistance method can also be executed by other devices or components, such as other components in the vehicle, including but not limited to: on-board terminals, on-board controllers, on-board modules, on-board modules, on-board components, on-board chips, on-board units, on-board radars or on-board cameras and other sensors, or devices or components outside the vehicle, such as cloud servers, terminal devices, RSUs or other vehicles. The controlled vehicle can display obstacles at the current parking position to the user by interacting with these devices or components to assist the user in quickly selecting a parking position where there are no obstacles.

[0241] Furthermore, this parking assistance solution can be extended to any device or system requiring efficient parking. For example, it can be applied to any movable device with parking capabilities, including but not limited to ships, airplanes, high-speed trains, trains, helicopters, lawn mowers, and so on. Alternatively, it can be applied to mobile robots. For example, when an obstacle is detected in the robot's automatically planned parking location, relevant information about the obstacle can be provided to the user, helping the user quickly select a suitable parking location. And so on.

[0242] Furthermore, with the evolution of system architecture and the emergence of new scenarios, the parking assistance method provided in this application is also applicable to similar technical problems, and this application does not make any specific limitations on this.

[0243] Based on the parking assistance method described above, the present application may also provide a parking assistance system. Please refer to Figure 13 for a possible architecture diagram of the parking assistance system. The parking assistance system 1300 may be integrated into a vehicle. The parking assistance system 1300 may include a control device 231 and a display device 222 connected to the control device 231. The control device 231 may be any device in the vehicle capable of performing control functions, such as an MDC, VCU, or VDC. The display device 222 may be an onboard display screen, such as an instrument panel, a central control screen, a passenger screen, or a projection screen, such as a windshield, side window, or skylight in a head-up display (HUD). The control device 231 is configured to generate a parking area based on the vehicle's surroundings and transmit it to the display device 222. The display device 222 is configured to display the parking area. The control device 231 is further configured to control the display device 222 to display a first parking position based on a first instruction and to mark the position of a first obstacle, including obstacles that interact with the first parking position.

[0244] In one possible implementation, referring to FIG. 13 , parking assistance system 1300 may further include sensor system 210 , which includes a camera for capturing environmental images. Control device 231 , connected to the camera, is configured to, upon detecting that the vehicle has activated the automatic parking function, capture the environmental image captured by the camera, generate a parking area based on the image, and transmit the image to display device 222 .

[0245] In a further possible implementation, referring to FIG. 13 , sensor system 210 may further include radar and / or sonar, which are used to collect environmental information and identify the location and / or movement trajectory of obstacles from the environmental information. Control device 231 is connected to the radar and / or sonar and is used to determine obstacles that interact with the first parking position based on the obstacle locations and / or movement trajectories identified by the radar and / or sonar.

[0246] In a possible implementation, referring to FIG. 13 , the control device 231 is further configured to control the vehicle to park in the first parking position when the first obstacle does not exist.

[0247] In a further possible implementation, referring to FIG. 13 , the parking assistance system 1300 may further include a mechanical steering device 241, a motor controller 242, and a motor 243. The mechanical steering device 241 is connected between the vehicle's steering wheel 251 and wheels 252, and the motor 243 is connected between the motor controller 242 and the wheels 252. The control device 231 is specifically configured to, when the first obstacle is absent, plan a parking trajectory for the vehicle based on the vehicle's current position and the first parking position, determine the next turning angle and speed based on the parking trajectory, and transmit the turning angle to the mechanical steering device 241 and the speed to the motor controller 242. The mechanical steering device 241 is configured to rotate the steering wheel 251 and wheels 252 to corresponding angles based on the turning angle received from the control device 231. The rotation of the steering wheel 251 can present steering information to the user, while the rotation of the wheels 252 can cause the vehicle to steer. The motor controller 242 is configured to generate a corresponding drive signal based on the speed received from the control device 231 and transmit it to the motor 243. The motor 243 is used to rotate according to the driving electrical signal from the motor controller 242 to drive the wheels 252 to rotate, thereby realizing forward or backward movement of the vehicle.

[0248] It should be noted that the functions of the various components in the parking assistance system 1300, the concepts involved and related to the technical solution provided by this application, the explanation and detailed description and other steps, please refer to the description of these contents in the aforementioned method embodiment, and will not be repeated here.

[0249] Based on the parking assistance method described above, the present application may also provide a parking assistance device, which may be used to execute the above parking assistance method. For related features, please refer to the above method embodiment and will not be repeated here.

[0250] In one possible implementation, please refer to FIG. 14 , which illustrates a schematic diagram of a possible parking assistance device structure. Parking assistance device 1400 may be a chip or circuit, such as one that can be installed in a vehicle, or in an electronic device outside the vehicle. Parking assistance device 1400 may correspond to the control device in the aforementioned method, such as control device 231. Parking assistance device 1400 may implement the steps performed by control device 231 in the method shown in FIG. 3 , or the steps performed by control device 231 in the system shown in FIG. 13 .

[0251] As shown in FIG14 , the parking assistance device 1400 may include a control module 1410 and a labeling module 1420. When the parking assistance device 1400 is in operation, the control module 1410 is configured to control the display device to display a parking area generated based on the vehicle's surrounding environment, and to control the display device to display a first parking position based on a first instruction. The labeling module 1420 is configured to label the position of a first obstacle, including obstacles that interact with the first parking position.

[0252] For the concepts, explanations, detailed descriptions, and other steps involved in the parking assistance device 1400 and related to the technical solutions provided in the embodiments of the present application, please refer to the descriptions of these contents in the aforementioned method or other embodiments, and will not be repeated here.

[0253] It should be understood that the above division of units within parking assistance device 1400 is merely a division of logical functions. In actual implementation, these units may be fully or partially integrated into a single physical entity, or physically separated. This application does not impose any specific limitations on this. The functions of each unit within parking assistance device 1400 can be found in the corresponding method embodiments and will not be further elaborated here.

[0254] In one possible implementation, please refer to FIG. 15 , which illustrates another possible structural diagram of a parking assistance device. The parking assistance device 1500 may be a chip or a system-on-chip. Optionally, the system-on-chip may consist of a chip alone or include a chip and other discrete components. As shown in FIG. 15 , the parking assistance device 1500 may include at least one processor 1510 and a memory 1520. The at least one processor 1510 is coupled to the memory 1520, which may be located within or outside the parking assistance device 1500. The memory 1520 stores the computer programs or instructions necessary to implement any of the aforementioned method embodiments. By executing the computer programs or instructions stored in the memory 1520, the at least one processor 1510 performs the parking assistance method described in any of the aforementioned method embodiments.

[0255] The parking assistance device 1500 may also include a communication interface 1530, through which the parking assistance device 1500 can exchange information with other devices. The communication interface 1530 may be a circuit, a bus, a transceiver, or any other device capable of exchanging information, or may be referred to as a signal transceiver unit. When the parking assistance device 1500 is a chip-type device or circuit, the communication interface 1530 in the parking assistance device 1500 may also be an input / output circuit capable of inputting (or receiving) and outputting (or transmitting) data. The at least one processor 1510 may be an integrated processor, microprocessor, or integrated circuit, and may determine output data based on input data.

[0256] When parking assistance device 1500 is a control device in a vehicle, at least one processor 1510 can access computer programs or instructions stored in memory 1520 to execute the steps of any of the above method embodiments. For example, at least one processor 1510 can communicate with a display device in the vehicle via communication interface 1530 to control the display device to display a parking area, and control the display device to display a first parking position based on a first instruction, where the parking area is generated based on the vehicle's surrounding environment. Furthermore, at least one processor 1510 can communicate with various onboard sensors in the vehicle via communication interface 1530 to mark the position of a first obstacle, where the first obstacle includes an obstacle that interacts with the first parking position.

[0257] When parking assistance device 1500 is a control device external to the vehicle, such as an electronic device external to the vehicle, at least one processor 1510 can access a computer program or instruction stored in memory 1520 to execute the steps of any of the above method embodiments. For example, at least one processor 1510 can communicate with the controlled vehicle via communication interface 1530, causing a display device of the controlled vehicle to display a parking area generated based on the vehicle's surroundings, control the display device to display a first parking position based on a first instruction, and mark the position of a first obstacle, where the first obstacle includes an obstacle that interacts with the first parking position.

[0258] The processor 1510 can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and can implement or execute the methods, steps, and logic block diagrams disclosed herein. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed herein can be directly implemented and executed by a hardware processor, or by a combination of hardware and software modules within the processor.

[0259] The above-mentioned memory 1520 can be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or a volatile memory (volatile memory), such as a random-access memory (RAM). The memory can also be any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this. The memory 1520 in this application can also be a circuit or any other device that can implement a storage function, for storing computer programs, computer programs or instructions and / or data.

[0260] For the concepts, explanations, detailed descriptions and other steps involved in the parking assistance device 1500 and related to the technical solutions provided in the embodiments of the present application, please refer to the descriptions of these contents in the aforementioned method or other embodiments, which are not repeated here.

[0261] Based on the parking assistance method described above, the present application can also provide a vehicle, which may include a unit or module for implementing the above parking assistance method, for example, it may include a parking assistance device as shown in Figure 14 or Figure 15 above, or it may include a parking assistance system as shown in Figure 13 above, which will not be repeated here.

[0262] For example, the vehicle may be a car, truck, motorcycle, bus, recreational vehicle, amusement park vehicle, construction equipment, tram, toy car, golf cart, train, etc., and this application does not impose any particular limitation. In addition, the vehicle may be a new energy vehicle, including an electric vehicle, such as a two-wheel drive electric vehicle or a four-wheel drive electric vehicle, or a fuel vehicle, and this application does not impose any particular limitation.

[0263] Based on the parking assistance method described above, the present application may also provide an electronic device that may include a unit or module for implementing the above parking assistance method, such as the parking assistance device shown in Figures 14 or 15 above. The electronic device is connected to a controlled vehicle and implements the above parking assistance method by communicating with the controlled vehicle.

[0264] Exemplarily, the electronic device may be a cloud server, a terminal device, an RSU or other vehicles, etc., and this application does not make any special limitation.

[0265] Based on the parking assistance method described above, the present application also provides a computer-readable storage medium, which stores a program or instruction. When the program or instruction runs on a parking assistance device, the parking assistance device executes the method shown in Figure 3 above.

[0266] Based on the parking assistance method described above, the present application may also provide a computer program product, which includes: computer program code, which, when executed on a computer, enables the computer to execute the method shown in FIG3 above.

[0267] In this application, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. "At least one" or similar expressions refer to any combination of these, including any combination of single or plural. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple. In addition, in this application, the word "optionally" or "exemplarily" is used to indicate an example, illustration or description. Any embodiment or design scheme described as "example" or "optional" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Alternatively, it can be understood that the use of the word "example" or "optional" is intended to present concepts in a specific way and does not constitute a limitation on this application.

[0268] In addition, the "connection" in this application can be understood as an electrical connection, and the connection between two electrical components can be a direct or indirect connection between the two electrical components. For example, the connection between A and B can be a direct connection between A and B, or an indirect connection between A and B through one or more other electrical components. For example, A and B can be connected, or A and C can be directly connected, and C and B can be directly connected, and A and B are connected through C. In some scenarios, "connection" can also be understood as coupling, such as electromagnetic coupling between two inductors. In short, the connection between A and B enables the transmission of electrical energy and signals between A and B.

[0269] It will be appreciated that the various numerical numbers involved in this application are merely for the purpose of describing the distinctions made, and are not intended to limit the scope of the embodiments of the present application. The size of the sequence numbers of the above-mentioned processes does not imply the order of execution, and the order of execution of each process should be determined by its function and inherent logic. Terms such as "first", "second", and similar expressions are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, comprising a series of steps or units. Methods, systems, products, or devices are not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products, or devices.

Claims

1. A parking assistance method, characterized in that: The method comprises: controlling a display device to display a parking area, wherein the parking area is generated based on a surrounding environment of the vehicle; controlling the display device to display a first parking position based on a first instruction; The position of a first obstacle is marked, where the first obstacle includes an obstacle that interacts with the first parking position.

2. The method according to claim 1, wherein The obstacles that interact with the first parking position include obstacles within the first parking position and / or obstacles outside the first parking position that will interact with the first parking position.

3. The method according to claim 1, wherein The first obstacle is displayed outside the first parking position, and the first obstacle is entirely or partially located within the first parking position. Whether the first obstacle is entirely or partially located within the first parking position is determined by the perception result of the vehicle through the on-board sensor.

4. The method according to any one of claims 1 to 3, characterized in that The first parking position is a parking position automatically generated by the vehicle, and / or the first parking position is a parking position determined by a user.

5. The method according to any one of claims 1 to 4, characterized in that The marking the position of the first obstacle includes: The first parking position is marked as occupied by an obstacle based on environmental perception performed by multiple sensors.

6. The method according to any one of claims 1 to 5, characterized in that The marking the position of the first obstacle includes: Marking the position of the first obstacle closest to the center point of the first parking position; or, marking the outline of the first obstacle; or, The center position of the first obstacle is marked.

7. The method according to any one of claims 1 to 5, characterized in that The marking the position of the first obstacle includes: The first parking position includes a plurality of sub-areas, and each sub-area is marked with an obstacle occupancy mark.

8. The method according to any one of claims 1 to 5, characterized in that The marking the position of the first obstacle includes: The first parking position is partially located outside the parking area, and the first obstacle occupies a portion outside the parking area. The edge of the parking area covered by the first parking position is marked as occupied by the obstacle.

9. The method according to any one of claims 1 to 8, characterized in that The marking the position of the first obstacle includes: The first obstacle is identified, and if the first obstacle has an impact on parking, the first obstacle is marked as occupied; if the first obstacle has no impact on parking, the first obstacle is marked as unoccupied.

10. The method according to claim 9, wherein The first obstacle has an impact on parking, including: The first obstacle satisfies at least one of the following conditions: a height of the first obstacle exceeds a set height, a maximum distance that the first obstacle intrudes into the first parking position is greater than a set distance, and the first obstacle is not in a preset whitelist.

11. The method according to claim 9 or 10, wherein: If the first obstacle has no effect on parking, the method further includes: issuing a first prompt message, where the first prompt message is used to prompt the user to choose whether to ignore the first obstacle; The vehicle is controlled to park in the first parking position based on a second instruction, where the second instruction is an instruction to ignore the first obstacle.

12. The method according to any one of claims 9 to 11, characterized in that The method further comprises: At least two different marking methods are used: occupied marking, unoccupied marking, or occupied marking with different degrees of influence.

13. The method according to any one of claims 1 to 12, characterized in that The first obstacles include dynamic obstacles and static obstacles, and the dynamic obstacles and the static obstacles are marked in different ways.

14. The method according to any one of claims 1 to 13, characterized in that The annotation includes at least one of a human-machine interface HMI annotation, a voice prompt annotation, a direction prompt sound annotation, a light annotation, and a projection annotation.

15. The method according to claim 14, wherein The HMI annotation includes at least one of the following: graphic annotation, text annotation, color annotation, and pattern annotation.

16. The method according to claim 15, wherein The direction prompt sound annotation includes: According to the position of the first obstacle relative to the first parking position, a prompt sound is emitted at a corresponding position relative to the driver in the vehicle.

17. The method according to any one of claims 1 to 16, characterized in that The method further comprises: Based on the position of the first obstacle and the surrounding environment of the vehicle, first recommendation information is provided, where the first recommendation information is used to indicate a recommended parking space adjustment direction and / or a recommended parking space adjustment position.

18. The method according to any one of claims 1 to 17, characterized in that The method further comprises: Marking the edge position of the first obstacle; and / or, A critical safety position is marked, where the critical safety position is a position that is a preset safety distance from an edge position of the first obstacle.

19. The method according to any one of claims 1 to 18, characterized in that The method further comprises: The first parking position is projected into the surroundings of the vehicle, and the first obstacle is marked in the projected first parking position.

20. A parking assistance device, characterized in that: include: a control module configured to control a display device to display a parking area, the parking area being generated based on a surrounding environment of the vehicle, and to control the display device to display a first parking position based on a first instruction; The marking module is used to mark the position of the first obstacle, where the first obstacle includes an obstacle that interacts with the first parking position.

21. The device according to claim 20, characterized in that The obstacles that interact with the first parking position include obstacles within the first parking position and / or obstacles outside the first parking position that will interact with the first parking position.

22. The device according to claim 20 or 21, characterized in that The first obstacle is displayed outside the first parking position, and the first obstacle is entirely or partially located within the first parking position. Whether the first obstacle is entirely or partially located within the first parking position is determined by the perception result of the vehicle through the on-board sensor.

23. The device according to any one of claims 20 to 22, characterized in that The annotation module is specifically used for: The first parking position is marked as occupied by an obstacle based on environmental perception performed by multiple sensors.

24. The device according to any one of claims 20 to 23, characterized in that The annotation module is specifically used for: The first parking position includes a plurality of sub-areas, and each sub-area is marked with an obstacle occupancy mark.

25. The device according to any one of claims 20 to 23, characterized in that The annotation module is specifically used for: The first parking position is partially located outside the parking area, and the first obstacle occupies a portion outside the parking area. The edge of the parking area covered by the first parking position is marked as occupied by the obstacle.

26. The device according to any one of claims 20 to 25, characterized in that The annotation module is specifically used for: The first obstacle is identified, and if the first obstacle has an impact on parking, the first obstacle is marked as occupied; if the first obstacle has no impact on parking, the first obstacle is marked as unoccupied.

27. The device according to any one of claims 20 to 26, characterized in that The annotation includes at least one of a human-machine interface HMI annotation, a voice prompt annotation, a direction prompt sound annotation, a light annotation, and a projection annotation.

28. The device according to any one of claims 20 to 27, characterized in that The annotation module is also used for: Based on the position of the first obstacle and the surrounding environment of the vehicle, first recommendation information is provided, where the first recommendation information is used to indicate a recommended parking space adjustment direction and / or parking space adjustment position.

29. The device according to any one of claims 20 to 28, characterized in that The annotation module is also used for: Marking the edge position of the first obstacle; and / or, A critical safety position is marked, where the critical safety position is a position that is a preset safety distance from an edge position of the first obstacle.

30. A parking assistance device, characterized in that: comprising a processor coupled to a memory; The processor is configured to execute the computer program or instructions stored in the memory, so as to enable the parking assistance device to perform the method according to any one of claims 1 to 19.

31. A vehicle, characterized in that: The method comprises the parking assistance device according to any one of claims 20 to 30.

32. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program or instruction, and when the program or instruction is executed, the method according to any one of claims 1 to 19 is implemented.

33. A computer program product, characterized in that The computer program product comprises a computer program code, which, when run on a computer, causes the computer to perform the method according to any one of claims 1 to 19.

Images