Parking interaction method and device, storage medium, electronic device and vehicle

Abstract

The application discloses a parking interaction method and device, a storage medium, an electronic device and a vehicle, and relates to the technical field of automobiles. The method comprises the following steps: outputting a currently generated virtual parking image and a corresponding real scene parking image; receiving a parking space selection instruction for the virtual parking image, wherein the parking space selection instruction comprises first position information of a target parking space in the virtual parking image; determining second position information of the target parking space in the real scene parking image according to the correspondence between the virtual parking image and the real scene parking image; and highlighting the target parking space in the virtual parking image according to the first position information and highlighting the target parking space in the real scene parking image according to the second position information. The application can enable a driver to intuitively obtain the correspondence between the target parking space in the virtual parking image and the real scene parking image, and the driver does not need to waste energy to make a comparison, thereby improving parking efficiency and the reliability of an automatic parking system.

Description

Technical Field

[0001] This application relates to the field of automotive technology, and more specifically, to a parking interaction method, device, storage medium, electronic device, and vehicle. Background Technology

[0002] Automated parking systems are an important component of advanced driver assistance systems (ADAS) or autonomous driving systems. Once activated, an automated parking system uses sensors to identify obstacles and available parking spaces around the vehicle, generating a virtual parking image (VR) that includes the vehicle, surrounding vehicles, and available parking spaces. The driver can then select the desired parking space from the VR image and park accordingly. However, whether parking automatically or manually, the driver must compare the VR image of the target parking space with the real physical world to determine its location within the displayed physical world before safely and confidently parking in the desired space. Therefore, for manual parking, the difference between VR images and the real physical world in visual presentation may take more time to compare, thus reducing parking efficiency. For automatic parking, this difference in visual presentation may cause comparison anxiety for the driver, resulting in a lack of security and forcing the driver to switch from automatic parking to manual parking, thereby reducing the reliability of the automatic parking system. Summary of the Invention

[0003] This application provides a parking interaction method, device, storage medium, electronic device, and vehicle, which can solve the following technical problems: During the parking process, the driver needs to compare the VR image of the target parking space with the real physical world to determine the position of the target parking space in the VR image in the displayed physical world before he can safely and confidently park in the target parking space. However, since there are differences in the visual presentation between the VR image and the real physical world, comparing the two will consume more time and cause comparison anxiety for the driver, resulting in reduced parking efficiency and reduced reliability of the automatic parking system.

[0004] The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide a parking interaction method, the method comprising:

[0006] The output displays the currently generated virtual parking image and its corresponding real-world parking image;

[0007] Receive a parking space selection instruction for a virtual parking image, the parking space selection instruction including the first location information of the target available parking space in the virtual parking image;

[0008] Based on the correspondence between virtual parking images and real-world parking images, determine the second location information of the target available parking space in the real-world parking image;

[0009] The target available parking space is highlighted in the virtual parking image based on the first location information, and the target available parking space is highlighted in the real-world parking image based on the second location information.

[0010] As can be seen from the above solution, the embodiments of this application can simultaneously display virtual parking images and their corresponding real-world parking images. When the driver selects a target parking space in the virtual parking image, the target parking space can be highlighted in both the virtual and real-world parking images. This allows the driver to intuitively obtain the correspondence between the target parking space in the virtual and real-world parking images without having to spend time comparing the virtual parking image with the real physical world. This not only improves parking efficiency but also enhances the reliability of the automatic parking system.

[0011] In a first possible implementation of the first aspect, determining the second location information of the target available parking space in the real-world parking image based on the correspondence between the virtual parking image and the real-world parking image includes:

[0012] Obtain the target radar point cloud corresponding to the first location information, and obtain the depth map corresponding to the real-scene parking image;

[0013] Find the target depth information corresponding to the target radar point cloud from the depth map;

[0014] The location information at the target depth information is determined as the second location information.

[0015] In the second possible implementation of the first aspect, before determining the position information at the target depth information as the second position information, the method further includes:

[0016] Based on computer vision algorithms, available parking spaces in real-world parking images are identified as candidate available parking spaces;

[0017] Determining the location information at the target depth information as the second location information includes:

[0018] If the depth information of the candidate parking space includes the target depth information, the location information at the target depth information is determined as the second location information.

[0019] As can be seen from the above scheme, the embodiments of this application can achieve distance (or depth) matching by matching the target radar point cloud corresponding to the first location information in the virtual parking image and the depth map corresponding to the real-scene parking image to obtain the target depth information corresponding to the target radar point cloud. Only when the target depth information is determined to be the depth information of a candidate parking space in the real-scene parking image is the location information at the target depth information determined as the second location information of the target parking space in the real-scene parking image. This can further ensure that there is a parking space at the determined second location information, rather than other locations, such as a parked parking space, lane, or other open space, thereby improving the accuracy of determining the second location information.

[0020] In the third possible implementation of the first aspect, the method also includes:

[0021] If the depth information of the candidate parking space does not include the target depth information, the features of the first reference object are determined based on the radar point cloud surrounding the target radar point cloud.

[0022] Based on computer vision algorithms, identify the features of second reference objects around each candidate parking space;

[0023] The location information of the target candidate parking space is determined as the second location information. The target candidate parking space is the candidate parking space corresponding to the second reference feature that successfully matches the first reference feature.

[0024] As can be seen from the above solution, when the second position information cannot be accurately obtained by matching based on distance (or depth), the second position information can be determined by matching the features of the reference object, thereby further improving the accuracy of determining the second position information.

[0025] In a fourth possible implementation of the first aspect, after highlighting the target available parking space in the virtual parking image based on the first location information and in the real-world parking image based on the second location information, the method further includes:

[0026] During the parking phase, the planned path from the vehicle to the second location information is highlighted in the real-view parking image.

[0027] In the fifth possible implementation of the first aspect, the output displaying the currently generated virtual parking image includes:

[0028] After receiving the automatic parking system activation command, it is determined whether there is a first vehicle component. The first vehicle component is a vehicle component whose current physical state does not meet the automatic parking conditions.

[0029] In the presence of a first vehicle component, the virtual state of the first vehicle component of the vehicle model in the initially generated virtual parking image is adjusted to the current physical state of the first vehicle component;

[0030] The output displays a virtual parking image after status adjustment, highlighting the first vehicle component;

[0031] The method also includes:

[0032] Output the first parking prompt message, which is used to prompt the driver to adjust the current physical state of the first vehicle component to the state that meets the automatic parking conditions.

[0033] As can be seen from the above scheme, when the conditions for automatic parking are not met, in addition to providing the driver with the first parking prompt information, the virtual state of the first vehicle component in the initially generated virtual parking image can be adjusted to the current physical state of the first vehicle component. When outputting and displaying the virtual parking image with the adjusted state, the first vehicle component is highlighted. This not only ensures that the state of the vehicle model is consistent with that of the real-world vehicle, providing the driver with an immersive experience, but also allows the driver to combine the first parking prompt information with the highlighted first vehicle component, intuitively and quickly seeing which first vehicle component does not meet the automatic parking conditions and how to adjust it, thereby improving the efficiency of the driver in adjusting the first vehicle component and thus improving parking efficiency.

[0034] In a sixth possible implementation of the first aspect, after highlighting the target available parking space in the virtual parking image based on the first location information and in the real-world parking image based on the second location information, the method further includes:

[0035] When a second vehicle component exists during the parking phase, the virtual state of the second vehicle component in the virtual parking image is updated as the physical state of the second vehicle component changes. The second vehicle component is highlighted when the updated virtual parking image is displayed; the second vehicle component is the one whose physical state undergoes visually dynamic changes during the parking process. And / or,

[0036] If a third vehicle component is present during the parking phase, the third vehicle component is highlighted when displaying the currently generated virtual parking image, and a second parking prompt message is output. The third vehicle component is the vehicle component where the parking operation error occurred, and the second parking prompt message is used to remind the driver of the parking operation error.

[0037] As demonstrated by the above scheme, during the parking phase, by ensuring that the virtual state of the second vehicle component in the vehicle model matches the physical state of the real-world vehicle and highlighting the second vehicle component, the driver can intuitively view the changes in the vehicle's status regardless of whether they are inside the car, thus providing an immersive parking interaction experience. Furthermore, when the driver makes a parking error, highlighting the third vehicle component where the error occurred on the vehicle model and providing a second parking prompt allows the driver to quickly and intuitively locate the incorrect parking operation and correct it promptly, thereby improving parking efficiency.

[0038] In the seventh possible implementation of the first aspect, the output displays the currently generated virtual parking image and its corresponding real-world parking image, including:

[0039] After the automatic parking system is started, the currently generated virtual parking image is displayed.

[0040] When a parking search command is received or the gear is switched to reverse (R), the system displays both the currently generated virtual parking image and the currently generated real-world parking image.

[0041] In the eighth possible implementation of the first aspect, the real-world parking image is located above the virtual parking image, and the display positions of the real-world parking image and the virtual parking image do not overlap.

[0042] Secondly, embodiments of this application provide a parking interaction device, the device comprising:

[0043] The output display unit is used to output and display the currently generated virtual parking image and its corresponding real-world parking image;

[0044] The receiving unit is used to receive a parking space selection instruction for a virtual parking image. The parking space selection instruction includes the first location information of the target available parking space in the virtual parking image.

[0045] The determining unit is used to determine the second location information of the target parking space in the real-scene parking image based on the correspondence between the virtual parking image and the real-scene parking image;

[0046] The highlighting unit is used to highlight the target available parking space in a virtual parking image based on the first location information, and to highlight the target available parking space in a real-world parking image based on the second location information.

[0047] As can be seen from the above solution, the embodiments of this application can simultaneously display virtual parking images and their corresponding real-world parking images. When the driver selects a target parking space in the virtual parking image, the target parking space can be highlighted in both the virtual and real-world parking images. This allows the driver to intuitively obtain the correspondence between the target parking space in the virtual and real-world parking images without having to spend time comparing the virtual parking image with the real physical world. This not only improves parking efficiency but also enhances the reliability of the automatic parking system.

[0048] In a first possible implementation of the second aspect, the determined unit includes:

[0049] The acquisition module is used to acquire the target radar point cloud corresponding to the first location information, and to acquire the depth map corresponding to the real-scene parking image;

[0050] The search module is used to find the target depth information corresponding to the target radar point cloud from the depth map;

[0051] The first determining module is used to determine the position information at the target depth information as the second position information.

[0052] In a second possible implementation of the second aspect, the determining unit further includes:

[0053] The identification module is used to identify available parking spaces in the real-world parking image as candidate available parking spaces based on computer vision algorithms before determining the location information at the target depth information as the second location information.

[0054] The first determining module is used to determine the location information at the target depth information as the second location information when the depth information of the candidate parking space includes the target depth information.

[0055] In a third possible implementation of the second aspect, the determining unit further includes:

[0056] The second determining module is used to determine the features of the first reference object based on the radar point cloud around the target radar point cloud when the depth information of the candidate parking space does not contain the target depth information.

[0057] The recognition module is also used to identify features of a second reference object around each candidate parking space based on computer vision algorithms;

[0058] The first determining module is further configured to determine the location information of the target candidate parking space as the second location information, wherein the target candidate parking space is the candidate parking space corresponding to the second reference feature that successfully matches the first reference feature.

[0059] In a fourth possible implementation of the second aspect, the highlighting unit is further configured to, during the parking phase, highlight the planned path from the vehicle to the second location information in the real-world parking image after highlighting the target available parking space in the virtual parking image based on the first location information and highlighting the target available parking space in the real-world parking image based on the second location information.

[0060] In the fifth possible implementation of the second aspect, the output display unit is used to determine whether a first vehicle component exists after receiving the automatic parking system activation command. The first vehicle component is a vehicle component whose current physical state does not meet the automatic parking conditions. If the first vehicle component exists, the virtual state of the first vehicle component in the initially generated virtual parking image of the vehicle model is adjusted to the current physical state of the first vehicle component. The adjusted virtual parking image is output and displayed, and the first vehicle component is highlighted.

[0061] The device also includes:

[0062] The first output prompt unit is used to output the first parking prompt information, which prompts the driver to adjust the current physical state of the first vehicle component to meet the conditions for automatic parking.

[0063] In a sixth possible implementation of the second aspect, the highlighting unit is further configured to, after highlighting the target available parking space in the virtual parking image based on the first location information and in the real-world parking image based on the second location information, update the virtual state of the second vehicle component of the vehicle model in the virtual parking image as the entity state of the second vehicle component changes, and highlight the second vehicle component when displaying the updated virtual parking image, wherein the second vehicle component is a vehicle component whose entity state undergoes a visually dynamic change during the parking process; and / or, in the case of a third vehicle component in the parking stage, highlight the third vehicle component when displaying the currently generated virtual parking image, wherein the third vehicle component is a vehicle component that has experienced a parking operation error;

[0064] The device also includes:

[0065] The second output prompt unit is used to output a second parking prompt message when the third vehicle component is highlighted. The second parking prompt message is used to prompt the driver for parking operation errors.

[0066] In the seventh possible implementation of the second aspect, the output display unit includes:

[0067] The first output display module is used to output and display the currently generated virtual parking image after the automatic parking system is started;

[0068] The second output display module is used to output and display the currently generated real-world parking image in addition to the currently generated virtual parking image when a parking search command is received or the gear is switched to R.

[0069] In the eighth possible implementation of the second aspect, the real-world parking image is located above the virtual parking image, and the display positions of the real-world parking image and the virtual parking image do not overlap.

[0070] Thirdly, embodiments of this application provide a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any possible implementation of the first aspect.

[0071] Fourthly, embodiments of this application provide an electronic device, which includes:

[0072] One or more processors;

[0073] Storage device for storing one or more programs.

[0074] When one or more programs are executed by one or more processors, the electronic device performs the method as described in any possible implementation of the first aspect.

[0075] Fifthly, embodiments of this application provide a vehicle that includes the means as described in any possible implementation of the second aspect, or includes electronic equipment as described in the fourth aspect. Attached Figure Description

[0076] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0077] Figure 1 A flowchart illustrating a parking interaction method provided in an embodiment of this application;

[0078] Figure 2 An example image of a parking interaction provided in an embodiment of this application;

[0079] Figure 3 Another example image of parking interaction provided in the embodiments of this application;

[0080] Figure 4 An example diagram of a vehicle model in a virtual parking image provided in this application embodiment;

[0081] Figure 5A block diagram illustrating the composition of a parking interaction device provided in an embodiment of this application;

[0082] Figure 6 This is a block diagram illustrating the composition of a vehicle as provided in an embodiment of this application. Detailed Implementation

[0083] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0084] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0085] Figure 1 This is a flowchart illustrating a parking interaction method. This method can be applied to electronic devices or computer devices, specifically to vehicles or other electronic devices that interact with vehicles (such as mobile terminals). The method may include the following steps:

[0086] S110: Output displays the currently generated virtual parking image and its corresponding real-world parking image.

[0087] Virtual parking images utilize radar to identify obstacles and available parking spaces around the vehicle, generating a VR image that includes the vehicle, occupied parking spaces, and available parking spaces. Real-world parking images are images of the view in front of (or behind) the vehicle captured by a camera. The radar includes lidar and millimeter-wave radar, and the cameras include monocular and binocular cameras.

[0088] When this application is applied to a vehicle, the vehicle can generate a virtual parking image and its corresponding real-world parking image, which can then be displayed on the in-vehicle central control screen or on the windshield via a HUD (Head-Up Display). When this application is applied to other electronic devices that interact with the vehicle (such as a mobile terminal), after the vehicle generates the virtual parking image and its corresponding real-world parking image, the virtual and real-world parking images are sent to the other electronic device, which then outputs and displays them.

[0089] Virtual parking images and real-world parking images can be arranged vertically or horizontally. Real-world parking images can be located above, below, or on the same layer as virtual parking images, as long as their display positions do not overlap. Here, "above," "below," and "on the same layer" refer to different layers.

[0090] In one implementation, since the difficulty and need for the driver to compare the virtual parking image with the real world increases when searching for a parking space or reversing after activating the automatic parking system, in order to save resources while improving the driver's parking experience and parking efficiency, the system can output and display the currently generated virtual parking image after activation; when a parking space search command is received or the gear is shifted to reverse, the system can output and display the currently generated real-world parking image in addition to the currently generated virtual parking image. In this case, the real-world parking image can be directly output on top of the virtual parking image, improving the output efficiency of the dual images.

[0091] S120: Receive a parking space selection instruction for a virtual parking image, the parking space selection instruction including the first location information of the target available parking space in the virtual parking image.

[0092] In practical applications, each available parking space in the virtual parking image can be set as a clickable control for users to select, or a button can be placed above each available parking space for users to select, or each available parking space can be numbered, allowing users to select using the number entered on the display device's input buttons. Regardless of the human-computer interaction method, the control or number can be bound to the location information of the corresponding available parking space. When the vehicle receives the user's parking space selection command, it can obtain the initial location information of the target available parking space in the virtual parking image.

[0093] S130: Based on the correspondence between the virtual parking image and the real-world parking image, determine the second location information of the target available parking space in the real-world parking image.

[0094] Whether it's a virtual parking image or a real-world parking image, the depth information and object features contained in the images correspond. For example, for the same parking space, the distance from the car to that space can be represented by the depth information in both the virtual and real-world parking images. Therefore, based on the correspondence between the virtual and real-world parking images, the second location information of the target available parking space in the real-world parking image can be determined.

[0095] Specifically, the second location information can be determined using distance (or depth) matching. First, the target radar point cloud corresponding to the first location information and the depth map corresponding to the real-world parking image are obtained. Then, the target depth information corresponding to the target radar point cloud is found in the depth map. Finally, the location information at the target depth information is determined as the second location information. The depth map includes depth information from the camera to the target object. A monocular camera can acquire the depth information contained in the real-world parking image through relative distance movement, while a binocular camera can directly acquire the depth information contained in the real-world parking image.

[0096] In one implementation, before determining the location information at the target depth information as the second location information, parking spaces in the real-world parking image can be identified as candidate parking spaces based on computer vision algorithms; if the depth information of the candidate parking spaces includes the target depth information, the location information at the target depth information is then determined as the second location information.

[0097] Identifying available parking spaces in real-world parking images using computer vision algorithms includes: identifying available parking spaces in real-world parking images based on a parking line recognition model. This parking line recognition model can be trained on a large number of historical real-world parking images marked with available parking line information using a neural network.

[0098] This application embodiment can achieve distance (or depth) matching by matching the target radar point cloud corresponding to the first location information in the virtual parking image with the depth map corresponding to the real-scene parking image to obtain the target depth information corresponding to the target radar point cloud. Only when the target depth information is determined to be the depth information of a candidate parking space in the real-scene parking image is the location information at the target depth information determined as the second location information of the target parking space in the real-scene parking image. This can further ensure that there is a parking space at the determined second location information, rather than other locations, such as a parked parking space, lane, or other open space, thereby improving the accuracy of determining the second location information.

[0099] In one implementation, when the depth information of the candidate parking space does not include the target depth information, this embodiment of the application can further determine a first reference feature based on the radar point cloud surrounding the target radar point cloud; identify a second reference feature around each candidate parking space based on a computer vision algorithm; and determine the position information of the target candidate parking space as the second position information. The target candidate parking space is the candidate parking space corresponding to the second reference feature that successfully matches the first reference feature. Here, "surrounding" can be a circular area with radius R centered on the target radar point cloud or the candidate parking space, or it can be an area of ​​other shapes; this embodiment of the application does not limit this. Reference feature includes features describing the size and shape of the reference object. For example, when the depth information of the candidate parking space does not include the target depth information, auxiliary matching can be performed using surrounding reference objects such as pillars to further determine the second position information.

[0100] In cases where matching based on distance (or depth) cannot accurately obtain the second location information, the embodiments of this application can determine the second location information by matching the features of a reference object, thereby further improving the accuracy of determining the second location information.

[0101] S140: Highlight the target available parking space in the virtual parking image based on the first location information, and highlight the target available parking space in the real-world parking image based on the second location information.

[0102] Highlighting includes methods such as highlighting, changing line colors, and adding markers to allow drivers to intuitively and quickly see the target available parking space. For example... Figure 2 As shown, the left side is a virtual parking image, and the right side is a real-world parking image. The target available parking spaces are highlighted and bolded, and a parking sign "P" is added to the target available parking spaces, so that the driver can quickly see the correspondence between the two.

[0103] The parking interaction method provided in this application can simultaneously display a virtual parking image and its corresponding real-world parking image. When the driver selects a target parking space in the virtual parking image, the target parking space can be highlighted in both the virtual and real-world parking images. This allows the driver to intuitively understand the correspondence between the target parking space in the virtual and real-world parking images without having to spend time comparing the virtual parking image with the real physical world. This not only improves parking efficiency but also enhances the reliability of the automatic parking system.

[0104] In one implementation, to allow drivers to intuitively and in advance know the actual driving path from their vehicle to the target available parking space, thereby further enhancing the driver's immersive parking experience, this application embodiment can highlight the target available parking space in the virtual parking image based on first location information, and then highlight the target available parking space in the real-world parking image based on second location information. During the parking phase, the planned path from the vehicle to the second location information is then highlighted in the real-world parking image. The method for highlighting this planned path can be the same as or different from that for highlighting the target available parking space. To avoid affecting the driver's visual experience, a white, slightly transparent layer can be overlaid on the road surface in the real-world parking image.

[0105] In one implementation, when the driver activates the automatic parking system, subsequent parking operations may fail unsuccessfully and safely due to reasons such as doors not being closed or trunks not being closed. To address this technical problem, this application embodiment can, upon receiving the automatic parking system activation command, determine whether a first vehicle component exists. The first vehicle component is a vehicle component whose current physical state does not meet the automatic parking conditions. If the first vehicle component exists, the virtual state of the first vehicle component in the initially generated virtual parking image is adjusted to the current physical state of the first vehicle component. The adjusted virtual parking image is output and displayed, highlighting the first vehicle component. A first parking prompt message is also output to prompt the driver to adjust the current physical state of the first vehicle component to meet the automatic parking conditions.

[0106] The first vehicle component includes doors, trunk, etc. The display method for highlighting the first vehicle component may be the same as or different from the aforementioned method for highlighting the target available parking space. The first parking prompt information may be displayed as text in a blank space of the virtual parking image, or it may be output in the form of voice, text message, etc.

[0107] For example, such as Figure 3 As shown, the front left door of the vehicle model in the virtual parking image is open and highlighted. The virtual parking image also displays the first parking prompt message "Please close the door". Thus, the driver can intuitively and quickly understand that the front left door needs to be closed before automatic parking can be performed.

[0108] The parking interaction method provided in this application embodiment can, when the automatic parking conditions are not met, not only provide the driver with a first parking prompt, but also adjust the virtual state of the first vehicle component of the self-vehicle model in the initially generated virtual parking image to the current physical state of the first vehicle component. When outputting and displaying the virtual parking image with the adjusted state, the first vehicle component is highlighted. This not only ensures that the self-vehicle model is consistent with the state of the real-world vehicle, providing the driver with an immersive experience, but also allows the driver to combine the first parking prompt and the highlighted first vehicle component to intuitively and quickly see which first vehicle component does not meet the automatic parking conditions and how to adjust it, thereby improving the efficiency of the driver in adjusting the first vehicle component and thus improving parking efficiency.

[0109] In one embodiment, after highlighting the target available parking space in a virtual parking image based on the first location information and in a real-world parking image based on the second location information, the present application embodiments further provide the following method:

[0110] When a second vehicle component exists during the parking phase, the virtual state of the second vehicle component in the virtual parking image is updated as the physical state of the second vehicle component changes. The second vehicle component is highlighted when the updated virtual parking image is displayed; the second vehicle component is the one whose physical state undergoes visually dynamic changes during the parking process. And / or,

[0111] If a third vehicle component is present during the parking phase, the third vehicle component is highlighted when displaying the currently generated virtual parking image, and a second parking prompt message is output. The third vehicle component is the vehicle component where the parking operation error occurred, and the second parking prompt message is used to remind the driver of the parking operation error.

[0112] The second vehicle component includes the steering wheel, and the third vehicle component includes the brake pedal, accelerator pedal, etc. The second parking prompt information can be displayed as text in a blank area of ​​the virtual parking image, or output via voice, SMS, etc. Unlike highlighting a target available parking space, the second and third vehicle components may be internal vehicle components. Therefore, when highlighting a second or third vehicle component, the displayed vehicle model range may not be a panoramic view, but rather a partial field of view including the second or third vehicle component. However, the display format can be the same as highlighting a target available parking space, such as highlighting. In other words, when the second or third vehicle component is an exterior component, the vehicle model displays a panoramic view; when the second or third vehicle component is an interior component, the vehicle model can be a partial field of view including the second or third vehicle component.

[0113] For example, such as Figure 4 As shown, during the parking phase, the steering wheel in the virtual car model rotates in sync with the steering wheel of the physical vehicle, and can also be highlighted, displaying a partial field of view when the virtual car model is shown. Furthermore, if the driver needs to use the brake or accelerator pedals during parking, and the driver makes a mistake, the brake or accelerator pedal can be highlighted in the virtual parking image's virtual car model, along with a textual warning about the parking error. This allows the driver to quickly identify the error and correct it promptly.

[0114] The parking interaction method provided in this application embodiment can, during the parking phase, ensure that the virtual state of the second vehicle component of the self-vehicle model remains consistent with the physical state of the real-world vehicle, and highlight the second vehicle component. This allows the driver to intuitively view the changes in the self-vehicle's state regardless of whether they are inside the vehicle, thus providing an immersive parking interaction experience. Furthermore, when the driver makes a parking error during the parking phase, by highlighting the third vehicle component where the error occurred on the self-vehicle model and providing a second parking prompt, the driver can intuitively and quickly locate the incorrect parking operation and correct it promptly, thereby improving parking efficiency.

[0115] Corresponding to the above method embodiments, another embodiment of this application provides a parking interaction device, such as... Figure 5 As shown, the device includes:

[0116] Output display unit 20 is used to output and display the currently generated virtual parking image and its corresponding real-world parking image;

[0117] The receiving unit 22 is used to receive a parking space selection instruction for a virtual parking image. The parking space selection instruction includes the first location information of the target available parking space in the virtual parking image.

[0118] The determining unit 24 is used to determine the second location information of the target parking space in the real-scene parking image based on the correspondence between the virtual parking image and the real-scene parking image;

[0119] The highlighting unit 26 is used to highlight the target available parking space in the virtual parking image based on the first location information, and to highlight the target available parking space in the real-world parking image based on the second location information.

[0120] In one embodiment, the determining unit 24 includes:

[0121] The acquisition module is used to acquire the target radar point cloud corresponding to the first location information, and to acquire the depth map corresponding to the real-scene parking image;

[0122] The search module is used to find the target depth information corresponding to the target radar point cloud from the depth map;

[0123] The first determining module is used to determine the position information at the target depth information as the second position information.

[0124] In one embodiment, the determining unit 24 further includes:

[0125] The identification module is used to identify available parking spaces in the real-world parking image as candidate available parking spaces based on computer vision algorithms before determining the location information at the target depth information as the second location information.

[0126] The first determining module is used to determine the location information at the target depth information as the second location information when the depth information of the candidate parking space includes the target depth information.

[0127] In one embodiment, the determining unit 24 further includes:

[0128] The second determining module is used to determine the features of the first reference object based on the radar point cloud around the target radar point cloud when the depth information of the candidate parking space does not contain the target depth information.

[0129] The recognition module is also used to identify features of a second reference object around each candidate parking space based on computer vision algorithms;

[0130] The first determining module is further configured to determine the location information of the target candidate parking space as the second location information, wherein the target candidate parking space is the candidate parking space corresponding to the second reference feature that successfully matches the first reference feature.

[0131] In one embodiment, the highlighting unit 26 is further configured to, during the parking phase, highlight the planned path from the vehicle to the second location information in the real-world parking image after highlighting the target available parking space in the virtual parking image based on the first location information and highlighting the target available parking space in the real-world parking image based on the second location information.

[0132] In one embodiment, the output display unit 20 includes:

[0133] After receiving the automatic parking system activation command, it is determined whether there is a first vehicle component. The first vehicle component is a vehicle component whose current physical state does not meet the automatic parking conditions.

[0134] In the presence of a first vehicle component, the virtual state of the first vehicle component of the vehicle model in the initially generated virtual parking image is adjusted to the current physical state of the first vehicle component;

[0135] The output displays a virtual parking image after status adjustment, highlighting the first vehicle component;

[0136] The device also includes:

[0137] The first output prompt unit is used to output the first parking prompt information, which prompts the driver to adjust the current physical state of the first vehicle component to meet the conditions for automatic parking.

[0138] In one embodiment, the highlighting unit 26 is further configured to, after highlighting the target available parking space in the virtual parking image based on the first location information and highlighting the target available parking space in the real-world parking image based on the second location information, if a second vehicle component exists during the parking phase, update the virtual state of the second vehicle component in the virtual parking image as the physical state of the second vehicle component changes, and highlight the second vehicle component when displaying the updated virtual parking image, wherein the second vehicle component is a vehicle component whose physical state undergoes a visually dynamic change during the parking process; and / or, if a third vehicle component exists during the parking phase, highlight the third vehicle component when displaying the currently generated virtual parking image, wherein the third vehicle component is a vehicle component that has experienced a parking operation error;

[0139] The device also includes:

[0140] The second output prompt unit is used to output a second parking prompt message when the third vehicle component is highlighted. The second parking prompt message is used to prompt the driver for parking operation errors.

[0141] In one embodiment, the output display unit 20 includes:

[0142] The first output display module is used to output and display the currently generated virtual parking image after the automatic parking system is started;

[0143] The second output display module is used to output and display the currently generated real-world parking image in addition to the currently generated virtual parking image when a parking search command is received or the gear is switched to R.

[0144] In one implementation, the real-world parking image is located above the virtual parking image, and the display positions of the real-world parking image and the virtual parking image do not overlap.

[0145] The parking interaction device provided in this application embodiment can simultaneously display a virtual parking image and its corresponding real-world parking image. When the driver selects a target parking space in the virtual parking image, the target parking space can be highlighted in both the virtual and real-world parking images. This allows the driver to intuitively understand the correspondence between the target parking space in the virtual and real-world parking images without having to spend time comparing the virtual parking image with the real physical world. This not only improves parking efficiency but also enhances the reliability of the automatic parking system.

[0146] Based on the above method embodiments, another embodiment of this application provides a storage medium storing executable instructions thereon, which, when executed by a processor, cause the processor to implement the method described in any of the above embodiments.

[0147] Based on the above method embodiments, another embodiment of this application provides an electronic device or computer device, including:

[0148] One or more processors;

[0149] Storage device for storing one or more programs.

[0150] When the one or more programs are executed by the one or more processors, the electronic device or computer device performs the method as described in any of the above embodiments.

[0151] Based on the above method embodiments, another embodiment of this application provides a vehicle that includes the apparatus as described in any of the above embodiments, or includes electronic devices as described above.

[0152] like Figure 6As shown, the vehicle includes a display device 30, an ECU (Electronic Control Unit) 32, a radar 34, and a camera 36. The radar 34 generates radar point clouds, the camera 36 captures real-world parking images, and the ECU 32 generates virtual parking images based on the radar point clouds. The ECU 32 then transmits both the virtual and real-world parking images to the display device 30 for output display. The display device 30 receives parking space selection instructions based on the virtual parking image. These instructions include the first position information of the target available parking space in the virtual parking image. The display device 30 sends the parking space selection instructions to the ECU 32, which determines the second position information of the target available parking space in the real-world parking image based on the correspondence between the virtual and real-world parking images. The ECU 32 then sends both the first and second position information to the display device 30. The display device 30 highlights the target available parking space in the virtual parking image based on the first position information and in the real-world parking image based on the second position information. The display device 30 includes an in-vehicle central control screen, a windshield based on HUD, etc.

[0153] The vehicle may also include: a GPS (Global Positioning System) positioning device, a T-Box (Telematics Box), and a V2X (Vehicle-to-Everything) module. The GPS positioning device is used to obtain the vehicle's current geographical location; the T-Box can act as a gateway to communicate with external devices; and the V2X module is used to communicate with other vehicles, roadside equipment, etc.

[0154] The above-described apparatus embodiments correspond to the method embodiments and have the same technical effects. For detailed descriptions, please refer to the method embodiments. The apparatus embodiments are derived from the method embodiments; detailed descriptions can be found in the method embodiments section, and will not be repeated here. Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application.

[0155] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A parking interaction method, characterized in that, The method includes: The output displays the currently generated virtual parking image and its corresponding real-world parking image. The virtual parking image is generated by using radar to identify obstacles and available parking spaces around the vehicle and to generate a VR image containing the vehicle, occupied parking spaces, and available parking spaces. The real-world parking image is a view of the front or rear of the vehicle captured by a camera. Receive a parking space selection instruction for the virtual parking image, the parking space selection instruction including the first location information of the target available parking space in the virtual parking image; Based on the correspondence between the virtual parking image and the real-world parking image, the second location information of the target available parking space in the real-world parking image is determined; The target available parking space is highlighted in the virtual parking image based on the first location information, and the target available parking space is highlighted in the real-world parking image based on the second location information; The step of determining the second location information of the target available parking space in the real-world parking image based on the correspondence between the virtual parking image and the real-world parking image includes: Obtain the target radar point cloud corresponding to the first location information, and obtain the depth map corresponding to the real-scene parking image; Find the target depth information corresponding to the target radar point cloud from the depth map; The location information at the target depth information is determined as the second location information.

2. The method according to claim 1, characterized in that, Before determining the location information at the target depth information as the second location information, the method further includes: The available parking spaces in the real-world parking image are identified as candidate available parking spaces based on computer vision algorithms; The step of determining the position information at the target depth information as the second position information includes: If the depth information of the candidate parking space includes the target depth information, the location information at the target depth information is determined as the second location information.

3. The method according to claim 2, characterized in that, The method further includes: If the depth information of the candidate parking space does not include the depth information of the target, the first reference object feature is determined based on the radar point cloud surrounding the target radar point cloud. Based on the computer vision algorithm, the features of the second reference object around each of the candidate parking spaces are identified; The location information of the target candidate parking space is determined as the second location information, wherein the target candidate parking space is the candidate parking space corresponding to the second reference feature that successfully matches the first reference feature.

4. The method according to claim 1, characterized in that, After highlighting the target available parking space in the virtual parking image based on the first location information, and highlighting the target available parking space in the real-world parking image based on the second location information, the method further includes: During the parking phase, the planned path from the vehicle to the second location information is highlighted in the real-view parking image.

5. The method according to claim 1, characterized in that, The output display of the currently generated virtual parking image includes: After receiving the automatic parking system activation command, it is determined whether there is a first vehicle component. The first vehicle component is a vehicle component whose current physical state does not meet the automatic parking conditions. In the presence of the first vehicle component, the virtual state of the first vehicle component of the vehicle model in the initially generated virtual parking image is adjusted to the current physical state of the first vehicle component; Output the virtual parking image after the status adjustment, and highlight the first vehicle component; The method further includes: Output a first parking prompt message, which is used to prompt the driver to adjust the current physical state of the first vehicle component to a state that meets the automatic parking conditions.

6. The method according to claim 1, characterized in that, After highlighting the target available parking space in the virtual parking image based on the first location information, and highlighting the target available parking space in the real-world parking image based on the second location information, the method further includes: When a second vehicle component is present during the parking phase, the virtual state of the second vehicle component in the virtual parking image is updated as the physical state of the second vehicle component changes. When displaying the updated virtual parking image, the second vehicle component is highlighted. The second vehicle component is the one whose physical state undergoes a visually dynamic change during the parking process; and / or, If a third vehicle component is present during the parking phase, the third vehicle component is highlighted when displaying the currently generated virtual parking image, and a second parking prompt message is output. The third vehicle component is the vehicle component where the parking operation error occurred, and the second parking prompt message is used to remind the driver of the parking operation error.

7. The method according to claim 1, characterized in that, The output displays the currently generated virtual parking image and its corresponding real-world parking image, including: After the automatic parking system is started, the currently generated virtual parking image is displayed. When a parking search command is received or the gear is switched to reverse (R), the system outputs and displays the currently generated real-world parking image in addition to the currently generated virtual parking image.

8. The method according to any one of claims 1-7, characterized in that, The real-view parking image is located above the virtual parking image, and the display positions of the real-view parking image and the virtual parking image do not overlap.

9. A parking interaction device, characterized in that, The device includes: The output display unit is used to output and display the currently generated virtual parking image and its corresponding real-scene parking image. The virtual parking image is generated by using radar to identify obstacles and available parking spaces around the vehicle and generate a VR image containing the vehicle, occupied parking spaces, and available parking spaces. The real-scene parking image is a view of the front or rear of the vehicle captured by a camera. The receiving unit is configured to receive a parking space selection instruction for the virtual parking image, wherein the parking space selection instruction includes the first location information of the target available parking space in the virtual parking image. The determining unit is used to determine the second location information of the target parking space in the real-scene parking image based on the correspondence between the virtual parking image and the real-scene parking image; The highlighting unit is configured to highlight the target available parking space in the virtual parking image based on the first location information, and to highlight the target available parking space in the real-world parking image based on the second location information; The determining unit includes: The acquisition module is used to acquire the target radar point cloud corresponding to the first location information and the depth map corresponding to the real-scene parking image; The search module is used to search for the target depth information corresponding to the target radar point cloud from the depth map; The first determining module is used to determine the position information at the target depth information as the second position information.

10. The apparatus according to claim 9, characterized in that, The determining unit further includes: The identification module is used to identify available parking spaces in the real-world parking image as candidate available parking spaces based on a computer vision algorithm before determining the location information at the target depth information as the second location information. The first determining module is used to determine the location information at the target depth information as the second location information when the depth information of the candidate parking space includes the target depth information.

11. The apparatus according to claim 10, characterized in that, The determining unit further includes: The second determining module is used to determine the first reference object feature based on the radar point cloud around the target radar point cloud when the depth information of the candidate parking space does not include the target depth information. The identification module is also used to identify features of a second reference object around each of the candidate parking spaces based on the computer vision algorithm. The first determining module is further configured to determine the location information of the target candidate parking space as the second location information, wherein the target candidate parking space is the candidate parking space corresponding to the second reference feature that successfully matches the first reference feature.

12. The apparatus according to claim 9, characterized in that, The highlighting unit is further configured to, during the parking phase, highlight the planned path from the vehicle to the second location information in the real-world parking image after highlighting the target available parking space in the virtual parking image based on the first location information and highlighting the target available parking space in the real-world parking image based on the second location information.

13. The apparatus according to claim 9, characterized in that, The output display unit is used to determine whether a first vehicle component exists after receiving an automatic parking system activation command. The first vehicle component is a vehicle component whose current physical state does not meet the automatic parking conditions. If the first vehicle component exists, the virtual state of the first vehicle component in the initially generated virtual parking image of the vehicle model is adjusted to the current physical state of the first vehicle component. Output the virtual parking image after the status adjustment, and highlight the first vehicle component; The device further includes: The first output prompt unit is used to output first parking prompt information, which prompts the driver to adjust the current physical state of the first vehicle component to a state that meets the automatic parking conditions.

14. The apparatus according to claim 9, characterized in that, The highlighting unit is further configured to, after highlighting the target available parking space in the virtual parking image according to the first location information and in the real-world parking image according to the second location information, update the virtual state of the second vehicle component of the vehicle model in the virtual parking image as the entity state of the second vehicle component changes during the parking phase, and highlight the second vehicle component when displaying the updated virtual parking image, wherein the second vehicle component is a vehicle component whose entity state undergoes a visually dynamic change during the parking process; and / or, when a third vehicle component exists during the parking phase, highlight the third vehicle component when displaying the currently generated virtual parking image, wherein the third vehicle component is a vehicle component that has experienced a parking operation error; The device further includes: The second output prompt unit is used to output a second parking prompt message when the third vehicle component is highlighted. The second parking prompt message is used to prompt the driver for parking operation errors.

15. The apparatus according to any one of claims 9-14, characterized in that, The output display unit includes: The first output display module is used to output and display the currently generated virtual parking image after the automatic parking system is started; The second output display module is used to output and display the currently generated real-world parking image in addition to the currently generated virtual parking image when a parking search command is received or the gear is switched to R.

16. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-8.

17. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the electronic device performs the method as described in any one of claims 1-8.

18. A vehicle, characterized in that, The vehicle includes the device as described in any one of claims 9-15, or the electronic device as described in claim 17.

Images