Navigation method, apparatus, and vehicle

Abstract

A navigation method, apparatus, and vehicle are disclosed. The method includes: acquiring a first instruction indicating a first intention, the first intention indicating an action desired by a user in the vehicle's cabin to be performed by the vehicle in a first target area; wherein the first target area is located outside the vehicle's cabin; acquiring environmental perception information indicating the positions of objects and areas included within a first range of the vehicle; determining the position of a first area based on the environmental perception information and the first instruction, the first area corresponding to the first target area; and controlling the vehicle to drive towards the first area. This technical solution can be applied to the field of intelligent cockpits in intelligent vehicles such as new energy vehicles and electric vehicles, as well as the field of intelligent driving. It can control the vehicle to drive to any drivable area indicated by the user's voice command and / or body gesture, helping to improve navigation efficiency and achieving "anything visible is drivable".

Description

Technical Field

[0001] This application relates to the field of smart cockpits, and more specifically, to a navigation method, device, and vehicle. Background Technology

[0002] With the electrification and intelligentization of vehicles, an increasing number of intelligent driving technologies have been developed to alleviate driver stress and improve driving convenience. Currently, when users want to drive to a specific location, they can typically input that location information through a human-machine interface or voice commands. However, when users input non-standard place names or vague keywords, the vehicle's infotainment system often fails to accurately match the target location, resulting in low driving efficiency and a poor user experience.

[0003] Therefore, a navigation solution that can improve navigation efficiency and user experience is urgently needed. Summary of the Invention

[0004] This application provides a navigation method, device, and vehicle that can control the vehicle to drive to any drivable area indicated by the user's voice commands and / or body gestures, thereby improving navigation efficiency and achieving "anything visible is drivable".

[0005] Firstly, a navigation method is provided, which can be executed by a device applied in a vehicle, for example, by a chip or circuitry used in a vehicle.

[0006] The method includes: acquiring a first instruction, the first instruction indicating a first intention, the first intention indicating an action that a user in the vehicle's cabin expects the vehicle to perform in a first target area; wherein the first target area is located outside the vehicle's cabin; acquiring environmental perception information, the environmental perception information indicating the positions of objects and areas included within a first range of the vehicle; determining the position of a first area based on the environmental perception information and the first instruction, the first area corresponding to the first target area; and controlling the vehicle to drive towards the first area.

[0007] The first region and the first target region correspond to each other, and may include: the first region and the first target region being the same region; or, the distance between the first region and the first target region being less than or equal to a certain threshold. For example, the aforementioned certain threshold may be a value between 0.3 meters and 0.5 meters, or it may be other values.

[0008] The above technical solution can combine sensory information outside the cabin to determine the coordinates of the area that the user inside the cabin expects the vehicle to drive to, and then control the vehicle to drive to that area, so as to achieve the purpose of navigation based on the user's pointing and seeing points of interest, which helps to improve navigation efficiency and user convenience, and can improve the user experience.

[0009] In conjunction with the first aspect, in certain implementations of the first aspect, determining the location of the first region based on environmental perception information and the first instruction includes: determining at least one region matching the first instruction based on the environmental perception information, the at least one region including the first region; controlling a prompting device of the vehicle to prompt at least one element, each of the at least one element indicating the location of each region in the at least one region; determining the location of the first region in response to a confirmation operation for the first element of the at least one element; wherein the first element indicates the location of the first region.

[0010] In the above technical solution, when the user command is not precise enough to determine multiple candidate areas based on the command, information indicating these candidate areas can be pushed to the user. This allows the user to select the area that best meets their needs, improving the match between the determined first area and the target area, thereby improving navigation efficiency. When the user command is precise enough to determine the coordinates of the user's desired target area, information about that area can also be pushed to the user for confirmation. This not only improves the reliability of the navigation method in this solution but also enhances the user's interactive experience during navigation.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the first instruction indicates the positional relationship between the first object and the first target region, the first object being located within a first range; at least one region is a region matching the first object; each of at least one element further indicates the positional relationship between each region in at least one region and the first object.

[0012] In the above technical solution, the coordinates of at least one region can be determined by the object indicated by the first instruction and the positional relationship between the target region and the object, which helps to improve the convenience for users to describe the target region. Furthermore, when the user instruction points to a specific object, the directionality of the user instruction can be improved, thereby increasing the degree of agreement between the determined first region and the user's desired target region.

[0013] In conjunction with the first aspect, in certain implementations of the first aspect, determining the location of the first region based on environmental perception information and the first instruction includes: determining a first feature associated with a first object based on the first instruction, the first feature indicating at least one of the following of the first object: type and location; determining a second feature associated with objects included in the first range based on the environmental perception information, the second feature indicating the type and location of each object, and the location of an empty region associated with each object; determining the location of the first object based on the first feature and the second feature; and determining the location of the first region based on the location of the first object.

[0014] In some implementations, determining the position of the first region based on the position of the first object may include: determining the position of the first region based on the position of the first object and the positional relationship between the first object and the first target region, so that the position between the first region and the first object matches the positional relationship between the first object and the first target region. For example, if the first instruction indicates that the first target region is located in front of the first object, then the first region can be the region located in front of the first object.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, the first object includes any of the following: a building, a traffic sign, vehicle-related infrastructure, a road traffic participant, or a natural reference point.

[0016] For example, buildings may include, but are not limited to: the gate of a park, a building, or a part of a building or house; traffic signs may include, but are not limited to, road facilities (such as signs) that provide guidance, restriction, warning or instruction information; vehicle-related infrastructure may include, but is not limited to: charging piles or gas stations; road traffic participants may include, but are not limited to: pedestrians, other vehicles; natural references may include, but are not limited to: trees, green belts and other naturally formed and / or naturally growing objects.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, the first instruction indicates that the first target area is the shadow area formed by the first object under sunlight, and the shadow area formed by the first object under sunlight includes the first area.

[0018] In the above technical solution, the shadow area associated with the object indicated by the user command can be determined, and the vehicle can be controlled to drive into the shadow area, thereby realizing navigation based on the user's point of interest.

[0019] In conjunction with the first aspect, in some implementations of the first aspect, the environmental perception information includes image information, and the method further includes: inputting the image information into the first model to obtain the location of the shadow area.

[0020] For example, the first model can be an image recognition model used to identify the pixel positions corresponding to shadowed areas in an image.

[0021] In the above technical solution, the location of the shadow area of ​​the first object can be determined through image recognition, which helps to reduce the processing complexity of determining the location of the shadow area.

[0022] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: obtaining virtual scene rendering information, wherein the virtual scene rendering information is rendered based on environmental perception information, and the virtual scene rendering information indicates the first object and the shadow area; and determining the position of the shadow area based on the virtual scene rendering information.

[0023] For example, the virtual scene rendering information may be information used to present the virtual driving scene displayed by the display device. The virtual scene rendering information may include the coordinates of the first object and the shadow area in the real scene, or the virtual scene rendering information may indicate the mapping relationship between the position of the first object and the shadow area and the coordinates of the real scene.

[0024] In the above technical solution, the position of the shadow area is determined based on the virtual scene rendering information. After processing the same set of data once, data for two purposes (display and control) can be obtained, which helps to save the overhead required for data processing.

[0025] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: obtaining sunlight position indication information, which indicates the position of the sun in the current time period; determining the position of the first region based on the position of the first object, including: determining the position of the first region based on the position of the first object and the sunlight position indication information.

[0026] The above technical solution can reduce the dependence on the accuracy of image processing. When the size of the shadow area cannot be accurately determined by image recognition, the accurate location of the shadow area can be determined by the sunlight position indication information.

[0027] In conjunction with the first aspect, in some implementations of the first aspect, each area in at least one region satisfies the conditions for vehicle parking or the conditions for vehicle passage.

[0028] The above technical solution can ensure the passability of the vehicle during navigation and / or the safety of the vehicle during parking.

[0029] In conjunction with the first aspect, in some implementations of the first aspect, the user in the cockpit includes the first user, and the first instruction includes the voice instruction of the first user.

[0030] In conjunction with the first aspect, in some implementations of the first aspect, the first instruction further includes: an instruction determined based on the first user's first limb movement; and / or an instruction determined based on the first user's first gaze direction; wherein the time difference between the start time of detecting the first limb movement or the first gaze and the start time of detecting the voice instruction is less than or equal to a time threshold.

[0031] By employing the above technical solution, when the navigation information indicated by the user's voice commands is not accurate enough, combining the user's body movements and / or line of sight can help improve the accuracy of the identified first area.

[0032] In conjunction with the first aspect, in some implementations of the first aspect, the user in the cockpit also includes a second user. During the time period when the first instruction is detected, a second instruction from the second user is also detected. When the second instruction and the first instruction conflict, the location of the first area is determined based on the environmental perception information and the first instruction, including: when the first user is a user in the main driving area of ​​the vehicle, the location of the first area is determined based on the environmental perception information and the first instruction.

[0033] When multiple users are present in the cockpit and vehicle control commands from multiple users are detected, the system responds to the primary driver's command and controls the vehicle's movement, which helps ensure driving safety.

[0034] In conjunction with the first aspect, in some implementations of the first aspect, the user in the cockpit also includes a third user. During the time period when the first instruction is detected, a third instruction from the third user is also detected. When the third instruction conflicts with the first instruction, the method further includes: determining the identity information of the first user based on the first instruction; determining the identity information of the third user based on the third instruction; and determining the location of the first area based on environmental perception information and the first instruction, including: when the first user is an authorized user of the vehicle and the third user is not an authorized user of the vehicle, determining the location of the first area based on environmental perception information and the first instruction.

[0035] In some implementations, the third instruction is a voice instruction, which can determine the identities of the first user and the third user based on the voiceprint information included in the first and third instructions.

[0036] In some other implementations, facial recognition, body features, and other methods can be used to identify the first and third users.

[0037] Authorized users of a vehicle may include the legal owner of the vehicle (such as the vehicle owner), or may include other users or persons authorized by the legal owner of the vehicle.

[0038] It should be noted that the user identification process is conducted with the user's consent. If the user does not agree to identification based on voiceprint, facial features, body language, or other information, then this implementation method can be omitted.

[0039] When multiple users are present in the cockpit and vehicle control commands from multiple users are detected, the vehicle responds to the vehicle control commands of the authorized user and controls the vehicle's movement, which helps ensure vehicle driving safety.

[0040] In conjunction with the first aspect, in some implementations of the first aspect, the first instruction further indicates a second intention, which indicates an action that a user in the cabin expects the vehicle to perform in a second target area, and the second target area is farther away from the vehicle than the first target area. The method further includes: determining the location of the second area according to the first instruction, the second area corresponding to the second target area, and controlling the vehicle to drive towards the second area after the vehicle has driven to the first area.

[0041] The second target area is farther away from the vehicle than the first target area, which may include: the straight-line distance between the second target area and the vehicle is greater than the straight-line distance between the first target area and the vehicle; or the distance the vehicle needs to travel to reach the second target area is longer than the distance the vehicle needs to travel to reach the first target area.

[0042] In the above technical solution, when a user gives multiple instructions with one command, the vehicle can be controlled to execute the actions of the multiple instructions in sequence, which helps to improve the intelligence of the vehicle, reduce the probability of the user having to issue repeated commands, and help to improve the user's driving experience.

[0043] In a second aspect, a navigation device is provided, comprising an acquisition unit and a processing unit, wherein the acquisition unit is configured to: acquire a first instruction, the first instruction indicating a first intention, the first intention indicating an action that a user in the vehicle's cabin expects the vehicle to perform in a first target area; wherein the first target area is located outside the vehicle's cabin; the acquisition unit is further configured to: acquire environmental perception information, the environmental perception information indicating the positions of objects and areas included within a first range of the vehicle; the processing unit is configured to: determine the position of a first area based on the environmental perception information and the first instruction, the first area corresponding to the first target area; and control the vehicle to travel towards the first area.

[0044] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is configured to: determine at least one region matching the first instruction based on environmental perception information, the at least one region including a first region; control a prompting device of the vehicle to prompt at least one element, each of the at least one element indicating the position of each region in the at least one region; and determine the position of the first region in response to a confirmation operation for the first element of the at least one element; wherein the first element indicates the position of the first region.

[0045] In conjunction with the second aspect, in some implementations of the second aspect, the first instruction indicates the positional relationship between the first object and the first target region, the first object being located within a first range; at least one region is a region matching the first object; each of at least one element further indicates the positional relationship between each region in at least one region and the first object.

[0046] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is configured to: determine a first feature associated with a first object according to a first instruction, the first feature indicating at least one of the following of the first object: type, location; determine a second feature associated with objects included in a first range according to environmental perception information, the second feature indicating the type and location of each object, and the location of an empty region associated with each object; determine the location of the first object according to the first feature and the second feature; and determine the location of the first region according to the location of the first object.

[0047] In conjunction with the second aspect, in some implementations of the second aspect, the first object includes any of the following: a building, a traffic sign, vehicle-related infrastructure, a road traffic participant, or a natural reference point.

[0048] In conjunction with the second aspect, in some implementations of the second aspect, the first instruction indicates that the first target area is the shadow area formed by the first object under sunlight, and the shadow area formed by the first object under sunlight includes the first area.

[0049] In conjunction with the second aspect, in some implementations of the second aspect, the environmental perception information includes image information, and the processing unit is further used to: input the image information into the first model to obtain the location of the shadow area.

[0050] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition unit is further configured to: acquire virtual scene rendering information, wherein the virtual scene rendering information is rendered based on environmental perception information, and the virtual scene rendering information indicates the first object and the shadow area; the processing unit is configured to: determine the position of the shadow area based on the virtual scene rendering information.

[0051] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition unit is further configured to: acquire sunlight position indication information, which indicates the position of the sun in the current time period; and the processing unit is configured to: determine the position of the first area based on the position of the first object and the sunlight position indication information.

[0052] In conjunction with the second aspect, in some implementations of the second aspect, each area in at least one region satisfies the conditions for vehicle parking or the conditions for vehicle passage.

[0053] In conjunction with the second aspect, in some implementations of the second aspect, the user in the cockpit includes the first user, and the first command includes the voice command of the first user.

[0054] In conjunction with the second aspect, in some implementations of the second aspect, the first instruction further includes: an instruction determined based on the first user's first limb movement; and / or an instruction determined based on the first user's first gaze direction; wherein the time difference between the start time of detecting the first limb movement or the first gaze direction and the start time of detecting the voice instruction is less than or equal to a time threshold.

[0055] In conjunction with the second aspect, in some implementations of the second aspect, the user in the cockpit also includes a second user. During the time period when the first instruction is detected, a second instruction from the second user is also detected. When the second instruction and the first instruction conflict, the processing unit is used to: when the first user is a user in the main driving area of ​​the vehicle, determine the location of the first area based on the environmental perception information and the first instruction.

[0056] In conjunction with the second aspect, in some implementations of the second aspect, the user in the cockpit also includes a third user. During the time period when the first instruction is detected, a third instruction from the third user is also detected. When the third instruction conflicts with the first instruction, the processing unit is further configured to: determine the identity information of the first user according to the first instruction; determine the identity information of the third user according to the third instruction; and when the first user is an authorized user of the vehicle and the third user is not an authorized user of the vehicle, determine the location of the first area according to the environmental perception information and the first instruction.

[0057] In conjunction with the second aspect, in some implementations of the second aspect, the first instruction also indicates a second intention, which indicates an action that the user in the cabin expects the vehicle to perform in the second target area; the processing unit is also configured to: determine the location of the second area according to the first instruction, the second area corresponding to the second target area; and control the vehicle to travel to the second area after the vehicle has traveled to the first area.

[0058] Thirdly, a navigation device is provided, the device comprising: a processor for executing a computer program stored in the memory, such that the device performs the method in any possible implementation of the first aspect described above.

[0059] In conjunction with the third aspect, in some implementations of the third aspect, the device also includes a memory.

[0060] Fourthly, a computer program product is provided, comprising: computer program code, which, when executed on a computer or processor, causes the computer or processor to perform the method in any possible implementation of the first aspect.

[0061] It should be noted that the above computer program code can be stored in whole or in part on a storage medium, which can be packaged together with the processor or packaged separately from the processor.

[0062] Fifthly, a computer-readable storage medium is provided, the computer-readable medium storing instructions that, when executed by a processor, cause the processor to implement the method in any possible implementation of the first aspect.

[0063] In a sixth aspect, a chip is provided that includes circuitry for performing the method in any of the possible implementations of the first aspect described above.

[0064] In a seventh aspect, a vehicle is provided that includes means as in any possible implementation of the second or third aspect, or the vehicle includes a computer-readable storage medium as in any possible implementation of the fifth aspect, or the vehicle includes a chip as in any possible implementation of the sixth aspect, or the vehicle is loaded with a computer program product as in any possible implementation of the fourth aspect.

[0065] In conjunction with the seventh aspect, in some implementations of the seventh aspect, the vehicle is a vehicle in a broad sense, such as a means of transportation (e.g., commercial vehicles, passenger cars, motorcycles, flying cars, trains, etc.), industrial vehicles (e.g., forklifts, trailers, tractors, etc.), engineering vehicles (e.g., excavators, bulldozers, cranes, etc.), agricultural equipment (e.g., lawnmowers, harvesters, etc.), amusement equipment, toy vehicles, etc. In practical implementation, the vehicle can also be a road vehicle, a water vehicle, an air vehicle, industrial equipment, agricultural equipment, or other intelligent driving equipment such as entertainment equipment.

[0066] For the beneficial effects not described in detail in aspects two through seven, please refer to the description in aspect one, which will not be repeated here. Attached Figure Description

[0067] Figure 1 This is a functional schematic block diagram of the vehicle provided in the embodiments of this application;

[0068] Figure 2 This is a schematic block diagram of the navigation system architecture provided in the embodiments of this application;

[0069] Figure 3 This is another schematic block diagram of the navigation system architecture provided in the embodiments of this application;

[0070] Figure 4 This is a schematic flowchart of the navigation method provided in the embodiments of this application;

[0071] Figure 5 This is a schematic diagram illustrating the application scenarios involved in the embodiments of this application;

[0072] Figure 6 This is yet another schematic diagram illustrating an application scenario involved in the embodiments of this application;

[0073] Figure 7 This is another schematic diagram illustrating the application scenario involved in the embodiments of this application;

[0074] Figure 8 This is another schematic diagram illustrating the application scenario involved in the embodiments of this application;

[0075] Figure 9 This is another schematic diagram illustrating the application scenario involved in the embodiments of this application;

[0076] Figure 10 This is another schematic flowchart of the navigation method provided in the embodiments of this application;

[0077] Figure 11 This is a schematic block diagram of the navigation device provided in the embodiments of this application;

[0078] Figure 12 This is another schematic block diagram of the navigation device provided in the embodiments of this application. Detailed Implementation

[0079] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0080] Figure 1 This is a functional block diagram of a vehicle provided in an embodiment of this application. For example... Figure 1 As shown, the vehicle 100 may include a perception system 120, a human-machine interaction system 130, and a computing platform 150. The perception system 120 may include several sensors for sensing information about the surrounding environment of the vehicle 100. For example, the perception system 120 may include a positioning system, which may be a Global Positioning System (GPS), a BeiDou system, or another positioning system. As another example, the perception system 120 may also include one or more of the following: an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, and a camera device. The camera device may include a camera device for collecting information inside the cockpit and a camera device for collecting information outside the cockpit.

[0081] The human-computer interaction system 130 includes a device for receiving user commands and a device for providing prompts to the user. The device for receiving user commands may include at least one of the following: a sound receiving device for receiving user voice commands, such as a microphone, transceiver, etc.; or a device for receiving commands input by the user through a screen, such as a human-machine interface (HMI); or a camera device for receiving commands such as user body posture, such as an in-cabin camera. The prompting device may include at least one of the following: a sound-emitting device and a display device. More specifically, the sound-emitting device may include a speaker, audio jack, or other device that plays audio. The display device is mainly divided into two categories: the first is an in-vehicle display screen; the second is a projection display screen, such as a head-up display (HUD). An in-vehicle display screen is a physical display screen and an important component of an in-vehicle infotainment system. It should be noted that an in-vehicle display screen may include an HMI. Head-up display, also known as a head-up display system, is mainly used to display driving information such as speed and navigation on a display device (such as a windshield) in front of the user, in order to reduce the time the user's eyes are shifted, avoid pupil changes caused by the shift of the user's eyes, and improve driving safety and comfort.

[0082] In some implementations, some or all of the devices in the human-computer interaction system 130 that receive user instructions may be included in the sensing system 120.

[0083] Some or all of the functions of vehicle 100 can be controlled by computing platform 150. Computing platform 150 may include processors 151 to 15n. A processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In reconfigurable hardware circuits, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the relevant functions. Furthermore, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. In addition, the computing platform 150 may also include a memory for storing instructions. Some or all of the processors 151 to 15n can call the instructions in the memory to implement the corresponding functions.

[0084] The computing platform 150 can control the operation of the intelligent driving system, which may include an advanced driving assistance system (ADAS) and / or an autonomous driving system (ADS). The intelligent driving system utilizes various sensors on the vehicle (including but not limited to: LiDAR, millimeter-wave radar, cameras, ultrasonic sensors, GPS, and inertial measurement units) to acquire information from the vehicle's surroundings, and analyzes and processes this information to achieve functions such as obstacle perception, target recognition, vehicle localization, path planning, and driver monitoring / alerts, thereby improving the safety, automation, and comfort of driving the vehicle.

[0085] At different levels of autonomous driving (or intelligent driving levels, ranging from L0 to L5, totaling six levels), intelligent driving systems can achieve different levels of automated driving assistance based on artificial intelligence algorithms and information acquired by multiple sensors. These levels of autonomous driving are based on the classification standards of the Society of Automotive Engineers (SAE). Specifically, L0 is no automation; L1 is driver assistance; L2 is partial automation; L3 is conditional automation; L4 is high automation; and L5 is full automation. At levels L1 to L3, the task of monitoring road conditions and reacting is jointly completed by the driver and the system, requiring the driver to take over dynamic driving tasks. Levels L4 and L5 allow the driver to completely transform into a passenger. Currently, the functions that intelligent driving systems can achieve mainly include, but are not limited to: adaptive cruise control, automatic emergency braking, automatic parking, blind spot monitoring, forward cross-traffic alert / braking, rear cross-traffic alert / braking, forward collision warning, lane departure warning, lane keeping assist, rear collision warning, traffic sign recognition, traffic jam assist, and highway assist. It should be understood that the above-mentioned functions can have specific modes at different levels of autonomous driving (L0-L5). The higher the level of autonomous driving, the more intelligent the corresponding mode.

[0086] The following is combined with Figure 2 The specific roles of the sensing system 120, the human-computer interaction system 130, and the computing platform 150 in this application are explained. Figure 2 A schematic block diagram of the navigation system architecture provided in an embodiment of this application is shown. The system includes a perception module 210, a destination calculation module 220, a human-computer interaction module 230, and a planning and control module 240. The perception module 210 may include… Figure 1 The sensing system 120 shown includes one or more sensors; the destination calculation module 220 and the planning and control module 240 may each include Figure 1The computing platform 150 shown includes one or more processors; the human-computer interaction module 230 may include... Figure 1 One or more devices in the human-computer interaction system 130 shown. Figure 2 The functions of each module in the system shown are as described in items (I) to (IV) below.

[0087] (i) The perception module 210 includes an in-cabin perception module 211 and an external perception module 212. The in-cabin perception module 211 is used to determine whether the user has issued a vehicle control command based on voice information collected by the in-cabin sound receiver; the in-cabin perception module 211 is also used to collect image information inside the cabin to determine the user's position inside the cabin and the user's body movements. The external perception module 212 is used to collect point cloud data and / or image information outside the vehicle cabin, which indicates the type and location of objects around the vehicle.

[0088] (ii) The destination calculation module 220 is used to determine the coordinates of the area indicated in the user command based on the user command from the in-cabin perception module 211 and the environmental perception information from the external perception module 212. The user command may be the voice information and / or gesture instructions mentioned in section (ii) above, and the area indicated by the user command is the area where the user expects the vehicle to travel or park; the environmental perception information may be the point cloud data and / or image information mentioned in section (ii) above.

[0089] In some implementations, the user instruction indicates the aforementioned area by pointing to an object. Furthermore, the destination calculation module 220 can also determine the coordinates of the area indicated in the user instruction by combining map information. For example, the map information indicates the location of static features along the road the vehicle is traveling on, including road boundaries, lane boundaries, intersection boundaries, and other similar features.

[0090] Furthermore, the destination resolution module 220 can send the coordinates of the area indicated in the user instruction to the human-computer interaction module 230.

[0091] (iii) The human-computer interaction module 230 is used to display the destination element, which indicates the area indicated in the user command.

[0092] In some implementations, the destination resolution module 220 determines multiple regions and the coordinates of each region based on user instructions and environmental perception information. The human-computer interaction module 230 can then display elements indicating these regions, allowing the user to select a target region. After the user selects a target region, the human-computer interaction module 230 can notify the destination resolution module 220 of this target region. In this case, the human-computer interaction module 230 can also directly send the coordinates of the target region to the planning and control module 240.

[0093] (iv) The planning and control module 240 can plan the path for the vehicle to travel to the target location based on the coordinates of the target area, and control the vehicle to travel along the path to the target area.

[0094] Understandably, after the destination calculation module 220 determines the target area, it can send the coordinates of the target area to the planning and control module 240 so that the planning and control module 240 can perform path planning and vehicle control.

[0095] In some implementations, the destination resolution module 220's ability to process user commands and environmental perception information can be based on a neural network model. This neural network model can be a multimodal large language model (MLLM), or it can be other neural network models. For example, Figure 3 An example of the specific structure of the destination resolution module 220 is shown, which includes at least an encoder 1, a semantic recognition model, a limb orientation recognition model, and a fusion module. Specifically:

[0096] Encoder 1 is used to extract feature information from external sensing information. If the external sensing information is image information, encoder 1 can extract image features from the image, for example, by mapping high-dimensional image data to a low-dimensional space to obtain a low-dimensional feature representation. If the external sensing information is point cloud data, encoder 1 can extract point cloud features from the point cloud data, for example, by mapping high-dimensional point clouds to a low-dimensional space to obtain a low-dimensional feature representation. In some implementations, the low-dimensional features obtained by encoder 1 can directly indicate the coordinates of each physical location or region indicated by the image information or point cloud data through position encoding.

[0097] In some implementations, encoder 1 can combine map information to perform feature extraction, which indicates lane boundaries, road boundaries (or may also include intersection boundaries), and coordinate information associated with the road.

[0098] The semantic recognition model is used to identify keywords from in-cabin voice information and determine the semantics expressed by the text composed of these keywords based on the temporal sequence of their association. For example, the in-cabin voice information may indicate the action the user expects the vehicle to perform, or it may indicate the area where the user expects the vehicle to perform that action. The expected action may include parking, with corresponding keywords including "stop," "park," etc.; the expected action may also include driving and / or turning, with corresponding keywords including "turn," "left turn," "right turn," "go straight," etc. The area where the expected action is performed can be directly indicated; for example, if the area is an intersection, the keyword could be "intersection." Alternatively, the area can be indicated by an object, with keywords including "object," or it may indicate the positional relationship between the area and the object. For example, if the user expects the vehicle to park near a fruit shop, the keywords could include "fruit shop," or "nearby." For example, the semantic recognition model can be a neural network model based on the Transformer architecture, or it can be a model based on a convolutional neural network (CNN). Furthermore, the information output by the semantic recognition model can be text information, or it can be other forms of information.

[0099] A limb orientation recognition model is used to identify the direction pointed by a user's specific limb posture from in-cabin image information. The model uses a human posture estimation algorithm to determine the 3D positions of key human points from the in-cabin image information, and then constructs the topological structure of limb joints based on these positions. These key human points may include, but are not limited to, shoulder, elbow, wrist, and finger joints. Further, based on the extrinsic parameters of the camera device acquiring the in-cabin image information, the position of the limb joint topological structure in the camera device coordinate system is transformed to the world coordinate system, and the pointing direction is determined based on the limb joint topological structure. In some implementations, a gaze estimation algorithm can be combined to determine the user's gaze direction, and the synergy between the limb pointing direction and the gaze direction can be used to improve the accuracy of the user's pointing direction.

[0100] The fusion module maps features from encoder 1, the semantic recognition model, and the limb orientation recognition model to the same semantic space, thereby determining the coordinates of the target region indicated by the user command, and / or the coordinates of at least one region associated with the object indicated by the user. For example, the fusion module may include a language layer and transition layers for connecting encoder 1 to the language layer, the semantic recognition model to the language layer, and the limb orientation recognition model to the language layer, respectively. These transition layers map the features output by encoder 1, the semantic recognition model, and the limb orientation recognition model to a language-compatible space, enabling effective interaction and understanding between these features.

[0101] Understandably, the fusion module can determine the vector direction in which the user's limb (or line of sight) is pointing, and the intersection point with the area or object indicated by the environmental perception information, thereby determining the area or object in which the user's limb (or line of sight) is pointing.

[0102] In some implementations, if the user instruction only includes voice instructions, the fusion module can determine the coordinates of the relevant region based on the features from encoder 1 and the features from the semantic recognition model; if the user instruction includes both voice instructions and body movements, the fusion module can determine the coordinates of the relevant region based on the features from encoder 1, the features from the semantic recognition model, and the features from the body direction recognition model.

[0103] In some implementations, a user may use a single voice command to instruct the vehicle to perform two or more actions. Taking a single voice command instructing the vehicle to perform two actions as an example, the semantic recognition model can identify two sets of keywords from the in-cabin voice information. One set of keywords instructs the vehicle to perform one action in one area, and the other set instructs the vehicle to perform another action in another area. Based on the temporal sequence of the keyword associations, the order in which the vehicle performs the relevant actions can be determined. For example, if the user's voice command includes "Turn right at the intersection ahead, then drive straight for 200 meters and pull over," the semantic recognition model can identify two sets of keywords: "intersection ahead" and "turn right"; "turn right and drive straight for 200 meters" and "park." Furthermore, the fusion module can output the coordinates of the intersection ahead and the coordinates of driving straight for 200 meters after turning right, thus enabling the control module 240 to control the vehicle to turn right at the intersection ahead and then drive 200 meters after turning right before parking.

[0104] In some implementations, in order to improve the accuracy and reliability of the results output by the destination solution module 220, the encoder 1, the semantic recognition model and the limb orientation recognition model can be pre-trained respectively, and the alignment pre-training of the aforementioned models and the transition layer between the language layer can be performed.

[0105] It should be understood that the above module is only an example, and in actual applications, it may be added or removed as needed. For example, Figure 2 In the system architecture shown, the destination calculation module 220 and the planning and control module 240 can be merged into one module. Alternatively, the planning and control module 240 can be further refined into a planning module and a control module, where the planning module plans the driving path for the vehicle, and the control module controls the vehicle to drive along the planned path. Another example is that the semantic recognition model and the limb orientation recognition model can be fused into one model to determine if the user has a vehicle control intention. When it is determined that the user has a vehicle control intention, the model is fused with the user's vehicle control intention and the output of encoder 1 to determine the coordinates of the relevant area indicated by the user's vehicle control intention.

[0106] The above describes the navigation system architecture provided in the embodiments of this application. The following details the system architecture based on... Figure 2 The navigation system shown implements the navigation method provided in the embodiments of this application.

[0107] Figure 4 A schematic flowchart of a navigation method provided in an embodiment of this application is shown. This method can be applied to... Figure 1 In the vehicle shown, or the method can be derived from Figure 2 The system shown is executed, for example, by Figure 2 The destination calculation module 220 and the planning and control module 240 in the process are executed. More specifically, the method 400 may include:

[0108] S401, retrieve user instructions.

[0109] For example, the form and meaning of user instructions can be referred to the description in the foregoing embodiments, and will not be repeated here.

[0110] S402, input the user command into the intent recognition model to obtain the user intent.

[0111] For example, the intent recognition model may include the semantic recognition model in the foregoing embodiments, or it may also include the limb orientation recognition model in the foregoing embodiments.

[0112] The user intent can indicate whether the user intends to control the vehicle to park or to drive the vehicle to a fixed area. Furthermore, when the user intent indicates that the user intends to control the vehicle to park or to drive the vehicle to a fixed area, the user intent can also indicate the target area that the user expects the vehicle to drive to. For example, if the intent recognition model detects keywords indicating actions the user expects the vehicle to perform in the aforementioned embodiments, it can determine that the user intent indicates that the user intends to control the vehicle to park or to drive the vehicle to a fixed area; further, if the intent recognition model detects keywords indicating areas indicating actions the user expects the vehicle to perform in the aforementioned embodiments, or if the intent recognition model detects a user pointing to an area or object outside the vehicle, it can determine the target area and / or target object associated with the user's area.

[0113] It should be noted that the time interval between the start time of obtaining the user instruction that instructs the user to perform the action the vehicle expects to execute and the start time of obtaining the user instruction for the area instructing the user to perform the action the vehicle expects to execute is less than or equal to a duration threshold. For example, the duration threshold can be a value between 1 second and 2 seconds, or it can be other values.

[0114] S403, when the user intends to direct the vehicle to a target area, acquires environmental perception information, which indicates the position of objects outside the vehicle and the location of each area.

[0115] For example, the form and content of environmental perception information can be referred to the description in the foregoing embodiments, and will not be repeated here.

[0116] S404, Determine the coordinates of region 1 based on environmental perception information and user intent.

[0117] For example, environmental perception information and user instructions indicating user intent can be input into the destination calculation module 220 in the aforementioned embodiment to determine the coordinates of region 1.

[0118] In one example, a user uses voice commands to specify object 1 and its positional relationship with a target area. For instance, the voice command might be "Park by the tree near the front door." In this case, the coordinates of object 1 can be obtained by processing the voice command. Then, based on these coordinates and the positional relationship between object 1 and the target area, the coordinates of area 1 can be determined. The positional relationship between area 1 and object 1 matches the positional relationship between the target area and object 1. In practical implementation, multiple areas, including area 1, may be determined based on the coordinates of object 1. Each of these areas satisfies the positional relationship between object 1 and the target area specified by the voice command.

[0119] In another example, if a user points to a target area using body language and / or eye contact, the intersection point between the user's body language (or eye contact) and the area indicated by environmental perception information can be determined based on the vector direction of the user's body language (or eye contact). This allows us to determine the coordinates of the area pointed to by the user's body language (or eye contact). In this case, area 1 and the target area can be considered the same area.

[0120] In another example, the user points to object 1 using gestures and / or eye contact, and indicates the positional relationship between object 1 and a target area using voice commands. For example, the voice command might say "Park here," while the gestures and / or eye contact indicate the specific location of "this." In this case, the intersection point between the vector direction pointed to by the user's limb (or eye contact) and the object indicated by the environmental perception information can be determined, thereby determining the coordinates of the object pointed to by the user's limb (or eye contact). Based on the object's coordinates and the positional relationship between object 1 and the target area indicated by the voice command, at least one area, including area 1, is determined, where each area within the at least one area satisfies the positional relationship between object 1 and the target area indicated by the voice command.

[0121] In some implementations, when multiple regions are identified, the location of these regions can be indicated by a display device, thereby determining region 1 in response to the user's selection.

[0122] In some other implementations, the user instruction may also indicate multiple user intentions, where each user intention indicates the action the user expects the vehicle to perform in the target area. Based on each user intention, at least one target area can be determined, and then, in response to the user's selection, the final area corresponding to each user intention can be determined.

[0123] In practice, the coordinates of region 1 can be determined based on map information and user intent. Alternatively, the coordinates of region 1 can be determined based on map information, environmental awareness information, and user intent.

[0124] S405, control the vehicle to move towards area 1.

[0125] In some implementations, when the user command instructs the user to perform the action that the vehicle is expected to do, the vehicle is controlled to drive towards area 1 and park in area 1.

[0126] In some implementations, user commands can originate from a single user or from multiple users. When user commands originate from multiple users, the areas or objects indicated by the commands issued by each user are consistent. If user commands originate from multiple users, and the areas or objects indicated by the commands issued by each user are inconsistent, then the vehicle is controlled according to the commands issued by the user in the main driving area or the authorized user of the vehicle.

[0127] To facilitate understanding of the navigation method provided in the embodiments of this application, the following is combined with... Figures 5 to 9 Taking a user command originating from a single user as an example, this paper introduces the application scenarios of the navigation method provided in this application.

[0128] Scenario 1: The user instructs the vehicle to park in a specific area via voice command. For example... Figure 5 As shown, when a vehicle approaches an intersection and detects the user's voice command "Turn right at the intersection ahead, then drive 200 meters and stop," the vehicle's position can be determined. Based on the vehicle's position and the road topology indicated by the map information, the nearest intersection in front of the vehicle is identified. Then, based on the intersection, the area 200 meters away from the intersection after the vehicle turns right (hereinafter referred to as the parking area) is determined, and the vehicle is controlled to park in this area. In some implementations, when the user's voice command "Turn right at the intersection ahead, then drive 200 meters and stop" is detected, the nearest intersection in front of the vehicle and its topological relationship can also be determined based on image information and / or point cloud data collected by the vehicle's perception system. Then, based on the topological relationship of the nearest intersection, the parking area 200 meters away from the intersection after the vehicle turns right is determined. The topological relationship of the intersection can indicate at least one of the following: which roads the intersection connects to, or the direction of the roads connected to the intersection. For a more specific method of determining the area 200 meters away from the intersection after a vehicle turns right, please refer to the description in the aforementioned embodiments, which will not be repeated here.

[0129] Furthermore, after the parking area is determined, and before the vehicle is parked in that area, the user can be prompted with information about the determined parking area through the navigation interface. For example, the navigation interface can be as follows: Figure 5 The lower right image shows the graphical user interface (GUI), which includes element 501 indicating the vehicle's location, element 502 indicating the parking area, and dialog box 503. Dialog box 503 includes the text prompting the user to confirm the parking area: "Please confirm if you want to park here," as well as controls for "Yes" and "No." Furthermore, when the "Yes" control is clicked, it can be determined that the aforementioned parking area matches the target area indicated by the user's voice command, allowing the vehicle to be driven to and parked in the area indicated by element 502.

[0130] In some implementations, if the "No" control in dialog box 503 is detected to be clicked, the navigation interface can be switched to display. Figure 6The interface shown includes a 3D virtual scene constructed from external sensory information collected based on the vehicle's perception information. This 3D virtual scene indicates the environmental characteristics around the vehicle. The interface includes element 502 to allow the user to intuitively perceive the positional relationship between the designated parking area and the vehicle. Furthermore, the interface includes a dialog box 504 containing the text "Drag the map to select the location where you want to park." Further, in response to the user dragging the 3D virtual scene and clicking on a location on the interface, the area associated with that location can be determined as the target area where the user intends to park the vehicle.

[0131] Scenario 2: The user instructs the vehicle to park in a non-specific area via voice command. For example, while the vehicle is driving on a road (such as a congested road in a scenic area), if the user in the cabin is detected to give the voice command "Please find an empty parking space on the side of the road", then based on the image information and / or point cloud data collected by the vehicle's perception system, an empty parking space is selected, and the vehicle is controlled to park in the selected empty parking space.

[0132] In some implementations, the vehicle can select multiple available parking spaces and determine the most suitable one as the target parking space, then control the vehicle to park in that target space. In other implementations, after the vehicle selects multiple available parking spaces, information about those spaces, such as the location of each space and whether there are any obstacles nearby, is pushed to the user through the vehicle's display device. Then, in response to the user's selection, the target parking space is determined, and the vehicle is controlled to park in the user-selected target parking space. For example, the aforementioned "most suitable available parking space" can be any of the following: the most convenient available parking space for the vehicle to enter, the most convenient available parking space for the vehicle to park in, or an available parking space that meets other conditions or other user needs.

[0133] In some implementation methods, such as Figure 7As shown, when the vehicle is driving on the road, if it detects a user's voice command "Find an empty parking space near the fruit shop ahead," it can determine the location of the fruit shop in front of the vehicle based on image information and / or point cloud data collected by the vehicle's perception system. This allows it to identify available parking spaces nearby and control the vehicle to park in an available spot. In one example, after detecting the fruit shop ahead, information about the fruit shop can be pushed to the user via a display device. If the user confirms that the fruit shop is the target fruit shop, then available parking spaces nearby are checked. If the user determines that the fruit shop is not the one they need, the vehicle can continue to filter suitable fruit shops based on image information and / or point cloud data collected by the vehicle's perception system. In another example, when a fruit shop is detected in front of the vehicle and it is determined that there is at least one vacant parking space nearby, the information of the fruit shop can be pushed to the user through the display device, or the information of at least one vacant parking space can also be pushed; if a fruit shop is detected in front of the vehicle but there is no vacant parking space nearby, the vehicle does not push the information of the fruit shop to the user, and the vehicle continues to filter fruit shops with vacant parking spaces nearby based on the image information and / or point cloud data collected by the vehicle perception system.

[0134] For example, the information about the aforementioned fruit store may include the name and / or location of the fruit store, and the available parking spaces near the fruit store can be: available parking spaces at a distance of less than or equal to a certain threshold from the fruit store. For example, the certain threshold can be a value between 20 meters and 50 meters, or it can be other values.

[0135] In some scenarios, after a vehicle identifies a target fruit store, if multiple vacant parking spaces are detected nearby, it can select the most suitable one as the target parking space and park the vehicle there. Alternatively, when multiple vacant parking spaces are detected near the fruit store, the vehicle's display can also push information about the fruit store and the detected vacant parking spaces to the user. For example, this can be achieved through... Figure 7 The interface shown in the right figure pushes the aforementioned information to the user. This interface includes dialog boxes 701 and 705, as well as elements 702 to 704. Dialog box 701 includes text information indicating the location of the fruit store, dialog box 705 prompts the user to select a target parking space from the detected available parking spaces, and elements 702 to 704 respectively indicate the location of the detected available parking spaces relative to the fruit store.

[0136] It should be noted that the above embodiments are only illustrated with a fruit store as a specific example. In actual implementation, the fruit store can also be other buildings, traffic signs, infrastructure associated with vehicles, road traffic participants, or natural reference points.

[0137] Scenario 3: The user instructs the vehicle to park next to a specific object using voice commands. For example... Figure 7 As shown, there are trees 611 on the road for vehicles to travel on, and the trees 611 form a shaded area 612 under sunlight. In one implementation, when the voice command "Park under the shade of that tree" from a user in the vehicle cabin is detected, the position and canopy size of the tree in front of the vehicle can be determined based on image information and / or point cloud data collected by the vehicle's perception system. Further, based on the position of the tree, the canopy size, and the position of the sun, the size and position of the shaded area are determined, thereby controlling the vehicle to drive towards the shaded area and park there.

[0138] For example, the sun's position indicates its altitude and azimuth. A larger altitude angle (the angle between sunlight and the horizon) results in a shorter object projection; a smaller altitude angle results in a longer projection. The projection length is inversely proportional to the tangent of the altitude angle. The azimuth determines the projection direction but does not directly affect the projection length. The projections of multiple adjacent objects may overlap or separate due to different azimuth angles. In practical implementation, the sun's position can be determined based on the current time period and the current date. The current time period is the period during which the voice command "park under the shade of that tree" was detected, and the current date is the date of the current time period. It is understandable that the sun's trajectory differs on different days of the year; therefore, the sun's position at the same time on different days may vary significantly.

[0139] In another implementation, when the voice command "Stop under the shade of that tree" is detected from the user in the vehicle cabin, the location of the tree in front of the vehicle and the area of ​​the shade formed by the tree can be determined based on the image information and / or point cloud data collected by the vehicle's perception system.

[0140] In another implementation, when the user's voice command "Stop under the shade of that tree" is detected in the vehicle cabin, the location of the tree in front of the vehicle and the area of ​​the shade formed by that tree can be determined based on virtual scene rendering information. This virtual scene rendering information is obtained based on image information and / or point cloud data collected by the vehicle's perception system, and is used to generate the three-dimensional virtual scene in the aforementioned embodiments.

[0141] In some implementations, after identifying a shaded area and before parking the vehicle in that area, the user can be prompted with information about the identified parking area via a navigation interface. For example, the navigation interface could be as follows: Figure 8As shown in the lower right corner of the interface, this interface includes a 3D virtual scene constructed from external perception information collected based on the vehicle's perception information. This 3D virtual scene indicates the location of the identified trees and their surrounding environmental features, such as the location of obstacles and / or buildings around the trees. This 3D virtual scene can be rendered based on the aforementioned virtual scene rendering information. For example, the interface includes elements 613 and 614, and a dialog box 615. Element 613 indicates the location of the identified trees, element 614 indicates the identified shaded area, and dialog box 615 includes the text message "Shaded area calculated; parking will soon proceed to the shaded area selected in the interface (such as the area highlighted by element 614)." Optionally, Figure 8 The dialog box 615 can also display the following text information: "The shaded area has been calculated. Please confirm parking in the shaded area selected in the interface," as well as the "Yes" and "No" controls. Furthermore, when the "Yes" control is clicked, it can be determined that the aforementioned shaded area matches the target area where the user expects the vehicle to park, as indicated by the user's voice command, and then the vehicle can be controlled to drive and park in the area indicated by element 614.

[0142] In some implementations, when multiple trees are in front of the vehicle, the target tree can be determined from among the multiple trees based on other information from the user's voice command. For example, if the vehicle arrives at the entrance of a park and the user's voice command instructs the vehicle to park under the shade of a tree near the park entrance, the target tree can be selected from among the multiple trees, and the shade area can be determined based on that target tree. If multiple trees are present at the park entrance, the most suitable shade area among the shade areas of each tree can be selected as the target parking space, and the vehicle can be controlled to park in the target parking space. Alternatively, when multiple trees are present at the park entrance, the shade area of ​​each tree can be determined, and the vehicle's display device can be controlled to push all shade areas, thereby responding to the user's selection, determining the target parking space, and controlling the vehicle to park in the target parking space selected by the user. The most suitable shade area can include the area most convenient for the vehicle to park and / or stop, or it can include shade areas that meet other conditions or other user needs.

[0143] Scenario 4: The user instructs the vehicle to park next to an object using voice commands and gestures. For example... Figure 9As shown, if there is a tree 602 on the road for vehicle travel, when the user's voice command "Stop in front of that tree" is detected in the vehicle cabin, it can be determined that the user expects the vehicle to park in front of a tree. Furthermore, simultaneously with detecting the aforementioned voice command, if the user's right index finger 601 is pointing in a certain direction, and if, based on the image information and / or point cloud data collected by the vehicle's perception system, the direction pointed to by the right index finger 601 intersects with the location of the tree 602, then the "tree" indicated in the user's voice command can be identified as tree 602. Then, based on the relationship between the target area desired by the user in the voice command and the tree, a parking area in front of tree 602 is determined, and the vehicle is controlled to park in that parking area.

[0144] In some implementations, after a parking area is determined but before the vehicle is parked in that area, the user is prompted with information about the determined parking area via a navigation interface. For example, the navigation interface may be as follows: Figure 9 As shown in the lower right corner of the interface, this interface includes a 3D virtual scene constructed from external perception information collected based on the vehicle's perception information. This 3D virtual scene indicates the location of trees determined by the user's voice commands and body movements, as well as the locations of buildings around the trees. Further, the 3D virtual scene displays element 501 indicating the vehicle's location, element 603 indicating the trees 602, element 604 indicating the parking area, and a dialog box 605. The dialog box 605 includes text prompting the user to confirm the parking area: "Do you want to park here?", as well as controls for "Yes" and "No". Further, when the "Yes" control is clicked, it can be determined that the aforementioned parking area matches the target area indicated by the user's voice command, and the vehicle is then controlled to drive and park in the area indicated by element 604. If the "No" control in dialog box 605 is clicked, the user is provided with controls or an interface to reselect or set the target area.

[0145] Understandable Figure 9 In the scenario shown, taking the area in front of tree 602 as an example (i.e., the side of tree 602 closer to the current position of the vehicle), the "front" of the vehicle can also be other locations, such as the side of tree 602 away from the current position of the vehicle.

[0146] It should be noted that in the aforementioned scenarios, when determining the area for vehicle parking, the user's line of sight and / or facial orientation can also be considered to improve the match between the determined area for vehicle parking and the user's desired target area.

[0147] The foregoing embodiments use parking as an example of the user's desired action for the vehicle. In actual implementation, the user may also desire other actions, such as driving straight through a certain location or area, or turning left, right, or making a U-turn at a certain location or area. When the user desires the vehicle to perform actions other than parking, the specific implementation of controlling the vehicle's movement is similar to the foregoing embodiments and will not be repeated here.

[0148] It should also be noted that the elements included in the GUIs of the foregoing embodiments are merely illustrative examples. In actual implementation, the GUIs of the foregoing embodiments may display more or fewer elements, controls, etc. For example, in addition to the elements involved in the text descriptions in the foregoing embodiments, the navigation-related GUI may also include elements indicating the driving direction at the nearest intersection in the driving direction, or it may also include controls for controlling intelligent driving functions, such as controls for implementing the following functions: controlling the vehicle to exit the intelligent driving function, controlling the sound device to broadcast navigation-related information, controlling the parking interface to switch to displaying navigation route information near the vehicle, displaying more controls, and setting relevant parameters of the intelligent driving function.

[0149] Figure 10 This illustration shows another schematic flowchart of the navigation method provided in an embodiment of this application, which can be applied to... Figure 1 In the vehicle shown, or the method can be derived from Figure 2 The system execution is shown. More specifically, the method 1000 may include:

[0150] S1010, Obtain a first instruction, the first instruction indicating a first intention, the first intention indicating an action that a user in the vehicle's cabin expects the vehicle to perform in a first target area; wherein, the first target area is located outside the vehicle's cabin.

[0151] In some implementations, the first instruction may be a user instruction as described in the foregoing embodiments, or the first instruction may be other instructions that indicate the user's intent.

[0152] In some implementations, the user in the cockpit includes the first user, and the first command includes the first user's voice command.

[0153] In some other implementations, the first instruction further includes: an instruction determined based on the first user's first limb movement; and / or an instruction determined based on the first user's first gaze direction; wherein the time difference between the start time of detecting the first limb movement or the first gaze and the start time of detecting the voice instruction is less than or equal to a time threshold.

[0154] For example, the time threshold can be a value between 1 second and 2 seconds, or it can be other values.

[0155] In some implementations, the users in the cockpit also include a second user. During the time period when the first command is detected, a second command from the second user is also detected. If the second command conflicts with the first command, S1030 is executed when the first user is the user in the driver's area of ​​the vehicle.

[0156] For example, a conflict between the second instruction and the first instruction may include at least one of the following: the first instruction and the second instruction indicate different actions that the vehicle is expected to perform, or the first instruction and the second instruction indicate different areas where the vehicle is expected to perform actions. For example, the first instruction and the second instruction respectively instruct the vehicle to stop and to drive; or, for another example, the first instruction and the second instruction respectively instruct the vehicle to stop in area 1 and to stop in area 2.

[0157] In some implementations, the users in the cockpit also include a third user. During the time period when the first instruction is detected, a third instruction from the third user is also detected. When the third instruction conflicts with the first instruction, the method further includes: determining the identity information of the first user based on the first instruction; determining the identity information of the third user based on the third instruction; and determining the location of the first area based on environmental perception information and the first instruction, including: when the first user is an authorized user of the vehicle and the third user is not an authorized user of the vehicle, determining the location of the first area based on environmental perception information and the first instruction.

[0158] For example, the biometric features (such as voiceprint features, facial features, or limb features) of the first user and the third user can be determined according to the first instruction and the third instruction, respectively, and then the identities of the first user and the third user can be determined based on the biometric features.

[0159] In some implementations, if two commands from the same user are detected within a certain time period, and the content indicated by the two commands is inconsistent, the vehicle can be controlled to move based on the user's later command. For example, the time period can be any duration from 1 minute to 3 minutes, or it can be any other duration.

[0160] S1020, Acquire environmental perception information, which indicates the location of objects and areas within a first range of the vehicle.

[0161] For example, the environmental perception information may include the environmental perception information in the foregoing embodiments.

[0162] In some implementations, the environmental perception information can be obtained when it is determined that the first instruction instructs the user to intend for the vehicle to reach a certain target area. For a more detailed explanation of how to determine the user's intent, please refer to the description in method 400, which will not be repeated here.

[0163] For example, the first range can be the maximum range that the vehicle's perception system can perceive; or, the first range can be the range within a circle with the vehicle's center as the center and a radius of a preset length; or, the first range can be the range within a semicircle with the vehicle's center as the center and a radius of a preset length; or, the first range can be other ranges. The straight side of the semicircle coincides with the Y-axis of the vehicle's overall coordinate system, and the center of the semicircle coincides with the origin O of the overall coordinate system. When the vehicle is traveling forward, the arc of the semicircle is located on one side of the positive half-axis of the X-axis. It should be noted that the origin O of the vehicle coordinate system can be located at the projection point of the rear axle center of the vehicle onto the ground. The positive half-axis of the X, Y, and Z axes are respectively the direction of the vehicle's front, the left side, and the direction vertically upwards perpendicular to the vehicle's plane. For example, the aforementioned preset length can be any value between 100 meters and 200 meters, or the preset length can be other values.

[0164] S1030, determine the location of the first area based on environmental perception information and the first instruction.

[0165] The first region corresponds to the first target region.

[0166] In some implementations, S1030 can be further refined as follows: based on environmental perception information, at least one area matching the first instruction is determined, the at least one area including a first area; a prompting device controlling the vehicle prompts at least one element, each of the at least one element indicating the position of each area in the at least one area; in response to a confirmation operation for the first element of the at least one element, the position of the first area is determined; wherein the first element indicates the position of the first area. Wherein, each area in the at least one area satisfies the conditions for vehicle parking or the conditions for vehicle passage.

[0167] The conditions for each area to meet the requirements for vehicle parking include: the width and length of the area are sufficient for vehicle parking; the conditions for each area to meet the requirements for vehicle passage include: the width of each area is sufficient for vehicle passage.

[0168] In one example, taking at least one region as an example, which includes the region indicated by element 604, element 604 can be an example of being regarded as the first element, and the confirmation operation for the first element can be the operation of clicking the "Yes" control in dialog box 605.

[0169] In another example, taking at least one region as an example, which includes the region indicated by element 613, element 613 can be an example of being regarded as the first element, and the confirmation operation for the first element can be the operation of clicking the "Yes" control in dialog box 615.

[0170] In another example, taking at least one area as an example, which includes the area indicated by elements 702, 703 and 704 in the aforementioned embodiments, at least one element may include elements 702, 703 and 704, and the confirmation operation for the first element may be the operation of clicking the first element.

[0171] In some implementations, the first instruction indicates the positional relationship between the first object and the target area, the first object being located within a first range; at least one area is a region matching the first object; each of at least one element further indicates the positional relationship between each region in at least one area and the first object.

[0172] For example, taking the fruit shop in the aforementioned embodiment as the first object, the interface can display at least one element and an element indicating the location of the fruit shop, thereby indicating the positional relationship between the determined area and the fruit shop. Similarly, taking the tree in the aforementioned embodiment as the first object, the interface can display at least one element and an element indicating the location of the tree, thereby indicating the positional relationship between the determined area and the tree.

[0173] In some implementations, S1030 can be refined as follows: Based on a first instruction, determine a first feature associated with a first object, the first feature indicating at least one of the following: type, location; based on environmental perception information, determine a second feature associated with objects included within a first range, the second feature indicating the type and location of each object, and the location of an empty area associated with each object; based on the first and second features, determine the location of the first object; based on the location of the first object, determine the location of the first region. Exemplarily, the first object includes any one of the following: a building, a traffic sign, infrastructure associated with a vehicle, or a natural reference point.

[0174] Specifically, buildings may include, but are not limited to: the gate of the park, a building, or a part of a building or house; traffic signs may include, but are not limited to, road facilities (such as signs) that provide guidance, restriction, warning or instruction information; vehicle-related infrastructure may include, but is not limited to: charging piles or gas stations; natural references may include, but are not limited to: trees and green belts.

[0175] The determined position of the first object can be its coordinates, and the position of the first region can be its coordinates. For more detailed methods on determining the position of the first object and the position of the first region, please refer to [reference needed]. Figure 3 The description of the corresponding part of method 400 will not be repeated here.

[0176] In some implementations, the first instruction indicates that the target area is the shadow area formed by the first object under sunlight, and the shadow area formed by the first object under sunlight includes the first area.

[0177] In one example, the environmental perception information includes image information, and the method further includes: inputting the image information into a first model to obtain the location of the shadow area.

[0178] In another example, the method further includes: obtaining virtual scene rendering information, which is rendered based on environmental perception information, and the virtual scene rendering information indicates the first object and the shadow area; and determining the position of the shadow area based on the virtual scene rendering information.

[0179] For example, the virtual scene rendering information can be information used to present the virtual driving scene displayed by the display device. The virtual scene rendering information can include the coordinates of the first object and the shadow area in the real scene, or the virtual scene rendering information can indicate the mapping relationship between the position of the first object and the shadow area and the coordinates of the real scene.

[0180] In another example, the method further includes: obtaining sunlight position indication information, which indicates the position of the sun in the current time period; and determining the position of the first region based on the position of the first object, including: determining the position of the first region based on the position of the first object and the sunlight position indication information.

[0181] For example, the sunlight position indication information can indicate the sun's altitude angle and azimuth angle. Based on the sun's altitude angle, and the height and size of the first object, the size of the shadow area formed by the first object under the sun can be determined. According to the sun's azimuth angle, the position of the shadow area relative to the first object can be determined.

[0182] A more specific implementation of determining the position of the shadow area of ​​the first object based on its position can be found in the description in the foregoing embodiments, and will not be repeated here.

[0183] S1040, control the vehicle to move towards the first area.

[0184] It is understandable that when the first instruction instructs the vehicle to perform the first action, the vehicle will be controlled to perform the first action when it reaches the first area. The first action can be any one of parking, going straight, turning left, or turning right, or it can be any other action.

[0185] In some implementations, the first instruction also indicates a second intention, which instructs the user in the cabin to perform an action that the vehicle will perform in a second target area, and the second target area is farther away from the vehicle than the first target area. The method further includes: determining the location of the second area according to the first instruction, the second area corresponding to the second target area, and controlling the vehicle to drive towards the second area after the vehicle has driven to the first area.

[0186] When the first instruction includes multiple intentions, determining the location of the area indicated by each intention, and the more specific implementation of controlling vehicle movement, can be found in [reference needed]. Figure 3 The descriptions of the corresponding parts will not be repeated here.

[0187] The navigation method provided in this application can combine sensory information outside the cabin to determine the coordinates of the area that the user inside the cabin expects the vehicle to drive to, and then control the vehicle to drive to that area, so as to achieve the purpose of navigation based on the user's pointed-to and seen points of interest, that is, to achieve "anything visible is achievable", which helps to improve navigation efficiency and the convenience of the user's vehicle use, thereby improving the user experience.

[0188] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between the various embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0189] The above text combines Figures 1 to 10 The methods provided in the embodiments of this application are described in detail below. Figure 11 and Figure 12 The apparatus provided in the embodiments of this application is described in detail. It should be understood that the description of the apparatus embodiments corresponds to the description of the method embodiments. Therefore, for content not described in detail, please refer to the method embodiments above. For the sake of brevity, it will not be repeated here.

[0190] Figure 11 A schematic block diagram of a navigation device 2000 provided in an embodiment of this application is shown. The device 2000 may include units for executing the embodiments described in the foregoing method. Furthermore, each unit in the device 2000 implements a corresponding process of the above-described method embodiments. The device 2000 includes an acquisition unit 2010, which can be used to implement corresponding data acquisition or transmission / reception functions. The device 2000 also includes a processing unit 2020, which can be used to implement corresponding processing functions.

[0191] Optionally, the device 2000 further includes a storage unit, which can be used to store instructions and / or data. The processing unit 2020 can read the instructions and / or data in the storage unit so that the device can perform the relevant actions in the aforementioned method embodiments.

[0192] It should be understood that the specific process of each unit performing the above-mentioned corresponding steps has been described in detail in the above method embodiments, and will not be repeated here for the sake of brevity.

[0193] It should also be understood that the device 2000 described herein is embodied in the form of a functional unit. The terms “module” or “unit” may refer to application-specific ASICs, electronic circuits, processors (e.g., shared processors, proprietary processors, or group processors) and memory for executing one or more software or firmware programs, integrated logic circuits, and / or other suitable components that support the described functions.

[0194] The apparatuses described above have the function of implementing the corresponding steps in the methods described above. These functions can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above; for example, the acquisition unit 2010 can be replaced by a transceiver, and other units, such as the processing unit, can be replaced by a processor, used to execute the relevant processing operations in each method embodiment.

[0195] For example, the acquisition unit 2010 and the processing unit 2020 can be set in Figure 1 In the vehicle 100 shown, or it can also be set in Figure 2 In the illustrated system, more specifically, the acquisition unit 2010 and processing unit 2020 can be located in the destination calculation module 220 and / or the planning and control module 240. Exemplarily, the operations performed by the acquisition unit 2010 and processing unit 2020 can be executed by a single processor, or by different processors. In specific implementations, the one or more processors can be located in... Figure 1 The processor in the vehicle 100 shown; or, the device 2000 described above may be a chip disposed in the vehicle 100.

[0196] In the specific implementation process, the units in the above device can be fully or partially integrated together, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a system-on-a-chip (SoC).

[0197] Figure 12 This is another schematic block diagram of the navigation device provided in the embodiments of this application. Figure 12The illustrated device 2100 may include a processor 2110, a transceiver 2120, and a memory 2130. The processor 2110, transceiver 2120, and memory 2130 are connected via internal interconnects. The memory 2130 stores instructions, and the processor 2110 executes the instructions stored in the memory 2130 to implement the methods described in the above embodiments. Optionally, the memory 2130 may be coupled to the processor 2110 via an interface or integrated with the processor 2110.

[0198] It should be noted that the transceiver 2120 mentioned above may include, but is not limited to, transceiver devices such as input / output interfaces, to realize communication between device 2100 and other devices or communication networks.

[0199] Memory 2130 can be volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM). For example, RAM can be used as an external cache. By way of example and not limitation, RAM includes various forms such as: static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0200] Transceiver 2120 uses transceiver devices, such as but not limited to transceivers, to enable communication between device 2100 and other devices or communication networks to receive / send data / information for implementing the methods in the above embodiments.

[0201] This application also provides an intelligent driving device, which includes the device 2000 or device 2100 in the above embodiments.

[0202] This application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to implement the methods described in the above embodiments of this application.

[0203] This application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to implement the methods described in the above embodiments of this application.

[0204] This application also provides a chip, including circuitry, for performing the methods described in the above embodiments of this application.

[0205] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0206] In the description of the embodiments of this application, unless otherwise stated, " / " means "or", for example, A / B can mean A or B; "and / or" in this document describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In this application, "at least one" means one or more, and "more" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0207] The use of prefixes such as "first" and "second" in this application embodiment is solely for distinguishing different descriptive objects and does not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes to distinguish descriptive objects in this application embodiment does not constitute a limitation on the described objects. The description of the described objects is found in the claims or the context of the embodiments, and the use of such prefixes should not constitute unnecessary restrictions.

[0208] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0209] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between the various embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0210] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0211] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0212] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A navigation method, characterized in that, include: Obtain a first instruction, the first instruction indicating a first intention, the first intention indicating an action that a user in the vehicle's cabin expects the vehicle to perform in a first target area; The first target area is located outside the vehicle's cabin; Acquire environmental perception information, which indicates the location of objects and areas within a first range of the vehicle; Based on the environmental perception information and the first instruction, the location of the first area is determined, and the first area corresponds to the first target area; Control the vehicle to travel towards the first area.

2. The method according to claim 1, characterized in that, Determining the location of the first area based on the environmental perception information and the first instruction includes: Based on the environmental perception information, at least one region matching the first instruction is determined, and the at least one region includes the first region; The vehicle's prompting device prompts at least one element, each of which indicates the location of each area in the at least one region; In response to a confirmation operation for a first element among the at least one elements, the location of the first region is determined; The first element indicates the location of the first region.

3. The method according to claim 2, characterized in that, The first instruction indicates the positional relationship between the first object and the first target area, wherein the first object is located within the first range; The at least one region is a region that matches the first object; Each of the at least one elements also indicates the positional relationship between each region in the at least one region and the first object.

4. The method according to claim 3, characterized in that, Determining the location of the first region based on the environmental perception information and the first instruction includes: According to the first instruction, a first feature associated with the first object is determined, the first feature indicating at least one of the following of the first object: type, location; Based on the environmental perception information, a second feature associated with the objects included in the first range is determined. The second feature indicates the type and location of each object, as well as the location of the empty area associated with each object. The position of the first object is determined based on the first feature and the second feature; The location of the first region is determined based on the location of the first object.

5. The method according to claim 4, characterized in that, The first object includes any of the following: a building, a traffic sign, vehicle-related infrastructure, a road traffic participant, or a natural reference point.

6. The method according to claim 4 or 5, characterized in that, The first instruction indicates that the first target area is the shadow area formed by the first object under sunlight, and the shadow area formed by the first object under sunlight includes the first area.

7. The method according to claim 6, characterized in that, The environmental perception information includes image information, and the method further includes: The image information is input into the first model to obtain the location of the shadow area.

8. The method according to claim 6, characterized in that, The method further includes: Obtain virtual scene rendering information, which is rendered based on the environment perception information, and the virtual scene rendering information indicates the first object and the shadow area; The location of the shadow area is determined based on the virtual scene rendering information.

9. The method according to claim 6, characterized in that, The method further includes: Obtain sunlight position indication information, which indicates the position of the sun in the current time period; Determining the location of the first region based on the location of the first object includes: The location of the first area is determined based on the position of the first object and the sunlight position indication information.

10. The method according to any one of claims 2 to 9, characterized in that, Each of the at least one region satisfies the conditions for vehicle parking or the conditions for vehicle passage.

11. The method according to any one of claims 1 to 10, characterized in that, The users in the cockpit include a first user, and the first instruction includes the first user's voice instructions.

12. The method according to claim 11, characterized in that, The first instruction further includes: an instruction determined based on the first user's first limb movement; and / or an instruction determined based on the first user's first line of sight. Wherein, the time difference between the start time of detecting the first limb movement or the first gaze and the start time of detecting the voice command is less than or equal to a time threshold.

13. The method according to claim 11 or 12, characterized in that, The users in the cockpit also include a second user. During the time period when the first instruction is detected, a second instruction from the second user is also detected. When the second instruction conflicts with the first instruction, determining the location of the first area based on the environmental perception information and the first instruction includes: When the first user is a user in the driver's area of ​​the vehicle, the location of the first area is determined based on the environmental perception information and the first instruction.

14. The method according to any one of claims 11 to 13, characterized in that, The users in the cockpit also include a third user. During the time period when the first instruction is detected, a third instruction from the third user is also detected. When the third instruction conflicts with the first instruction, the method further includes: Based on the first instruction, determine the identity information of the first user; The identity information of the third user is determined according to the third instruction; Determining the location of the first region based on the environmental perception information and the first instruction includes: When the first user is an authorized user of the vehicle and the third user is not an authorized user of the vehicle, the location of the first area is determined based on the environmental perception information and the first instruction.

15. The method according to any one of claims 1 to 14, characterized in that, The first instruction further indicates a second intent, which in turn indicates an action that a user in the cabin expects the vehicle to perform in a second target area, wherein the second target area is farther from the vehicle than the first target area. The method further includes: According to the first instruction, the location of the second region is determined, and the second region corresponds to the second target region; After the vehicle has traveled to the first area, it is then controlled to travel to the second area.

16. A navigation device, characterized in that, include: The acquisition unit is configured to acquire a first instruction, the first instruction having a first intent, the first intent indicating an action that a user in the vehicle's cabin expects the vehicle to perform in a first target area. The first target area is located outside the vehicle's cabin; The acquisition unit is further configured to: acquire environmental perception information, wherein the environmental perception information indicates the location of objects and areas included within a first range of the vehicle; The processing unit is configured to: determine the location of a first region based on the environmental perception information and the first instruction, wherein the first region corresponds to the first target region; Control the vehicle to travel towards the first area.

17. The apparatus according to claim 16, characterized in that, The processing unit is used for: Based on the environmental perception information, at least one region matching the first instruction is determined, and the at least one region includes the first region; The vehicle's prompting device prompts at least one element, each of which indicates the location of each area in the at least one region; In response to a confirmation operation for a first element among the at least one elements, the location of the first region is determined; The first element indicates the location of the first region.

18. The apparatus according to claim 17, characterized in that, The first instruction indicates the positional relationship between the first object and the first target area, wherein the first object is located within the first range; The at least one region is a region that matches the first object; Each of the at least one elements also indicates the positional relationship between each region in the at least one region and the first object.

19. The apparatus according to claim 18, characterized in that, The processing unit is used for: According to the first instruction, a first feature associated with the first object is determined, the first feature indicating at least one of the following of the first object: type, location; Based on the environmental perception information, a second feature associated with the objects included in the first range is determined. The second feature indicates the type and location of each object, as well as the location of the empty area associated with each object. The position of the first object is determined based on the first feature and the second feature; The location of the first region is determined based on the location of the first object.

20. The apparatus according to claim 19, characterized in that, The first object includes any of the following: a building, a traffic sign, vehicle-related infrastructure, a road traffic participant, or a natural reference point.

21. The apparatus according to claim 19 or 20, characterized in that, The first instruction indicates that the first target area is the shadow area formed by the first object under sunlight, and the shadow area formed by the first object under sunlight includes the first area.

22. The apparatus according to claim 21, characterized in that, The environmental perception information includes image information, and the processing unit is further configured to: The image information is input into the first model to obtain the location of the shadow area.

23. The apparatus according to claim 21, characterized in that, The acquisition unit is also used for: Obtain virtual scene rendering information, which is rendered based on the environment perception information, and the virtual scene rendering information indicates the first object and the shadow area; The processing unit is used to: determine the position of the shadow area based on the virtual scene rendering information.

24. The apparatus according to claim 21, characterized in that, The acquisition unit is also used for: Obtain sunlight position indication information, which indicates the position of the sun in the current time period; The processing unit is used for: The location of the first area is determined based on the position of the first object and the sunlight position indication information.

25. The apparatus according to any one of claims 17 to 24, characterized in that, Each of the at least one region satisfies the conditions for vehicle parking or the conditions for vehicle passage.

26. The apparatus according to any one of claims 16 to 25, characterized in that, The users in the cockpit include a first user, and the first instruction includes the first user's voice instructions.

27. The apparatus according to claim 26, characterized in that, The first instruction further includes: an instruction determined based on the first user's first limb movement; and / or an instruction determined based on the first user's first line of sight. Wherein, the time difference between the start time of detecting the first limb movement or the first gaze and the start time of detecting the voice command is less than or equal to a time threshold.

28. The apparatus according to claim 26 or 27, characterized in that, The users in the cockpit also include a second user. During the time period when the first instruction is detected, a second instruction from the second user is also detected. When the second instruction conflicts with the first instruction, the processing unit is used to: When the first user is a user in the driver's area of ​​the vehicle, the location of the first area is determined based on the environmental perception information and the first instruction.

29. The apparatus according to any one of claims 26 to 28, characterized in that, The users in the cockpit also include a third user. During the time period when the first instruction is detected, a third instruction from the third user is also detected. When the third instruction conflicts with the first instruction, the processing unit is further configured to: Based on the first instruction, determine the identity information of the first user; The identity information of the third user is determined according to the third instruction; When the first user is an authorized user of the vehicle and the third user is not an authorized user of the vehicle, the location of the first area is determined based on the environmental perception information and the first instruction.

30. The apparatus according to any one of claims 16 to 29, characterized in that, The first instruction also indicates a second intent, which indicates that the user in the cabin expects the vehicle to perform an action in a second target area, and the second target area is farther from the vehicle than the first target area; the processing unit is further configured to: According to the first instruction, the location of the second region is determined, and the second region corresponds to the second target region; After the vehicle has traveled to the first area, it is then controlled to travel to the second area.

31. A navigation device, characterized in that, include: A processor for executing a computer program stored in memory to cause the apparatus to perform the method as described in any one of claims 1 to 15.

32. A computer-readable storage medium, characterized in that, It stores instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 15.

33. A chip, characterized in that, The chip includes circuitry for performing the method as described in any one of claims 1 to 15.

34. A computer program product, characterized in that, The computer program product includes: computer program code, which, when executed by a processor, implements the method as described in any one of claims 1 to 15.

35. A vehicle, characterized in that, Includes the apparatus as described in any one of claims 16 to 31, or the computer-readable storage medium as described in claim 32, or the chip as described in claim 33, or the vehicle is equipped with the computer program product as described in claim 34.

Images