Navigation control method, device, equipment, medium and program product

Abstract

The invention relates to a navigation control method and device, equipment, a medium and a program product, the navigation control method comprises the following steps: in response to a received target navigation instruction, obtaining a target image, the target image comprising an image of a target interface matched with the target navigation instruction; and determining a target address based on the image content of the target image, and executing a navigation process on the target address. The target address is determined according to the image content of the target image, and the navigation process is executed, so that the integration of the image content understanding capability and the navigation capability is realized, the interaction efficiency and the intelligent level of the navigation process and the linkage capability of the navigation function and the specific application can be enhanced; it is ensured that the target address is matched with the complex or scenarized navigation requirement of the user, and the navigation experience of the user is improved.

Description

Technical Field

[0001] This disclosure relates to the field of navigation, specifically to a navigation control method, apparatus, device, medium, and program product. Background Technology

[0002] Navigation, as a comprehensive service integrating positioning, mapping, and route planning technologies, helps users complete travel and mobility processes efficiently, safely, and conveniently, and has been widely used in personal, commercial, and public service fields. Users need to interact with terminal devices or in-vehicle systems to control the devices or systems to execute a navigation process that matches their navigation needs. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a navigation control method, device, equipment, medium, and program product.

[0004] According to a first aspect of the present disclosure, a navigation control method is provided, the navigation control method comprising: In response to receiving a target navigation instruction, a target image is acquired, the target image including an image of a target interface that matches the target navigation instruction; Based on the image content of the target image, the target address is determined, and a navigation process is performed on the target address.

[0005] In this embodiment, upon receiving a target navigation command, a target image is acquired, and the target address is determined based on the image content. Then, a navigation process is executed at the target address. This enables the execution of a navigation process that matches the navigation requirements corresponding to the target navigation command, providing users with intelligent navigation services. Determining the target address and executing the navigation process based on the image content integrates image content understanding and navigation capabilities. This enhances the interactive efficiency and intelligence of the navigation process, as well as the linkage between navigation functions and specific applications. It ensures that the target address matches the user's complex or scenario-based navigation needs, thus improving the user's navigation experience.

[0006] In some embodiments of this disclosure, the step of acquiring a target image in response to receiving a target navigation command includes: Based on the target navigation instruction, semantic understanding information is determined, and the semantic understanding information includes the instruction content of the target navigation instruction; Based on the semantic understanding information, a target application matching the target navigation command is determined; The target image is determined based on the target application and the semantic understanding information.

[0007] In this embodiment, semantic understanding information is determined based on the target navigation command, and a target application matching the target navigation command is identified based on the semantic understanding information. Then, a target image is acquired based on the target application and the semantic understanding information, thus providing a basis for determining the target address. Determining the target application and acquiring the target image based on the semantic understanding information ensures that both the target application and the target image match the content of the target navigation command, thereby guaranteeing that the image content matches the actual navigation requirements. This improves the efficiency and accuracy of determining the target address, further enhancing the user's navigation experience.

[0008] In some embodiments of this disclosure, determining the target image based on the target application and the semantic understanding information includes: Run the target application and acquire the target image based on the semantic understanding information.

[0009] In this embodiment, by running the target application and acquiring the target image based on semantic understanding information, the acquisition of the target image is achieved, providing a basis for determining the target address. This is beneficial for improving the efficiency and accuracy of acquiring the target image through the operation of the target application.

[0010] In some embodiments of this disclosure, obtaining the target image based on the semantic understanding information includes: Perform at least one application interface interaction operation corresponding to the semantic understanding information for the target application; Based on the application interface interaction, the target interface is determined, and the image of the target interface is used as the target image.

[0011] In this embodiment, at least one application interface interaction operation corresponding to the semantic understanding information is performed on the target application, and the target interface is determined based on the application interface interaction operation. The image of the target interface is then used as the target image, thus achieving target image acquisition and providing a basis for determining the target address. By automatically executing the application interface interaction operation corresponding to the semantic understanding information, it is ensured that the image content of the target image matches the actual navigation requirements, which helps improve the efficiency and accuracy of determining the target address and further enhances the user's navigation experience. In some embodiments of this disclosure, determining the target interface based on the application interface interaction operation includes: Obtain the interface image switched to when the application interface interaction operation is performed, perform image understanding on the interface image, and obtain the image understanding information of the interface image; Based on the image understanding information of the interface image, in response to the result that the switched application interface is the target interface, the switched application interface is determined as the target interface, and the image understanding information includes the image content of the interface image.

[0012] In this embodiment, by acquiring the interface image switched to during the application interface interaction operation and performing image understanding on the interface image, image understanding information of the interface image is obtained. Based on the image understanding information, it is possible to determine when to switch the application interface to the target interface. The image understanding capability is used to realize the identification and judgment of the target interface, which facilitates the timely acquisition of the target image and ensures that the image content of the target image can match the actual navigation needs, which is conducive to further improving the efficiency and accuracy of determining the target address.

[0013] In some embodiments of this disclosure, the step of performing image understanding on the interface image to obtain image understanding information of the interface image includes: An intelligent agent performs image understanding on the interface image to obtain image understanding information of the interface image.

[0014] In this embodiment, an intelligent agent performs image understanding on the interface image, thereby generating image understanding information. This provides a basis for the identification and judgment of the target interface. It is beneficial to use the interactive operation execution capability and image understanding capability of the intelligent agent to avoid the functional reduction of the target application and insufficient understanding of the target navigation instructions, and further improve the navigation performance of the navigation system in multiple dimensions.

[0015] In some embodiments of this disclosure, the navigation control method further includes: The application interface is displayed and the application interface corresponding to the interactive operation of the application interface is executed.

[0016] In this embodiment, by displaying and executing the application interface corresponding to the application interface interaction operation, the user can intuitively observe the execution status of the application interface interaction operation and the recognition status of the target interface through the visualization of the application interface. This facilitates timely correction or confirmation of the target image acquisition process, ensuring the efficiency and accuracy of acquiring the target image and determining the target address, and further improving the user's navigation experience.

[0017] In some embodiments of this disclosure, determining the target address based on the image content of the target image and performing a navigation process on the target address includes: Determine image understanding information for the target image; In response to the image understanding information of the target image including address information, the target address is determined based on the address information, and a navigation process is performed on the target address.

[0018] In this embodiment, by determining the image understanding information of the target image, and when the image understanding information of the target image contains address information, determining the target address based on the address information, and executing the navigation process for the target address, the system realizes the judgment of the scene in which the target image can support the determination of the target address, and combines image content understanding ability with navigation ability, thereby further improving the efficiency and accuracy of determining the target address.

[0019] In some embodiments of this disclosure, determining the target address based on the address information includes: In response to the address information containing an address that meets a first preset condition, the address that meets the first preset condition is determined as the target address.

[0020] In this embodiment, when the image understanding information of the target image contains an address that meets the first preset condition, the address that meets the first preset condition is determined as the target address, thereby realizing the judgment of the scene where the target address can be directly determined, and further improving the efficiency and accuracy of determining the target address.

[0021] In some embodiments of this disclosure, determining the target address based on address information includes: In response to the address information containing two or more addresses, a first prompt message is issued, the first prompt message including prompting the user to provide feedback on the target address; Based on the feedback operation, the target address is determined.

[0022] In this embodiment, when the address information contains more than two addresses, a first prompt message is issued to prompt the user to provide feedback on the target address. Based on the feedback, the target address is determined. This enables the judgment of scenarios where further selection or confirmation of the target address is required, and the further selection or confirmation of the target address is achieved by using the first prompt message and feedback, thus ensuring the efficiency and accuracy of determining the target address.

[0023] In some embodiments of this disclosure, determining the target address based on the address information further includes: In response to the fact that the address information contains two or more addresses, a pre-selected address is determined based on a second preset condition, and the first prompt information includes prompting the user to confirm whether the pre-selected address is the target address; In response to receiving a user's instruction to confirm the pre-selected address, the pre-selected address is determined as the target address.

[0024] In this embodiment, when the address information contains more than two addresses, a pre-selected address is determined based on the second preset condition, and a first prompt message is sent asking the user whether to use the pre-selected address as the target address. When the user's confirmation instruction for the pre-selected address is detected, the pre-selected address is determined as the target address. In scenarios where further selection or determination of the target address is required, the determination of the target address is achieved through the automatic determination of the pre-selected address and the user's confirmation instruction for the pre-selected address, ensuring the accuracy of the target address in complex scenarios.

[0025] In some embodiments of this disclosure, determining the target address based on address information further includes: In response to receiving a user's negative instruction for the pre-selected address, a second prompt message is issued, the second prompt message including prompting the user to select a target address from at least some of the addresses; In response to the user's selection operation on the second prompt information, the address corresponding to the selection operation is determined as the target address.

[0026] In this embodiment, when the received user feedback information is a negative instruction for the pre-selected address, a second prompt message including selecting the target address from at least some addresses is issued. When the address selection instruction is received, the address corresponding to the address selection instruction is determined as the target address. In scenarios where further selection or determination of the target address is required, the negative instruction and selection operation realize the negation of the pre-selected address and the re-selection of the target address, ensuring the accuracy of the target address in complex scenarios.

[0027] In some embodiments of this disclosure, determining the target address based on the address information includes: The intelligent agent determines the target address based on the address information.

[0028] In this embodiment, the intelligent agent determines the target address based on the address information, thus realizing the determination of the target address and providing a basis for the execution of the navigation process. This helps to utilize the address extraction and navigation capabilities of the intelligent agent to avoid inefficient interaction and lack of collaboration, and further improves the navigation performance of the navigation system in multiple dimensions.

[0029] In some embodiments of this disclosure, the navigation control method further includes: In response to the fact that the image understanding information of the target image does not contain the address information, a third prompt message is issued based on the image content understanding information of the target image.

[0030] In this embodiment, when the image understanding information of the target image does not contain address information, a third prompt is issued based on the image understanding information of the target image. This enables the determination of scenarios where the target image does not support the determination of the target address, and the third prompt is used to provide prompts to the user for the scenario and the image content of the target image. This allows the user to adjust their navigation needs in a timely manner, which is beneficial to improving the user's navigation experience.

[0031] According to a second aspect of the present disclosure, a navigation control method is provided, characterized in that the navigation control method includes: In response to receiving a target navigation instruction sent by the application, a target image is acquired, the target image including an image of a target interface that matches the target navigation instruction; Based on the image content of the target image, the target address is determined and sent to the application.

[0032] In this embodiment, upon receiving a target navigation instruction from the application, the target image is acquired, and the target address is determined based on the image content. The target address is then sent to the application so that the application can execute the corresponding navigation process. Through the collaboration between the application and the server, the integration of image content understanding and navigation capabilities is achieved, thereby fulfilling complex or scenario-based navigation needs. The operation is simple, the interaction is efficient, and the user's operation steps are greatly reduced, which helps to improve the user's navigation experience.

[0033] According to a third aspect of the present disclosure, a navigation control device is provided, the navigation control device comprising: The acquisition module is configured to acquire a target image in response to receiving a target navigation instruction, the target image including an image of a target interface that matches the target navigation instruction; A navigation module is used to determine a target address based on the image content of the target image and to perform a navigation process on the target address.

[0034] In some embodiments of this disclosure, the acquisition module is further configured to: Based on the target navigation instruction, semantic understanding information is determined, and the semantic understanding information includes the instruction content of the target navigation instruction; Based on the semantic understanding information, a target application matching the target navigation command is determined; The target image is determined based on the target application and the semantic understanding information.

[0035] In some embodiments of this disclosure, the acquisition module is further configured to: Perform at least one application interface interaction operation corresponding to the semantic understanding information for the target application; Based on the application interface interaction, the target interface is determined, and the image of the target interface is used as the target image.

[0036] In some embodiments of this disclosure, the acquisition module is further configured to: Obtain the interface image switched to when the application interface interaction operation is performed, perform image understanding on the interface image, and obtain the image understanding information of the interface image; Based on the image understanding information of the interface image, in response to the application interface being switched to as the target interface, the switched application interface is determined as the target interface, and the image understanding information includes the image content of the interface image.

[0037] In some embodiments of this disclosure, the acquisition module is further configured to: An intelligent agent performs image understanding on the interface image to obtain image understanding information of the interface image.

[0038] In some embodiments of this disclosure, the navigation control device further includes: The display module is used to display the application interface corresponding to the application interface interaction operation.

[0039] In some embodiments of this disclosure, the navigation module is further configured to: Determine image understanding information for the target image; In response to the image understanding information of the target image including address information, the target address is determined based on the address information, and a navigation process is performed on the target address.

[0040] In some embodiments of this disclosure, the navigation module is further configured to: In response to the address information containing an address that meets a first preset condition, the address that meets the first preset condition is determined as the target address.

[0041] In some embodiments of this disclosure, the navigation module is further configured to: In response to the address information containing two or more addresses, a first prompt message is issued, the first prompt message including prompting the user to provide feedback on the target address; Based on the feedback operation, the target address is determined.

[0042] In some embodiments of this disclosure, the navigation module is further configured to: In response to the fact that the address information contains two or more addresses, a pre-selected address is determined based on a second preset condition, and the first prompt information includes prompting the user to confirm whether the pre-selected address is the target address; In response to receiving a user's instruction to confirm the pre-selected address, the pre-selected address is determined as the target address.

[0043] In some embodiments of this disclosure, the navigation module is further configured to: In response to receiving a user's negative instruction for the pre-selected address, a second prompt message is issued, the second prompt message including prompting the user to select a target address from at least some of the addresses; In response to the user's selection operation on the second prompt information, the address corresponding to the selection operation is determined as the target address.

[0044] In some embodiments of this disclosure, the navigation module is further configured to: The intelligent agent determines the target address based on the address information. In some embodiments of this disclosure, the navigation module is further configured to: In response to the fact that the image understanding information of the target image does not contain the address information, a third prompt message is issued based on the image content understanding information of the target image.

[0045] According to a fourth aspect of the present disclosure, an electronic device is provided, the electronic device comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute the navigation control method as described in the first aspect.

[0046] According to a fifth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the navigation control method as described in the first aspect.

[0047] According to a sixth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the navigation control method as described in the first aspect.

[0048] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: Upon receiving a target navigation command, an image of the matching target interface can be obtained as the target image based on the target navigation command. The target address is determined based on the image content of the target image, and the navigation process is executed. The user only needs to issue a target navigation command. Through the integration of image content understanding and navigation capabilities, the user's navigation intent can be determined, thereby fulfilling complex or scenario-based navigation needs. The operation is simple, the interaction is efficient, and the user's operation steps are greatly reduced, improving the safety of the driving process. The target interface of the corresponding target application can be obtained based on the target navigation command, realizing cross-application collaboration and the linkage between navigation functions and specific applications. This enables full-function support for target applications in scenarios such as vehicles, as well as the configuration and access of system frameworks and standardized interfaces, reducing expansion and maintenance costs, improving iteration and update speed, and achieving a seamless experience across devices, which is beneficial to improving the user's navigation experience.

[0049] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0051] Figure 1 This is a schematic diagram of a navigation system according to an exemplary embodiment.

[0052] Figure 2 This is a flowchart illustrating a navigation control method according to an exemplary embodiment.

[0053] Figure 3 This is a flowchart illustrating, according to an exemplary embodiment, the process of acquiring a target image in response to receiving a target navigation command.

[0054] Figure 4 This is a flowchart illustrating the acquisition of a target image based on semantic understanding information according to an exemplary embodiment.

[0055] Figure 5 This is a flowchart illustrating, according to an exemplary embodiment, how to determine a target interface based on application interface interaction operations.

[0056] Figure 6 This is a schematic diagram of an application interface according to an exemplary embodiment.

[0057] Figure 7 This is a schematic diagram of an application interface according to another exemplary embodiment.

[0058] Figure 8This is a schematic diagram of an application interface according to another exemplary embodiment.

[0059] Figure 9 This is a flowchart illustrating, according to an exemplary embodiment, how to determine a target interface based on application interface interaction operations.

[0060] Figure 10 This is a flowchart illustrating a process of determining a target address based on the image content of a target image and performing a navigation process on the target address, according to an exemplary embodiment.

[0061] Figure 11 This is a flowchart illustrating a process for determining a target address based on address information, according to an exemplary embodiment.

[0062] Figure 12 This is a flowchart illustrating the determination of a target address based on address information, according to another exemplary embodiment.

[0063] Figure 13 This is a flowchart illustrating a navigation control method according to another exemplary embodiment.

[0064] Figure 14 This is a block diagram of a navigation control device according to an exemplary embodiment.

[0065] Figure 15 This is a block diagram of an electronic device according to an exemplary embodiment.

[0066] In the picture: 10 - Acquisition module; 20 - Navigation module; 400 - Electronic device; 402 - Processing component; 404 - Memory; 406 - Power supply component; 408 - Multimedia component; 410 - Audio component; 412 - Input / output interface; 414 - Sensor component; 416 - Communication component; 420 - Processor. Detailed Implementation

[0067] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0068] Navigation, as a rapidly developing and increasingly mature comprehensive service, helps users complete their travel and movement efficiently, safely, and conveniently, and has gradually become an indispensable tool in people's daily lives. Users need to interact with terminal devices or in-vehicle systems to control the corresponding navigation process.

[0069] In related technologies, users need to provide specific addresses or store names through interactive operations such as voice input, text input, or control selection in navigation software to achieve the positioning-planning-guidance navigation process. During automatic navigation control, only simple basic commands can be executed. For example, after setting the specific location of home, the basic command "Navigate home" can be issued to achieve automatic navigation home. However, the navigation control methods using these technologies cannot directly understand complex or ambiguous commands. For example, when a user has specific navigation needs and scenarios such as "Navigate to stores saved in app A," due to the fragmented ecosystem of applications, cross-application collaboration is not possible. It requires semi-automatic or manual methods such as multiple rounds of command input or complex interactive operations to determine the address or store matching the navigation needs. For example, for the navigation request "Navigate to stores saved in app A," the user needs to first open app A, then retrieve the stores saved in app A, then open the navigation software and input the saved stores into the navigation software. The operation is very complex, the interaction is inefficient, the level of intelligence is insufficient, and there are safety hazards during driving. Using customized applications results in severely limited functionality, high integration costs with new platforms, slow iteration updates, and difficulty in achieving a seamless experience across devices. This leads to issues such as feature reduction, user experience degradation, and high expansion and maintenance costs, significantly impacting the user's navigation experience.

[0070] Based on this, this disclosure provides a navigation control method. Upon receiving a target navigation command, it can acquire a matching target interface image as the target image, determine the target address based on the image content, and execute the navigation process. The user only needs to issue the target navigation command; through the integration of image content understanding and navigation capabilities, the user's navigation intent can be determined, thereby fulfilling complex or scenario-based navigation needs. The method is simple to operate, highly efficient in interaction, significantly reduces user steps, and improves driving safety. The target interface of the corresponding target application can be obtained based on the target navigation command, realizing cross-application collaboration and linkage between navigation functions and specific applications. This enables full-function support for target applications in scenarios such as in-vehicle systems, as well as configuration and access to system frameworks and standardized interfaces, reducing expansion and maintenance costs, improving iteration and update speed, and achieving a seamless cross-device experience, thus enhancing the user's navigation experience.

[0071] In one exemplary embodiment, a navigation control method is provided, applied to a navigation system, with reference to... Figure 1As shown, a navigation system may include an application and a server. The application may include a terminal device or a vehicle infotainment system. Users can input commands through interaction with the application, and the server determines the target address, allowing the application to execute the navigation process at the target address. It should be noted that there can be one or more applications. For example, there may be one application, where command input, target application execution, and navigation execution can be achieved through a separate terminal device such as a mobile phone or a vehicle infotainment system. Alternatively, there may be multiple applications, where commands can be input through the vehicle infotainment system, the target application on the mobile phone or other terminal device can be invoked, and the navigation process can then be executed through the vehicle infotainment system.

[0072] refer to Figure 2 As shown, the navigation control method includes: S100, in response to receiving a target navigation command, acquire a target image, the target image including an image of a target interface that matches the target navigation command.

[0073] In step S100, upon receiving a target navigation instruction, a target image, i.e., an image of the target interface matching the target navigation instruction, is acquired as the basis for subsequently determining the target address. This ensures that the target address matches the navigation requirements for the target interface. The target interface, for example, is an interface within a target application that matches the target navigation instruction and contains address-related information matching the navigation requirements indicated by the target navigation instruction. For instance, the target navigation instruction can be a voice instruction. The application can receive the target navigation instruction through its microphone or other audio acquisition device and, through collaboration with the server, acquire the target image based on the target navigation instruction. The target application can be an application with a collection function that matches the target navigation instruction. Examples include photo album applications, social applications, and lifestyle service applications. A photo album application can implement its collection function by saving images or screenshots, while social applications and lifestyle service applications can implement their collection function by collecting posts or stores. The target interface can be the application interface of the target application containing address-related information. When the target navigation instruction is "Navigate to stores collected in application A," the target interface can be the collection interface of application A.

[0074] S200: Based on the image content of the target image, determine the target address and perform a navigation process for the target address.

[0075] In step S200, since the target image is an image of the target interface that matches the target navigation command, its image content can reflect the navigation requirements corresponding to the target navigation command. The target address that matches the target navigation command can be determined based on the image content of the target image, and a navigation process including positioning, planning and guidance can be performed on the target address to meet the user's navigation requirements for the target interface and realize intelligent navigation services.

[0076] For example, the server can use the Map Agent Skill module to call its image understanding service to determine the image understanding information of the target image, and then call its navigation service to determine the corresponding target address. The Map Agent Skill module is a module integrated into the server that implements map functions by executing a specific thread. When the target image is an image from the favorites interface of application A, the image understanding service can determine whether the target image contains address information, and the navigation service can extract the target address from the address information.

[0077] Understandably, compared to related technologies that require multiple rounds of navigation commands or step-by-step interactive operations to determine the address or store matching the navigation needs, the navigation control method shown in the above embodiments obtains a target image that matches the target navigation command, i.e., the navigation needs, and determines the target address based on the image content of the target image. It utilizes the image understanding and navigation capabilities of the navigation system to understand and execute complex target navigation commands, thereby realizing the linkage between the navigation function and the target application and the end-to-end automated process. This can compress the complex interactive process in related technologies into a single command, avoiding safety hazards in driving scenarios caused by multiple interactive operations and eye shifts. It can improve the navigation performance of the navigation system in multiple dimensions such as experience, safety, intelligence, cost, and ecosystem.

[0078] In this embodiment, upon receiving a target navigation command, a matching target interface image can be obtained as the target image. The target address is determined based on the image content, and the navigation process is executed. The user only needs to issue the target navigation command; through the integration of image content understanding and navigation capabilities, the user's navigation intent can be determined, thus fulfilling complex or scenario-based navigation needs. The operation is simple, the interaction is efficient, significantly reducing user steps and improving driving safety. The target navigation command can retrieve the target interface of the corresponding target application, enabling cross-application collaboration and linkage between navigation functions and specific applications. This allows for full functional support of target applications in scenarios such as in-vehicle systems, as well as configuration and access to the system framework and standardized interfaces, reducing expansion and maintenance costs, increasing iteration and update speed, and achieving a seamless cross-device experience, ultimately enhancing the user's navigation experience.

[0079] In some embodiments, reference Figure 3 As shown, in response to receiving a target navigation command, acquiring a target image includes: S110. Based on the target navigation instructions, determine the semantic understanding information, which includes the instruction content of the target navigation instructions.

[0080] In step S110, semantic understanding information is determined based on the target navigation instruction. This semantic understanding information, containing the instruction content of the target navigation instruction, reflects the navigation requirements corresponding to the target navigation instruction, thus achieving semantic analysis of the target navigation instruction. For example, when the target navigation instruction is a voice instruction, it can be converted into corresponding code. The instruction content can then be determined by extracting specific fields using tools such as Map Agent Parser, generating corresponding semantic understanding information. Map Agent Parser is a tool that enables information interaction between the application and the server through semantic analysis. When the target navigation instruction is "Navigate to a store saved in application A," the semantic understanding information could be "The target address is the address of a store saved in application A."

[0081] S120. Based on semantic understanding information, determine the target application that matches the target navigation instructions.

[0082] In step S120, since semantic understanding information can reflect the navigation requirements corresponding to the target navigation command, the target application that matches the target navigation command, i.e., the navigation requirements, can be determined based on the semantic understanding information. For example, when the semantic understanding information is "the target address is the address of a store that application A has saved", then application A is determined as the target application.

[0083] S130. Based on the target application and semantic understanding information, determine the target image.

[0084] In step S130, after the target application is determined, the application can obtain the target image based on the instruction content included in the semantic understanding information during the operation or invocation of the target application through autonomous operation or interface call, thereby ensuring that the target image is the image of the target interface that matches the target navigation instruction, and providing a basis for determining the target address.

[0085] For example, when the application is a terminal device, the target application can be run directly on the terminal device. When the application is an in-vehicle infotainment system, the target application can be run directly on the in-vehicle infotainment system, or the application interface of the terminal device can be called through the interaction between the in-vehicle infotainment system and the terminal device. When running the target application, the corresponding application interface interaction operation can be automatically executed for the target application based on semantic understanding information. When the semantic understanding information is "the target address is the address of a store that application A has saved", the application interface interaction operation of opening the saved page can be automatically executed for application A, and the image of the saved page switched to after executing the application interface interaction operation can be used as the target image.

[0086] In this embodiment, semantic understanding information is determined based on the target navigation command, and a target application matching the target navigation command is identified based on the semantic understanding information. Then, a target image is acquired based on the target application and the semantic understanding information, thus providing a basis for determining the target address. Determining the target application and acquiring the target image based on the semantic understanding information ensures that both the target application and the target image match the content of the target navigation command, thereby guaranteeing that the image content matches the actual navigation requirements. This improves the efficiency and accuracy of determining the target address, further enhancing the user's navigation experience.

[0087] In some embodiments, determining a target image based on a target application and semantic understanding information includes: running the target application and acquiring the target image based on semantic understanding information.

[0088] When determining the target image based on the target application and semantic understanding information, the target application matching the target navigation instruction can be run first through the application terminal or virtual display screen that receives the target navigation instruction. During the operation of the target application, the target image is obtained based on the instruction content included in the semantic understanding information, thereby ensuring that the target image is the image of the target interface that matches the target navigation instruction, providing a basis for determining the target address.

[0089] In this embodiment, by running the target application and acquiring the target image based on semantic understanding information, the acquisition of the target image is achieved, providing a basis for determining the target address. This is beneficial for improving the efficiency and accuracy of acquiring the target image through the operation of the target application.

[0090] In some embodiments, determining a target image based on a target application and semantic understanding information includes: calling the application interface of the target application and obtaining the target image based on semantic understanding information.

[0091] When determining the target image based on the target application and semantic understanding information, the application receiving the target navigation command can first call the application interface of other devices for the target application. During the interface call process, the target image is obtained based on the command content included in the semantic understanding information. This ensures that the target image is an image of the target interface that matches the target navigation command, providing a basis for determining the target address. For example, the vehicle's infotainment system can call the application interface of the mobile phone for the target application to obtain the target image based on the semantic understanding information.

[0092] In this embodiment, the application interface of the target application is called, and the target image is obtained based on semantic understanding information. This achieves the acquisition of the target image, provides a basis for determining the target address, and enables a seamless experience across devices using the application interface.

[0093] In some embodiments, reference Figure 4 As shown, acquiring the target image based on semantic understanding information includes: S131. Perform at least one application interface interaction operation corresponding to the semantic understanding information for the target application.

[0094] In step S131, since semantic understanding information can reflect navigation needs associated with the target application, one or more application interface interaction operations can be automatically executed for the target application based on the semantic understanding information, thereby enabling the retrieval of specific function pages of the target application. For example, the automatic execution of application interface interaction operations can be achieved through a user interface agent (UI Agent). A single application interface interaction operation can be executed to achieve one switch of the application interface, or multiple application interface interaction operations can be executed to achieve multiple switches of the application interface.

[0095] S132. Based on the application interface interaction operation, determine the target interface and use the image of the target interface as the target image.

[0096] In step S132, after the user performs an interactive operation on the application interface, the application interface changes accordingly. Based on the interactive operation on the application interface, the target interface that matches the target navigation instruction can be determined, and the image of the target interface can be captured as the target image to provide a basis for determining the target address.

[0097] In this embodiment, at least one application interface interaction operation corresponding to the semantic understanding information is performed on the target application. Based on the application interface interaction operation, the target interface is determined, and its image is used as the target image, thus achieving target image acquisition and providing a basis for determining the target address. By automatically executing the application interface interaction operation corresponding to the semantic understanding information, it is ensured that the image content of the target image matches the actual navigation requirements, which helps improve the efficiency and accuracy of determining the target address and further enhances the user's navigation experience.

[0098] In some embodiments, reference Figure 5 As shown, based on the application interface interaction, the target interface is determined, including: S132-1, Perform application interface interaction operations to switch the application interface of the target application.

[0099] In step S132-1, each time an application interface interaction operation is performed, the target application can switch the current application interface, thereby enabling the retrieval of specific function pages for the target application. For example, the automatic execution of the application interface interaction operation can be achieved through a user interface agent (UI Agent). A single application interface interaction operation can be performed to achieve one switch of the application interface, or multiple application interface interaction operations can be performed to achieve multiple switches of the application interface.

[0100] S132-2. Determine the target interface based on the application interface being switched.

[0101] In step S132-2, when the target application switches to an application interface matching the target navigation instruction based on the interface interaction operation, this application interface can be determined as the target interface, and the image of the target interface can be used as the target image to achieve the determination of the target interface and the acquisition of the target image. It can be understood that switching to the target interface can be achieved by performing one application interface interaction operation; that is, after running the target application, performing one application interface interaction operation will switch the target application to the target interface. Alternatively, multiple application interface interaction operations can be performed before switching to the target interface. For example, taking a social application as the target application, when the semantic understanding information is "the target address is the address of a store saved by application A," then for application A... Figure 6 The application interface shown automatically executes the application interface interaction operation of clicking "My" (control E), switching to the interface shown. Figure 7 The application interface shown will then automatically execute the application interface interaction operation of clicking "Favorite" (control F), switching to the interface shown. Figure 8 The application interface shown is used, and the image of the "Favorites" application interface is used as the target image.

[0102] In this embodiment, by performing application interface interaction operations to switch the application interface of the target application, and determining the target interface based on the switched application interface, a basis is provided for obtaining the target image. This enables the target application to automatically switch the target interface that matches the target navigation command, ensuring that the image content of the target image matches the actual navigation needs. This helps to improve the efficiency and accuracy of determining the target address and further enhances the user's navigation experience.

[0103] In some embodiments, reference Figure 9 As shown, based on the application interface interaction, the target interface is determined, including: S133. Obtain the interface image switched to when performing the application interface interaction operation, perform image understanding on the interface image, and obtain the image understanding information of the interface image.

[0104] In step S133, the interface image of the application interface switched to each time an application interface interaction operation is performed is acquired, and image understanding is performed on each interface image to obtain image understanding information for each interface image, including the image content of the corresponding interface image. For example, the acquisition of interface images can be achieved through a Push service for data interaction, and the image understanding of interface images can be achieved through a user interface agent, i.e., a UI Agent.

[0105] S134. Based on the image understanding information of the interface image, in response to the application interface being switched to as the target interface, the switched application interface is determined as the target interface, and the image understanding information includes the image content of the interface image.

[0106] In step S134, for each application interface image switched to by performing an application interface interaction operation, the corresponding image understanding information is used to determine when to switch the application interface to the target interface, that is, to determine whether the current application interface is the target interface. This realizes the identification and judgment of the target interface and provides a basis for determining the target image.

[0107] For example, the application client can capture and upload the interface image of the application interface switched to each time an application interface interaction operation is performed. This allows the server to perform image understanding each time it receives an interface image, obtain image understanding information of the interface image, and determine whether the switched application interface is the target interface based on conditions such as whether the image content of the interface image included in the image understanding information contains address information that meets the requirements. If it is determined that it is not the target interface, the subsequent application interface interaction operation is continued; if it is determined that it is the target interface, the corresponding interface image is used as the target image.

[0108] In this embodiment, by acquiring the interface image switched to during the application interface interaction operation and performing image understanding on the interface image, image understanding information of the interface image is obtained. Based on the image understanding information, it is possible to determine when to switch the application interface to the target interface. The image understanding capability is used to realize the identification and judgment of the target interface, which facilitates the timely acquisition of the target image and ensures that the image content of the target image can match the actual navigation needs, which is conducive to further improving the efficiency and accuracy of determining the target address.

[0109] In some embodiments, performing image understanding on the interface image to obtain image understanding information of the interface image includes: performing image understanding on the interface image using an intelligent agent to obtain image understanding information of the interface image.

[0110] When performing image understanding on the interface, the navigation system can use a server-side intelligent agent to perform image understanding on the interface. By leveraging the image content understanding capabilities of the intelligent agent, image understanding information can be generated, providing a basis for the identification and judgment of the target interface and the subsequent determination of the target address.

[0111] For example, the intelligent agent can be a general-purpose user interface agent, i.e., a UI Agent. In this case, using a general-purpose user interface agent to perform application interface interaction operations and image understanding processes is beneficial for accessing existing application frameworks, ensuring the integrity of the native functions of each application, and ensuring the efficiency and accuracy of generating image understanding information.

[0112] In this embodiment, an intelligent agent performs image understanding on the interface image, thereby generating image understanding information. This provides a basis for the identification and judgment of the target interface. It is beneficial to use the interactive operation execution capability and image understanding capability of the intelligent agent to avoid the functional reduction of the target application and insufficient understanding of the target navigation instructions, and further improve the navigation performance of the navigation system in multiple dimensions.

[0113] In some embodiments, the navigation control method further includes: displaying the application interface corresponding to the application interface interaction operation.

[0114] Each time an application interface interaction is performed that causes the target application to switch to its corresponding interface, the display device on the application terminal can show the application interface corresponding to the interaction. This allows the user to intuitively observe the interface corresponding to the interaction and to accurately assess the execution of the interaction and the recognition of the target interface. For example, when the application terminal is a mobile phone or tablet, the interface can be displayed on the device's screen; when the application terminal is an in-vehicle infotainment system, the interface can be displayed on the system's in-vehicle screen.

[0115] In this embodiment, by displaying and executing the application interface corresponding to the application interface interaction operation, the user can intuitively observe the execution status of the application interface interaction operation and the recognition status of the target interface through the visualization of the application interface. This facilitates timely correction or confirmation of the target image acquisition process, ensuring the efficiency and accuracy of acquiring the target image and determining the target address, and further improving the user's navigation experience.

[0116] In other embodiments, the application can execute interactive operations on each application interface by running the target application in the background or by using a virtual display screen, without displaying the application interface corresponding to each interactive operation. This directly shows the navigation process to the user, which helps save display power consumption and reduce the user's visual burden.

[0117] In some embodiments, reference Figure 10 As shown, based on the image content of the target image, the target address is determined, and a navigation process is performed on the target address, including: S210. Determine the image understanding information for the target image.

[0118] S220. In response to the image understanding information of the target image including address information, the target address is determined based on the address information, and a navigation process is performed on the target address.

[0119] In steps S210 and S220, as described above, image understanding of the interface image can be performed to obtain image understanding information, including the image content of the interface image. During the process of determining the target address after obtaining the target image, the target address can be determined based on the image content of the interface image corresponding to the target interface, thus providing a basis for the execution of the navigation process. When the image understanding information of the target image contains address information, it means that the image content of the target image contains address-related content. The target address can be determined based on the address information through address extraction or other methods. Therefore, the target address is determined based on the address, and the navigation process is executed at the target address.

[0120] For example, the server can determine the image understanding information of the target image through the image content understanding capabilities provided by the user interface agent. The image understanding information may include, for example, text, labels, and names in the interface image. If it is determined that the image content of the target image contains address-related content, then it is determined that the current scenario requires navigation. The server then invokes the navigation agent by constructing instructions, so as to determine the target address based on the address information using the precise address extraction capabilities provided by the navigation agent, and execute the navigation process to the target address.

[0121] In this embodiment, by determining the image understanding information of the target image, and when the image understanding information of the target image contains address information, determining the target address based on the address information, and executing the navigation process for the target address, the system realizes the judgment of the scene in which the target image can support the determination of the target address, and combines image content understanding ability with navigation ability, thereby further improving the efficiency and accuracy of determining the target address.

[0122] In other embodiments, the navigation control method further includes: issuing a third prompt based on the image content understanding information of the target image in response to the fact that the image understanding information of the target image does not contain address information.

[0123] When the image understanding information of the target image does not contain address information, it means that the image content of the target image does not contain content related to the address, and the target address cannot be determined based on the image content of the target image. In this case, a third prompt message is issued based on the image content understanding information of the target image to achieve the transparent transmission of the image understanding result and to inform the user that the navigation process cannot be executed smoothly at present.

[0124] For example, if the server determines that the image content of the target image does not contain content related to the address through the image content understanding capability provided by the user interface agent, it will issue a third prompt message through voice broadcast, screen display, etc., to inform the user that the navigation process cannot be performed at present, and display the image content of the target image.

[0125] In this embodiment, when the image understanding information of the target image does not contain address information, a third prompt is issued based on the image understanding information of the target image. This enables the determination of scenarios where the target image does not support the determination of the target address, and the third prompt is used to provide prompts to the user for the scenario and the image content of the target image. This allows the user to adjust their navigation needs in a timely manner, which is beneficial to improving the user's navigation experience.

[0126] In some embodiments, determining a target address based on address information includes: in response to the address information containing an address that meets a first preset condition, determining the address that meets the first preset condition as the target address.

[0127] When the address information contains an address that meets the first preset condition, it can be determined that the current scenario is one in which the target address can be directly determined. The address that meets the first preset condition can be directly used as the target address without performing further address selection. This directly determines the target address and provides a basis for the execution of the navigation process.

[0128] For example, the first preset condition could be that the corresponding address is the only address among multiple addresses included in the address information that meets the user's or scenario's needs. For instance, the user could choose the address of a restaurant with a mild flavor as the target address based on their preference. If the address information includes two addresses, one for a coconut chicken restaurant and the other for a spicy hot pot restaurant, the address of the coconut chicken restaurant with a mild flavor could be determined as the target address based on the user's memory of their preferences. Alternatively, the first preset condition could also be that the corresponding address is the only address included in the address information, and there are no other address options; in this case, the unique address included in the address information would be directly determined as the target address.

[0129] In this embodiment, when the image understanding information of the target image contains an address that meets the first preset condition, the address that meets the first preset condition is determined as the target address, thereby realizing the judgment of the scene where the target address can be directly determined, and further improving the efficiency and accuracy of determining the target address.

[0130] In other embodiments, reference is made to... Figure 11 As shown, the target address is determined based on the address information, including: S221. In response to the address information containing two or more addresses, issue a first prompt message, the first prompt message including prompting the user to provide feedback on the target address.

[0131] In step S221, when the address information contains more than two addresses, further address selection is required. A first prompt message is then issued to prompt the user to select or confirm the target address. It should be noted that "more than two addresses" includes two separate addresses.

[0132] S222. Determine the target address based on feedback operations.

[0133] In step S222, when a feedback operation is detected, it means that the user has selected or determined the target address. Based on the feedback operation, the target address is determined, thus providing a basis for the execution of the navigation process.

[0134] For example, when the address information includes address B, address C and address D, the control application will issue a first prompt message such as "There are multiple addresses, please select one" or "There are multiple addresses, please confirm whether to use address B as the target address" through voice broadcast or screen display. The user can perform feedback operations through interaction with the application to determine the final target address based on the feedback operations.

[0135] In this embodiment, when the address information contains more than two addresses, a first prompt message is issued to prompt the user to provide feedback on the target address. Based on the feedback, the target address is determined. This enables the judgment of scenarios where further selection or confirmation of the target address is required, and the further selection or confirmation of the target address is achieved by using the first prompt message and feedback, thus ensuring the efficiency and accuracy of determining the target address.

[0136] In some embodiments, determining the target address based on address information further includes: in response to the address information containing two or more addresses, determining a pre-selected address based on a second preset condition, wherein the first prompt information includes prompting the user to confirm whether the pre-selected address is the target address. In response to receiving a user's instruction to confirm the pre-selected address, the pre-selected address is determined as the target address.

[0137] As mentioned earlier, when the address information contains more than two addresses, further address selection is required. First, a pre-selected address is automatically determined based on a second preset condition. This pre-selected address can be automatically determined, for example, by an intelligent agent. The first prompt message then asks the user to confirm whether the pre-selected address is the target address.

[0138] When the detected feedback operation is a confirmation instruction for the pre-selected address, it means that the user agrees to the pre-selected address and the pre-selected address can be used as the target address. In this way, the pre-selected address is determined as the target address. In scenarios where further confirmation of the target address is required, the target address is determined through the automatic determination of the pre-selected address and the user's confirmation of the pre-selected address.

[0139] The second preset condition may include, for example, the conditions that the address must meet to meet the user's or scenario's requirements. Based on the historical data corresponding to the user's or scenario's requirements, a pre-selected address can be initially selected from multiple addresses according to the second preset condition. The user is then prompted to confirm the pre-selected address through the first prompt information. Thus, if the pre-selected address meets the requirements, the target address can be determined through the confirmation instruction of the pre-selected address.

[0140] In this embodiment, when the address information contains more than two addresses, a pre-selected address is determined based on the second preset condition, and a first prompt message is sent asking the user whether to use the pre-selected address as the target address. When the user's confirmation instruction for the pre-selected address is detected, the pre-selected address is determined as the target address. In scenarios where further selection or determination of the target address is required, the determination of the target address is achieved through the automatic determination of the pre-selected address and the user's confirmation instruction for the pre-selected address, ensuring the accuracy of the target address in complex scenarios.

[0141] In some embodiments, reference Figure 12As shown, determining the target address based on address information also includes: S223. In response to receiving a user's negative instruction for the pre-selected address, a second prompt message is issued, the second prompt message including prompting the user to select a target address from at least some of the addresses.

[0142] In step S223, when the feedback operation is detected as a negative instruction for the pre-selected address, it means that the user does not agree with the confirmation result of the pre-selected address and the pre-selected address cannot be used as the target address. Then, a second prompt message is issued to prompt the user to manually select the target address from all or part of the addresses.

[0143] S224. In response to the user's selection operation on the second prompt information, determine the address corresponding to the selection operation as the target address.

[0144] In step S224, when the user's selection operation on the second prompt information is detected, it means that the user has completed the manual selection of the target address. The address corresponding to the selection operation is then determined as the target address. In scenarios where further selection or determination of the target address is required, the selection of the target address is achieved through the user's rejection of the pre-selected address, the second prompt information, and the selection operation.

[0145] In this embodiment, when the received user feedback information is a negative instruction for the pre-selected address, a second prompt message including selecting the target address from at least some addresses is issued. When the address selection instruction is received, the address corresponding to the address selection instruction is determined as the target address. In scenarios where further selection or determination of the target address is required, the negative instruction and selection operation realize the negation of the pre-selected address and the re-selection of the target address, ensuring the accuracy of the target address in complex scenarios.

[0146] In some embodiments, determining a target address based on address information includes: having an agent determine the target address based on address information.

[0147] When determining the target address based on address information, the navigation system can use a server-side intelligent agent to analyze image understanding information. By leveraging the agent's precise address extraction capabilities, the target address can be determined, providing a basis for the execution of the navigation process.

[0148] For example, the intelligent agent can be a navigation intelligent agent, in which case the combination of image content understanding capability and navigation capability can be achieved through the integration of a general user interface intelligent agent and a navigation intelligent agent.

[0149] In this embodiment, the intelligent agent determines the target address based on the address information, thus realizing the determination of the target address and providing a basis for the execution of the navigation process. This helps to utilize the address extraction and navigation capabilities of the intelligent agent to avoid inefficient interaction and lack of collaboration, and further improves the navigation performance of the navigation system in multiple dimensions.

[0150] In some embodiments, in response to the address information containing address information that meets a first preset condition, the agent determines the address that meets the first preset condition as the target address. In response to the address information containing two or more addresses, the agent issues a first prompt and determines the target address based on a feedback operation. In response to the address information containing two or more addresses, the agent determines a pre-selected address based on a second preset condition, and in response to receiving a user's confirmation instruction for the pre-selected address, determines the pre-selected address as the target address. In response to receiving a user's rejection instruction for the pre-selected address, the agent issues a second prompt and determines the address corresponding to the selection operation as the target address.

[0151] It should be noted that the agent used for image understanding of the interface images and the agent used for determining the target address based on the address information can be the same or different. If they are different agents, the agent used for image understanding of the interface images and the agent used for determining the target address based on the address information can be a general-purpose user interface agent and a navigation agent, respectively. At least one of the agent used for image understanding of the interface images and the agent used for determining the target address based on the address information can be deployed on the application side (i.e., locally) or on the server side (i.e., in the cloud connected to the local machine).

[0152] In some embodiments, target navigation instructions include voice instructions.

[0153] In this embodiment, voice commands are used as target navigation commands. The characteristic of voice commands requiring no manual operation ensures the convenience and safety of inputting target navigation commands in driving scenarios. The native voice reception and recognition capabilities of the navigation system can be used to reduce the development cost of the navigation system and further improve the navigation performance of the navigation system in multiple dimensions.

[0154] In one exemplary embodiment, a navigation control method is provided, applied to a server, with reference to... Figure 13 As shown, the navigation control method includes: S300: In response to receiving a target navigation command sent by the application, acquire a target image, the target image including an image of the target interface that matches the target navigation command.

[0155] In step S300, when the server receives the target navigation instruction sent by the application, the server obtains the target image, i.e., the image of the target interface that matches the target navigation instruction, as the basis for subsequently determining the target address, thereby ensuring that the target address can match the navigation requirements for the target interface. The target interface is, for example, an interface in the target application that matches the target navigation instruction and contains address-related information that matches the navigation requirements indicated by the target navigation instruction.

[0156] S400: Based on the image content of the target image, determine the target address and send the target address to the application.

[0157] In step S400, since the target image is an image of the target interface that matches the target navigation command, its image content can reflect the navigation requirements corresponding to the target navigation command. The target address that matches the target navigation command can be determined based on the image content of the target image, and the target address is sent to the application terminal, so that the application terminal can perform a navigation process including positioning, planning and guidance on the target address, thereby meeting the user's navigation requirements for the target interface and realizing intelligent navigation services.

[0158] In this embodiment, upon receiving a target navigation instruction from the application, the target image is acquired, and the target address is determined based on the image content. The target address is then sent to the application so that the application can execute the corresponding navigation process. Through the collaboration between the application and the server, the integration of image content understanding and navigation capabilities is achieved, thereby fulfilling complex or scenario-based navigation needs. The operation is simple, the interaction is efficient, and the user's operation steps are greatly reduced, which helps to improve the user's navigation experience.

[0159] In one exemplary embodiment, a navigation control device is provided, applied to a navigation system, with reference to... Figure 14 As shown, the navigation control device includes an acquisition module 10 and a navigation module 20. The acquisition module 10 is used to acquire a target image in response to receiving a target navigation command. The target image includes an image of a target interface that matches the target navigation command. The navigation module 20 is used to determine the target address based on the image content of the target image and perform a navigation process on the target address.

[0160] In this embodiment, when a target navigation command is received, the acquisition module 10 acquires the target image, and the navigation module 20 determines the target address based on the image content. Then, a navigation process is executed at the target address. This enables the execution of a navigation process that matches the navigation requirements corresponding to the target navigation command, providing users with intelligent navigation services. Determining the target address based on the image content and executing the navigation process integrates image content understanding and navigation capabilities, enhancing the interactive efficiency and intelligence of the navigation process, as well as the linkage between navigation functions and specific applications. This ensures that the target address matches the user's complex or scenario-based navigation needs, thus improving the user's navigation experience.

[0161] In one embodiment, the acquisition module 10 is further configured to: determine semantic understanding information based on the target navigation instruction, the semantic understanding information including the instruction content of the target navigation instruction; determine the target application matching the target navigation instruction based on the semantic understanding information; and determine the target image based on the target application and the semantic understanding information.

[0162] In one embodiment, the acquisition module 10 is further configured to: run the target application and acquire the target image based on semantic understanding information.

[0163] In one embodiment, the acquisition module 10 is further configured to: perform at least one application interface interaction operation corresponding to the semantic understanding information for the target application; determine the target interface based on the application interface interaction operation, and use the image of the target interface as the target image. In one embodiment, the acquisition module 10 is further configured to: acquire the interface image switched to during the execution of the application interface interaction operation, perform image understanding on the interface image to obtain image understanding information of the interface image; based on the image understanding information of the interface image, in response to the switched application interface being the target interface, determine the switched application interface as the target interface, and the image understanding information includes the image content of the interface image.

[0164] In one embodiment, the acquisition module 10 is further configured to: perform image understanding on the interface image using an intelligent agent to obtain image understanding information of the interface image.

[0165] In one embodiment, the navigation control device further includes a display module, which is used to display the application interface corresponding to the application interface interaction operation.

[0166] In one embodiment, the navigation module 20 is further configured to: determine image understanding information of the target image; in response to the image understanding information of the target image including address information, determine the target address based on the address information, and perform a navigation process on the target address.

[0167] In one embodiment, the navigation module 20 is further configured to: determine the address that meets the first preset condition as the target address in response to the address information containing an address that meets the first preset condition.

[0168] In one embodiment, the navigation module 20 is further configured to: issue a first prompt message in response to the address information containing two or more addresses, the first prompt message including prompting the user to provide feedback on the target address; and determine the target address based on the feedback operation.

[0169] In one embodiment, the navigation module 20 is further configured to: in response to the address information containing two or more addresses, determine a pre-selected address based on a second preset condition, wherein the first prompt information includes prompting the user to confirm whether the pre-selected address is the target address; and in response to receiving the user's instruction to confirm the pre-selected address, determine the pre-selected address as the target address.

[0170] In one embodiment, the navigation module 20 is further configured to: in response to receiving a user's negative instruction for a pre-selected address, issue a second prompt message, the second prompt message including prompting the user to select a target address from at least some addresses; and in response to the user's selection operation on the second prompt message, determine the address corresponding to the selection operation as the target address.

[0171] In one embodiment, the navigation module 20 is further configured to: determine the target address based on address information by the intelligent agent.

[0172] In one embodiment, the navigation module 20 is further configured to: issue a third prompt message based on the image content understanding information of the target image in response to the fact that the image understanding information of the target image does not contain address information.

[0173] In one exemplary embodiment, an electronic device is provided. The electronic device may be a terminal device such as a mobile phone, tablet computer, or wristband. The electronic device may also be an in-vehicle system. The electronic device is capable of executing the navigation control method shown in the above embodiments.

[0174] refer to Figure 15 As shown, the electronic device 400 may include one or more of the following components: processing component 402, memory 404, power supply component 406, multimedia component 408, audio component 410, input / output (I / O) interface 412, sensor component 414, and communication component 416.

[0175] Processing component 402 typically controls the overall operation of electronic device 400, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 402 may include one or more processors 420 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 402 may include one or more modules to facilitate interaction between processing component 402 and other components. For example, processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402.

[0176] Memory 404 is configured to store various types of data to support the operation of electronic device 400. Examples of such data include instructions for any application or method operating on electronic device 400, contact data, phonebook data, messages, pictures, videos, etc. Memory 404 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0177] Power supply component 406 provides power to various components of electronic device 400. Power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 400.

[0178] Multimedia component 408 includes a screen that provides an output interface between electronic device 400 and user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 408 includes a front-facing camera module and / or a rear-facing camera module. When electronic device 400 is in an operating mode, such as shooting mode or video mode, the front-facing camera module and / or rear-facing camera module may receive external multimedia data. Each front-facing camera module and rear-facing camera module may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0179] Audio component 410 is configured to output and / or input audio signals. For example, audio component 410 includes a microphone (MIC) configured to receive external audio signals when electronic device 400 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or transmitted via communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.

[0180] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0181] Sensor assembly 414 includes one or more sensors for providing state assessments of various aspects of electronic device 400. For example, sensor assembly 414 may receive the on / off state of electronic device 400, the relative positioning of components such as the display and keypad of electronic device 400, changes in position of electronic device 400 or a component of electronic device 400, the presence or absence of user contact with electronic device 400, orientation or acceleration / deceleration of electronic device 400, and temperature changes of electronic device 400. Sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 414 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0182] Communication component 416 is configured to facilitate wired or wireless communication between electronic device 400 and other terminals. Electronic device 400 can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G, 5G, or combinations thereof. In one exemplary embodiment, communication component 416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 416 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0183] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing terminals (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods shown in the above embodiments or combinations thereof.

[0184] In one exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, which can be executed by a processor 420 of an electronic device 400 to perform the methods shown in the above embodiments or combinations thereof. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage terminal, etc. When the instructions in the storage medium are executed by the processor of the terminal, the terminal is able to perform the methods shown in the above embodiments or combinations thereof.

[0185] In one exemplary embodiment, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the method described above.

[0186] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0187] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0188] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0189] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0190] The examples in this document may involve user data, data acquisition, and / or use. All of these aspects comply with relevant laws, regulations, and rules. In the examples, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, when implementing each example, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained through appropriate means, in accordance with relevant laws and regulations. The specific methods of notification and / or authorization can vary depending on the actual situation and application scenario; the scope of the solution is not limited in this regard.

[0191] In this manual and the sample solutions, any processing of personal information will be conducted only under legal grounds (such as obtaining the consent of the data subject or being necessary for the performance of a contract) and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0192] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A navigation control method, characterized in that, The navigation control method includes: In response to receiving a target navigation instruction, a target image is acquired, the target image including an image of a target interface that matches the target navigation instruction; Based on the image content of the target image, the target address is determined, and a navigation process is performed on the target address.

2. The navigation control method according to claim 1, characterized in that, The step of acquiring a target image in response to receiving a target navigation command includes: Based on the target navigation instruction, semantic understanding information is determined, and the semantic understanding information includes the instruction content of the target navigation instruction; Based on the semantic understanding information, a target application matching the target navigation command is determined; The target image is determined based on the target application and the semantic understanding information.

3. The navigation control method according to claim 2, characterized in that, Determining the target image based on the target application and the semantic understanding information includes: Run the target application and acquire the target image based on the semantic understanding information.

4. The navigation control method according to claim 2, characterized in that, The step of obtaining the target image based on the semantic understanding information includes: Perform at least one application interface interaction operation corresponding to the semantic understanding information for the target application; Based on the application interface interaction, the target interface is determined, and the image of the target interface is used as the target image.

5. The navigation control method according to claim 4, characterized in that, Determining the target interface based on the application interface interaction includes: Obtain the interface image switched to when the application interface interaction operation is performed, perform image understanding on the interface image, and obtain the image understanding information of the interface image; Based on the image understanding information of the interface image, in response to the result that the switched application interface is the target interface, the switched application interface is determined as the target interface, and the image understanding information includes the image content of the interface image.

6. The navigation control method according to claim 5, characterized in that, The step of performing image understanding on the interface image to obtain image understanding information of the interface image includes: An intelligent agent performs image understanding on the interface image to obtain image understanding information of the interface image.

7. The navigation control method according to claim 4, characterized in that, The navigation control method further includes: The application interface is displayed and the application interface corresponding to the interactive operation of the application interface is executed.

8. The navigation control method according to any one of claims 1 to 7, characterized in that, The process of determining the target address based on the image content of the target image and performing a navigation process on the target address includes: Determine image understanding information for the target image; In response to the image understanding information of the target image including address information, the target address is determined based on the address information, and a navigation process is performed on the target address.

9. The navigation control method according to claim 8, characterized in that, Determining the target address based on the address information includes: In response to the address information containing an address that meets a first preset condition, the address that meets the first preset condition is determined as the target address.

10. The navigation control method according to claim 8, characterized in that, Determining the target address based on the address information includes: In response to the address information containing two or more addresses, a first prompt message is issued, the first prompt message including prompting the user to provide feedback on the target address; Based on the feedback operation, the target address is determined.

11. The navigation control method according to claim 10, characterized in that, Determining the target address based on the address information further includes: In response to the address information containing two or more addresses, a pre-selected address is determined based on a second preset condition, and the first prompt information includes prompting the user to confirm whether the pre-selected address is the target address; in response to receiving the user's instruction to confirm the pre-selected address, the pre-selected address is determined as the target address.

12. The navigation control method according to claim 11, characterized in that, Determining the target address based on the address information further includes: In response to receiving a user's negative instruction for the pre-selected address, a second prompt message is issued, the second prompt message including prompting the user to select a target address from at least some of the addresses; In response to the user's selection operation on the second prompt information, the address corresponding to the selection operation is determined as the target address.

13. The navigation control method according to claim 8, characterized in that, Determining the target address based on the address information includes: The intelligent agent determines the target address based on the address information.

14. The navigation control method according to claim 8, characterized in that, The navigation control method further includes: In response to the fact that the image understanding information of the target image does not contain the address information, a third prompt message is issued based on the image content understanding information of the target image.

15. A navigation control method, characterized in that, The navigation control method includes: In response to receiving a target navigation instruction sent by the application, a target image is acquired, the target image including an image of a target interface that matches the target navigation instruction; Based on the image content of the target image, the target address is determined and sent to the application.

16. A navigation control device, characterized in that, The navigation control device includes: The acquisition module is configured to acquire a target image in response to receiving a target navigation instruction, the target image including an image of a target interface that matches the target navigation instruction; A navigation module is used to determine a target address based on the image content of the target image and to perform a navigation process on the target address.

17. The navigation control device according to claim 16, characterized in that, The acquisition module is also used for: Based on the target navigation instruction, semantic understanding information is determined, and the semantic understanding information includes the instruction content of the target navigation instruction; Based on the semantic understanding information, a target application matching the target navigation command is determined; The target image is determined based on the target application and the semantic understanding information.

18. The navigation control device according to claim 17, characterized in that, The acquisition module is also used for: Perform at least one application interface interaction operation corresponding to the semantic understanding information for the target application; Based on the application interface interaction, the target interface is determined, and the image of the target interface is used as the target image.

19. The navigation control device according to claim 18, characterized in that, The acquisition module is also used for: Obtain the interface image switched to when the application interface interaction operation is performed, perform image understanding on the interface image, and obtain the image understanding information of the interface image; Based on the image understanding information of the interface image, in response to the result that the switched application interface is the target interface, the switched application interface is determined as the target interface, and the image understanding information includes the image content of the interface image.

20. The navigation control device according to claim 19, characterized in that, The acquisition module is also used for: An intelligent agent performs image understanding on the interface image to obtain image understanding information of the interface image.

21. The navigation control device according to claim 18, characterized in that, The navigation control device also includes: The display module is used to display the application interface corresponding to the application interface interaction operation.

22. The navigation control device according to any one of claims 16 to 21, characterized in that, The navigation module is also used for: Determine image understanding information for the target image; In response to the image understanding information of the target image including address information, the target address is determined based on the address information, and a navigation process is performed on the target address.

23. The navigation control device according to claim 22, characterized in that, The navigation module is also used for: In response to the address information containing an address that meets a first preset condition, the address that meets the first preset condition is determined as the target address.

24. The navigation control device according to claim 22, characterized in that, The navigation module is also used for: In response to the address information containing two or more addresses, a first prompt message is issued, the first prompt message including prompting the user to provide feedback on the target address; Based on the feedback operation, the target address is determined.

25. The navigation control device according to claim 24, characterized in that, The navigation module is also used for: In response to the fact that the address information contains two or more addresses, a pre-selected address is determined based on a second preset condition, and the first prompt information includes prompting the user to confirm whether the pre-selected address is the target address; In response to receiving a user's instruction to confirm the pre-selected address, the pre-selected address is determined as the target address.

26. The navigation control device according to claim 25, characterized in that, The navigation module is also used for: In response to receiving a user's negative instruction for the pre-selected address, a second prompt message is issued, the second prompt message including prompting the user to select a target address from at least some of the addresses; In response to the user's selection operation on the second prompt information, the address corresponding to the selection operation is determined as the target address.

27. The navigation control device according to claim 22, characterized in that, The navigation module is also used for: The intelligent agent determines the target address based on the address information.

28. The navigation control method according to claim 22, characterized in that, The navigation module is also used for: In response to the fact that the image understanding information of the target image does not contain the address information, a third prompt message is issued based on the image content understanding information of the target image.

29. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store processor-executable instructions; The processor is configured to perform the navigation control method as described in any one of claims 1 to 14 or claim 15.

30. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the navigation control method as described in any one of claims 1 to 14 or claim 15.

31. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the navigation control method as described in any one of claims 1 to 14 or claim 15.

Images