Information interaction method and wearable device

Abstract

The invention provides an information interaction method and wearable equipment, and relates to the technical field of computers. The method is executed by a wearable device and comprises the steps that a first image is collected, and the first image comprises a first article seen by a user; displaying information of a second article, wherein the second article is an article matched with the first article in a merchant system; in response to a first instruction of the user, order information for the second article and information of a first resource transfer tool are displayed, and the first instruction indicates to order the second article; and in response to a second instruction of the user, a resource transfer result of the order is displayed, and the second instruction indicates resource transfer of the order. Through the above method, the whole process of real-time shopping and payment based on the wearable device is realized, and the service experience of the user is improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an information interaction method and a wearable device. Background Technology

[0002] Currently, in scenarios like parties and shopping, when we find an item we like, we often manually search for it on a mobile shopping platform by entering the item's key information, then go through a cumbersome interface to place an order and pay. This method requires a lot of manual operation from the user, resulting in a poor user experience.

[0003] With the development of artificial intelligence technology, wearable devices such as smart glasses are being used more and more widely. Currently, smart glasses are mainly used for offline payments. Summary of the Invention

[0004] This disclosure provides an information interaction method and a wearable device.

[0005] According to a first aspect of this disclosure, an information interaction method is proposed, executed by a wearable device, comprising: acquiring a first image, the first image including a first item seen by a user; displaying information about a second item, the second item being an item matched with the first item in a merchant system; in response to a first instruction from the user, displaying order information for the second item and information about a first resource transfer tool, the first instruction indicating that an order be placed for the second item; and in response to a second instruction from the user, displaying the resource transfer result of the order, wherein the second instruction indicating that a resource transfer be performed on the order.

[0006] In some embodiments, displaying the resource transfer result of the order in response to the user's second instruction includes: after receiving the user's second instruction, displaying and / or playing verification guidance information via voice, the verification guidance information including an instruction for the user to perform a specified operation; and displaying the resource transfer result of the order in response to the user performing the specified operation.

[0007] In some embodiments, the specified operation is reading aloud specified information. The step of displaying the resource transfer result of the order in response to the user performing the specified operation includes: during the period when the user reads the specified information, collecting the user's first voice information and transmitting the first voice information to the intelligent agent so that the intelligent agent can perform voiceprint verification on the user based on the first voice information, wherein the specified information is dynamically determined by the intelligent agent; and displaying the resource transfer result of the order if the voiceprint verification is successful.

[0008] In some embodiments, the step of collecting the user's first voice information and transmitting the first voice information to an intelligent agent so that the intelligent agent can perform voiceprint verification on the user based on the first voice information includes: during the collection of the first voice information through the wearable device, triggering a mobile terminal to collect the user's third voice information; and sending the first voice information and the third voice information to the intelligent agent through the mobile terminal so that the intelligent agent can perform voiceprint verification on the user based on the first voice information and the third voice information.

[0009] In some embodiments, the display and / or voice playback verification guidance information is executed when the user's identity verification fails based on at least one risk control decision factor, wherein the at least one risk control decision factor includes the user's previously collected voiceprint information; or, the display and / or voice playback verification guidance information is executed when the agent determines that voiceprint verification cannot be performed based on the user's previously collected second voice information, or when voiceprint verification based on the second voice information fails.

[0010] In some embodiments, displaying the resource transfer result of the order in response to the user's second instruction further includes: displaying the resource transfer result of the order upon receiving the user's second instruction and upon successful authentication of the user based on the at least one risk control decision factor; or, displaying the resource transfer result of the order upon receiving the user's second instruction and upon successful voiceprint verification by the intelligent agent based on the previously collected second voice information of the user.

[0011] In some embodiments, displaying the resource transfer result of the order in response to the user's second instruction includes: sending a resource transfer notification message to an intelligent agent in response to the user's second instruction, wherein the intelligent agent, upon receiving the notification message, verifies the user based on a target verification method, and sends the resource transfer result of the order to the wearable device after successful verification; displaying the resource transfer result of the order, wherein the target verification method includes one or more of voice verification, voiceprint verification, face verification, fingerprint verification, iris verification, gesture verification, SMS verification code verification, and password verification.

[0012] In some embodiments, displaying the resource transfer result of the order in response to a second instruction from the user includes: displaying information about a second resource transfer tool in response to a third instruction from the user, the third instruction indicating a switch to a resource transfer tool; and displaying the resource transfer result of the order in response to the second instruction.

[0013] In some embodiments, the information interaction method further includes: displaying and / or voice-playing operation guidance information to assist the user in performing the next operation while displaying at least one of information about the second item, order information for the second item, and information about the first resource transfer tool.

[0014] In some embodiments, displaying information about the second item includes: displaying a second image, the second image containing a plurality of items in the first image and numbers of the plurality of items; and displaying information about the second item in response to a fourth instruction from the user, wherein the fourth instruction indicates the number information of the first item selected by the user from the plurality of items.

[0015] In some embodiments, acquiring the first image includes: displaying a shooting page in response to a first operation by the user, the first operation being used to invoke a target application; and acquiring the first image based on the target application in response to a second operation by the user, the second operation being used to trigger the acquisition.

[0016] In some embodiments, the wearable device is smart glasses, and displaying information about the second item includes: displaying first information about the second item on a first screen of the smart glasses, and displaying second information about the second item on a second screen of the smart glasses, wherein the first information is different from the second information; or displaying an information display page for the second item, wherein the display page includes a background and an image of the second item, and the image of the second item presents a three-dimensional effect.

[0017] In some embodiments, the first resource transfer tool is determined by an agent from at least one candidate resource transfer tool upon instruction or triggering by the wearable device, based on at least one of the user's session context, detection information of the current environment, and user profile features.

[0018] According to a second aspect of this disclosure, a wearable device is provided, comprising: a module for performing the information interaction method as described above.

[0019] According to a third aspect of this disclosure, a wearable device is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to perform the information interaction method as described above based on instructions stored in the memory.

[0020] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the information interaction method as described above.

[0021] According to a fifth aspect of this disclosure, a computer program product is provided having computer program instructions stored thereon, which, when executed by a processor, implement the information interaction method as described above.

[0022] Other features and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0023] The accompanying drawings, which form part of this specification, illustrate embodiments of this disclosure and, together with the specification, serve to explain the principles of this disclosure.

[0024] Figure 1 This is a flowchart illustrating an information interaction method according to some embodiments of the present disclosure;

[0025] Figure 2 This is a flowchart illustrating an information interaction method according to other embodiments of this disclosure;

[0026] Figure 3 This is a flowchart illustrating an information interaction method according to some embodiments of the present disclosure;

[0027] Figure 4 This is a schematic diagram of an interactive interface presented in an information interaction method according to some embodiments of the present disclosure;

[0028] Figure 5 This is a schematic diagram of the structure of smart glasses according to some embodiments of the present disclosure;

[0029] Figure 6 This is a schematic diagram of the structure of a wearable device according to other embodiments of this disclosure;

[0030] Figure 7 This is a schematic diagram of the structure of an information interaction system according to some embodiments of the present disclosure.

[0031] This disclosure can be more clearly understood with reference to the accompanying drawings and the following detailed description. Detailed Implementation

[0032] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0033] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0034] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0035] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0036] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0037] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0038] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0039] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0040] To address the shortcomings of existing solutions in related technologies, this disclosure provides an information interaction method and a wearable device to realize the entire process of instant shopping and payment based on the wearable device, and improves the user's service experience from at least one aspect of the intelligence, convenience, and security of the instant shopping and payment process.

[0041] Figure 1 This is a flowchart illustrating an information interaction method according to some embodiments of this disclosure. For example... Figure 1 As shown, the information interaction method can be executed by a wearable device, and the method includes steps 11 to 14.

[0042] In step 11, a first image is acquired. This first image includes the first item seen by the user.

[0043] In some embodiments, the wearable device captures a first image upon user triggering. For example, it captures the first image in response to a user issuing a voice command to "take a picture." Alternatively, it captures the first image in response to a user's gaze duration being greater than or equal to a set duration. Furthermore, other user-triggered methods can be used in specific implementations to trigger the capture of the first image. Or, the wearable device may actively capture the first image.

[0044] In some embodiments, the wearable device may acquire a first image in the following exemplary manner: determining the user's gaze point or visual focus; taking an image of the scene the user is currently gazing at, centered on the user's gaze point or visual focus, and using this image as the first image. Furthermore, in specific implementations, the wearable device may also acquire the first image in other ways.

[0045] In some embodiments, after acquiring a first image, the wearable device sends the first image to the server of a first application so that the server can perform processing such as image recognition and product search based on the recognition results. For example, after acquiring the first image, the wearable device sends the first image to the server of the first application via a mobile terminal. Alternatively, the wearable device preprocesses the first image and then sends the preprocessed image to the server of the first application. Or, after acquiring the first image and receiving a user's instruction to search for the same product, the wearable device sends the first image to the server of the first application. The server of the first application may have a large model deployed on it or be able to call upon a large model to achieve accurate image recognition and related product search using the large model. For example, the first application may be a shopping application or other applications.

[0046] In step 12, information about the second item is displayed. The second item is the object in the merchant system that matches the first item.

[0047] Step 12 can be implemented in several ways. The following describes three examples of implementation methods.

[0048] In the first embodiment, after capturing the first image, the wearable device can display information about the second item without any further interaction with the user.

[0049] In the second implementation, after acquiring the first image, the user needs to provide an instruction to perform an image search, and the wearable device then displays information about the second item. The instruction to perform the image search can be a voice command, a user clicking or touching a designated interactive control on the wearable device, or a specific gesture from the user.

[0050] In a third implementation, the first image includes multiple items. In this implementation, after acquiring the first image, the user needs to first identify the first item of interest from the multiple items through interaction with the user, and then the wearable device displays information about a second item from the merchant's system that matches the first item.

[0051] For example, in the third embodiment, step 12 may specifically include steps 121 and 122.

[0052] Step 121: After acquiring the first image, display the second image.

[0053] The second image contains multiple items from the first image, along with their corresponding numbers. These numbers can be numbers, letters, or text. For example, if the first image contains glasses, a shirt, and a scarf, the second image can be generated through processing. The second image not only contains images of the glasses, shirt, and scarf, but also their numbers, such as "1", "2", and "3" respectively.

[0054] Step 122: In response to the user's fourth instruction, display information about the second item.

[0055] The fourth instruction is used to instruct the user to select the number of the first item from a set of items. For example, the fourth instruction can be a voice command, a click or touch of a designated interactive control on the wearable device, or a specific gesture. For instance, when the displayed second image includes glasses, a shirt, and a scarf, and the numbers of these three items are "1", "2", and "3" respectively, the user can say "select item 3" to the wearable device or show a gesture indicating "select item 3". Next, the wearable device can send the user's instruction to select item 3 (i.e., the scarf) to the server of the first application. Then, the server searches for a matching product (i.e., the second item) from the merchant system and sends the information of the matching product to the wearable device, so that the wearable device can display the information of the matching product to the user.

[0056] In this embodiment of the disclosure, when the first image contains multiple items, by showing the user a second image with item number information, the user can more quickly inform the wearable device of the items they are truly interested in, thereby improving the user's service experience.

[0057] In some embodiments, the wearable device is smart glasses. The smart glasses can display information about a second item in an exemplary manner: displaying first information about the second item on a first screen of the smart glasses, and displaying second information about the second item on a second screen of the smart glasses. The first information and the second information are different. For example, a magnified view of the second item is displayed on one screen of the smart glasses (e.g., the left eye screen); detailed information about the second item is displayed on the other screen of the smart glasses (e.g., the right eye screen). Alternatively, different products can be displayed on the two screens of the smart glasses. In this embodiment of the disclosure, by displaying different information on the two screens of the smart glasses, the amount of content that the smart glasses can display at one time is increased, which helps to reduce the amount of operation required for users to switch between displayed content, and further improves the convenience for users to place orders and make payments through the wearable device.

[0058] Alternatively, smart glasses can also display information about a second item in the following exemplary manner: displaying an information display page for the second item, which includes a background and an image of the second item, and the image of the second item is presented in a three-dimensional effect. By presenting an image of an item with a three-dimensional effect in smart glasses, it helps to improve the user's visual experience during shopping and payment using wearable devices.

[0059] In step 13, in response to the user's first instruction, order information for the second item and information for the first resource transfer tool are displayed.

[0060] The first instruction directs the user to place an order for the second item. For example, the first instruction could be a user's voice command, a user's click or touch of a designated interactive control on the wearable device, or a specific gesture. For instance, the first instruction could be natural language such as the user saying "Please place an order" or "Buy it." The order information for the second item includes, but is not limited to, the item's product specifications, delivery address, shipping information, and total amount.

[0061] The primary resource transfer instrument can be understood as a payment instrument, payment channel, or payment method. For example, the primary resource transfer instrument can be a bank-related payment instrument, such as account 1 and account 2 of bank A, or account 1 of bank B. The primary resource transfer instrument can also be a non-bank payment instrument, such as payment channel 1 and payment channel 2 of company C, or payment channel 2 of company D, etc.

[0062] In some embodiments, the wearable device obtains order information for the second item and information about the first resource transfer tool through the following communication path: After receiving a first instruction, the device sends the first instruction to the server of the first application to obtain the order information for the second item from the server. Furthermore, the wearable device also sends a resource transfer request for the order to an intelligent agent through the first application to obtain information about the first resource transfer tool from the intelligent agent. This intelligent agent can be an intelligent agent deployed in the second application or an intelligent agent that the second application can invoke. The intelligent agent can determine the first resource transfer tool from at least one candidate resource transfer tool. In specific implementations, the first application and the second application can be different applications or platforms. For example, the first application is a shopping application, and the second application is a payment application.

[0063] Furthermore, in practical implementation, the wearable device can also obtain order information for the second item and information about the first resource transfer tool through other communication paths. For example, the wearable device obtains order information for the second item and information about the first resource transfer tool through the following communication path: During the user's access to a third application in the wearable device (e.g., a general-purpose smart application), the wearable device receives a first instruction through the third application and sends the first instruction to the server of the first application to obtain order information for the second item from the server. Additionally, the wearable device also sends a resource transfer request for the order to a smart agent through the third application to obtain information about the first resource transfer tool from that smart agent. This smart agent can be a smart agent deployed in the second application or a smart agent that the second application can invoke.

[0064] In step 14, in response to the user's second instruction, the resource transfer result of the order is displayed.

[0065] The second instruction directs the transfer of resources for the order. This second instruction can be a user's voice command, a user's click or touch of a designated interactive control on the wearable device, or a specific gesture. For example, the second instruction could be natural language such as the user saying "go to payment," "go to make payment," or "confirm payment."

[0066] Step 14 can be implemented in several ways. The following describes step 14 by way of two examples.

[0067] In the first implementation, upon receiving the user's second instruction, the resource transfer result of the order is displayed without further interaction with the user. This resource transfer result includes, but is not limited to, a payment completion notification, the payment account information used, and information about the purchased second item.

[0068] For example, in one embodiment, after receiving a second instruction from the user, the wearable device can directly transmit the second instruction to the server of a second application, so that the server can execute a resource transfer for the order. Then, the wearable device obtains the resource transfer result from the server of the second application.

[0069] For example, in the first embodiment, after receiving the user's second instruction, the wearable device sends a resource transfer notification message to the intelligent agent. Upon receiving this notification message, the intelligent agent determines the target verification method from at least one verification method and then verifies the user based on the target verification method. After successful verification, the resource transfer for the order is executed through the server of the second application. Then, the wearable device obtains the resource transfer result from the server of the second application. By employing a user-insensitive authentication method, the frequency of interaction between the wearable device and the user can be minimized while ensuring a good user experience, thus reducing the energy consumption of the wearable device.

[0070] In the second implementation, after receiving the user's second instruction, a verification prompt is displayed and / or played via voice to the user, and the order's resource transfer result is displayed after successful verification through the target verification method. The target verification method can be a biometric verification method, an SMS verification method, or other verification methods. Biometric information includes, but is not limited to, at least one of the user's fingerprint, face, iris, and voiceprint information. The verification prompt can be "Verifying for you," "Voiceprint verification for you," or "Face verification for you," etc. By employing the above-mentioned user-aware method for identity verification, users can be promptly informed of the operation currently being performed by the device, which helps improve the user's service experience when shopping and paying using wearable devices.

[0071] In some embodiments, the information interaction method further includes: displaying information about a second resource transfer tool in response to a third instruction from the user. The third instruction is used to instruct switching resource transfer tools. The third instruction can be a user's voice command, a user's click or touch of a designated interactive control on a wearable device, or a specific gesture from the user. For example, the third instruction could be natural language such as the user saying "Change payment method." The second resource transfer tool can be determined by the intelligent agent based on at least one of the user's conversation context, current environment detection information, and user profile features.

[0072] For example, the wearable device first displays a first resource transfer tool to the user, such as "Account 1 of Bank A". When the user says "Change payment method", it then displays a second resource transfer tool, such as "Account 2 of Bank B". After the user issues a payment confirmation instruction, the wearable device displays the resource transfer result of the order.

[0073] For example, the wearable device first displays a second resource transfer tool to the user, such as "Payment Channel 1 for Company C". When the user says "Change payment method", the first resource transfer tool is then displayed to the user, such as "Payment Channel 2 for Company D". After the user issues a payment confirmation instruction, the wearable device then displays the resource transfer result of the order.

[0074] In this embodiment of the disclosure, by supporting dynamic switching of resource transfer tools, the intelligence, convenience and flexibility of users shopping and paying through wearable devices are further improved.

[0075] In this embodiment of the disclosure, by Figure 1 The process shown realizes the entire process of shopping and payment based on wearable devices, which meets the user's need to easily and quickly search and purchase the same products without the need for users to perform cumbersome manual search, application switching and other operations, thus improving the user service experience.

[0076] Figure 2 This is a flowchart illustrating an information interaction method according to other embodiments of this disclosure. For example... Figure 2 As shown, the information interaction method can be executed by a wearable device, and the method includes steps 21 to 26.

[0077] In step 21, in response to the user's first action, the shooting page is displayed.

[0078] The first operation is used to activate the target application. For example, the first operation can be a user's voice command, such as "Open this application for me." Alternatively, the first operation can be the user opening the target application by swiping, clicking, or touching interactive controls on the wearable device. After activating the target application, the camera page is displayed. The target application can be a shopping app, a general smart app, or another type of app.

[0079] In step 22, in response to the user's second operation, the first image is acquired.

[0080] The first image includes the first item seen by the user. In specific implementations, the first image collected by the target application may include only the first item, or it may include the first item and other items.

[0081] The second operation triggers the data acquisition. For example, the second operation could be a user's voice command, such as capturing the first image after the user says "take a picture." Alternatively, the second operation could be a prolonged gaze action by the user, such as capturing the first image after the user's gaze lasts longer than two seconds. Upon detecting the user's second operation, the first image is acquired based on the target application. For example, the target application might invoke the image acquisition module (e.g., camera) on the wearable device to capture a picture of the scene within the user's field of view.

[0082] For details on how to perform step 22, please refer to the relevant descriptions in the embodiments described above.

[0083] In step 23, information about the second item is displayed. The second item is the one that matches the first item in the merchant system.

[0084] In practice, the wearable device may display multiple products matching the first item to the user. When the user sees the product they want to buy, they can trigger step 24 by outputting the first command.

[0085] For details on how to perform step 23, please refer to the relevant descriptions in the embodiments described above.

[0086] In step 24, in response to the user's first instruction, order information for the second item and information about the first resource transfer tool are displayed. The first instruction is used to instruct the user to place an order for the second item. For details on how step 24 is executed, please refer to the relevant descriptions in the previously described embodiments.

[0087] In step 25, after receiving the user's second instruction, verification guidance information is displayed and / or played via voice. The second instruction is used to instruct resource transfer for the order.

[0088] In some examples, the verification guidance information includes instructions for the user to perform a specified action. This specified action could be reading a specified message aloud, making a specified gesture, or other actions.

[0089] In step 26, in response to the user performing the specified operation, the resource transfer result of the order is displayed.

[0090] Step 26 can be implemented in various ways. Two examples are described below.

[0091] In the first implementation, if the wearable device detects that the user has performed a specified operation, it confirms that the user has been authenticated. Next, the wearable device can perform a resource transfer for that order through the resource transfer server and display the result to the user.

[0092] In the second implementation, after detecting that the user has performed a specified operation, the wearable device further verifies the information through an intelligent agent to confirm whether the user has passed authentication. Only after confirming the user's authentication is successful is the order's resource transfer result displayed. This dual authentication method, involving both the user's specified operation and further verification, improves the accuracy and reliability of user authentication results, further enhancing the security of shopping and payments made through wearable devices.

[0093] The second implementation method will be described in detail below with two examples.

[0094] In the first example, the specified operation is reading aloud specified information. In this example, step 26 includes: while the user reads the specified information, the wearable device collects the user's first voice information and transmits it to the intelligent agent, so that the intelligent agent can perform voiceprint verification on the user based on the first voice information; if the voiceprint verification is successful, the resource transfer result of the order is displayed. The specified information can be dynamically determined by the intelligent agent or it can be fixed information. In this way, by using voice interaction and voiceprint verification to authenticate the user, not only is the security of the user's shopping and payment process based on the wearable device improved, but the convenience of the verification process is also enhanced.

[0095] In the first example, the wearable device can collect only the user's voice output while the user reads a specified message.

[0096] In the first example, the wearable device and mobile terminal can jointly collect the user's voice information while the user is reading the specified information. For example, during the user's reading, in addition to the wearable device collecting the first voice information, the mobile terminal is also triggered to collect the user's third voice information. Furthermore, the mobile terminal sends the first and third voice information to the intelligent agent, so that the intelligent agent can perform voiceprint verification on the user based on the first and third voice information. Here, the first and third voice information represent the voice information collected by different devices during the user's reading of the specified information.

[0097] In this embodiment of the disclosure, by having wearable devices and mobile terminals jointly collect users' voice information, it helps to enhance the accuracy of subsequent voiceprint verification, thereby improving the security of users shopping and making payments based on wearable devices.

[0098] In the second example, the specified operation is to perform a specific facial movement, such as blinking or opening the mouth. In this example, after the user performs the specified facial movement, the wearable device uses an intelligent agent to perform facial verification, liveness detection, and other checks on the user. After successful verification, the wearable device displays the resource transfer result for the order.

[0099] In some embodiments, the information interaction method further includes: before executing steps 25 and 26, determining, by the intelligent agent, that voiceprint verification cannot be performed based on the previously collected second voice information of the user, or that voiceprint verification based on the second voice information has failed. The second voice information may be voice information collected by the wearable device during a user's shopping and payment process when deciding to verify the user's identity. Alternatively, the second voice information may be voice information collected by the wearable device within a set time period when deciding to verify the user's identity. Furthermore, the information interaction method may also include: upon receiving a second instruction from the user and successfully performing voiceprint verification by the intelligent agent based on the previously collected second voice information of the user, displaying the resource transfer result of the order.

[0100] In this embodiment of the disclosure, by combining a user-invisible voiceprint verification method with a user-aware voiceprint verification method, the security and convenience of users shopping and paying based on wearable devices can be further improved, thereby enhancing the user's service experience.

[0101] In some embodiments, the information interaction method further includes: authenticating the user based on at least one risk control decision factor before performing steps 25 and 26. If user authentication based on at least one risk control decision factor fails, steps 25 and 26 are performed. If user authentication based on at least one risk control decision factor is successful, steps 25 and 26 do not need to be performed. The at least one risk control decision factor includes at least one of previously collected user voiceprint information and the user's IP address. In this embodiment, by combining a user-insensitive verification method with a user-conscious voiceprint verification method, the security and convenience of users shopping and paying using wearable devices can be further improved, thus enhancing the user's service experience.

[0102] In this embodiment of the disclosure, by Figure 2 The process shown enables a convenient, secure, and flexible shopping and payment process based on wearable devices, meeting users' needs for easy, fast, and secure searching and purchasing of the same products without requiring cumbersome manual searching or application switching, thus further improving the user's service experience.

[0103] Figure 3 This is a flowchart illustrating an information interaction method according to some embodiments of the present disclosure. For example... Figure 3 As shown, in some embodiments of this disclosure, the information interaction method can be jointly executed by a user, a wearable device, a mobile terminal, and an intelligent agent.

[0104] In some embodiments of this disclosure, the intelligent agent includes a first intelligent agent and a second intelligent agent. The first intelligent agent is mainly responsible for recommending similar items and generating orders, while the second intelligent agent is mainly responsible for recommending resource transfer tools and verifying user identity. In specific implementations, the first and second intelligent agents can both be deployed on mobile terminals, both on the cloud, or other deployment methods can be used. For example, the first intelligent agent can be deployed on a wearable device, and the second intelligent agent can be deployed on the cloud. The first and second intelligent agents can be associated with different applications. For example, the first intelligent agent is associated with a first application (e.g., a shopping application), and it can be deployed on or invoked by the first application. The second intelligent agent is associated with a second application (e.g., a payment application), and it can be deployed on or invoked by the second application.

[0105] To simplify the communication process between wearable devices, the first intelligent agent, and the second intelligent agent, in Figure 3 In this context, the first and second intelligent agents are collectively represented as "intelligent agent." For example... Figure 3 As shown, the information interaction method includes steps 301 to 318.

[0106] In step 301, the user issues a photo-taking command. The photo-taking command can be a voice message, a gesture, or another form of instruction. For example, the photo-taking command could be the user saying "take a photo." Alternatively, the photo-taking command could be the user gazing ahead for an extended period of time.

[0107] In step 302, the wearable device captures a first image. The first image includes a first object seen by the user.

[0108] In step 303, the wearable device sends the first image to the mobile terminal.

[0109] In some embodiments, after acquiring a first image, the wearable device first performs preprocessing such as cropping and compression on the first image to obtain a preprocessed first image, and then sends the preprocessed first image to a mobile terminal. For example, the mobile terminal is a device such as a mobile phone or a tablet computer. By performing the above preprocessing, the amount of data that needs to be transmitted during shopping and payment based on the wearable device can be reduced, thereby helping to improve the real-time response to users and further enhancing the user's service experience.

[0110] In step 304, the mobile terminal sends the first image to the intelligent agent. Specifically, the mobile terminal sends the first image to the first intelligent agent so that the first intelligent agent can subsequently determine the matching product.

[0111] In step 305, the agent determines a second item in the merchant system that matches the first item.

[0112] Specifically, in step 305, the first agent may determine the matching item. Considering that the first image may contain a single item or multiple items, the processing flow of the first agent differs in some embodiments when the first image contains a single item and when it contains multiple items.

[0113] For example, if the first image contains only a single item, the first agent can employ the following processing flow: upon receiving the first image, it uses a large model or other algorithms / models to identify the first item contained in the first image; based on the similarity between the first item and products in the merchant system, it determines the product to be recommended and sends the product information to the wearable device. The product to be recommended includes a second item that matches the first item.

[0114] For example, if the first image includes multiple items, the first intelligent agent may employ the following processing flow: After receiving the first image, it uses a large model or other algorithms / models to identify the multiple items contained in the first image and assigns numbers to the multiple items in the first image to obtain a second image; the first intelligent agent sends the second image to a wearable device so that the second image can be displayed to the user through the wearable device, wherein the second image contains the multiple items in the first image and the numbers of the multiple items; after receiving the user's fourth instruction through the wearable device, it searches and determines the recommended products from the merchant system according to the number of the first item selected by the user as indicated by the fourth instruction, and sends the information of the recommended products to the wearable device. The recommended products include second items that match the first items.

[0115] Furthermore, when the first image includes multiple items, after identifying the multiple items contained in the first image, the first intelligent agent can also preliminarily determine items that the user may be interested in from among the multiple items by combining other information. Then, the first intelligent agent can only number the items that the user may be interested in to obtain a second image, and send the second image to the wearable device. The other information can be one or more of the following: the user's gaze point or visual focus information, and the user's voice information. In this way, when displaying the second image, the wearable device only needs to display the items that the user may be interested in and their numbering information, instead of displaying all the items and their numbers in the first image, thereby helping to improve the intelligence of the interaction with the user.

[0116] In step 306, the intelligent agent sends information about the second item to the mobile terminal. Specifically, the first intelligent agent sends information about the item (i.e., the second item) that matches the first item in the merchant system to the mobile terminal.

[0117] In step 307, the mobile terminal sends information about the second item to the wearable device.

[0118] In step 308, the wearable device displays information about the second item.

[0119] In step 309, the user issues an order placement command. The order placement command can be voice, gesture, or other form of instruction. For example, the order placement command could be the user saying "buy it" or "place an order." Alternatively, the order placement command could be the user clicking on a designated interactive control on the wearable device.

[0120] In step 310, the wearable device transmits an order instruction to the smart agent via a mobile terminal.

[0121] In some examples, wearable devices transmit order instructions to the first and second intelligent agents via mobile terminals. Alternatively, wearable devices may also transmit order instructions directly to the first and second intelligent agents without the assistance of mobile terminals.

[0122] In other examples, the wearable device transmits an order placement instruction to the first intelligent agent via a mobile terminal, triggering the first intelligent agent to confirm the order information; then the first intelligent agent transmits the instruction to the second intelligent agent, triggering the second intelligent agent to confirm the first resource transfer tool information.

[0123] In step 311, the agent determines the order information and the first resource transfer tool information.

[0124] In some embodiments, the first intelligent agent determines and generates order information, and the second intelligent agent determines the first resource transfer tool information.

[0125] In some embodiments, the second agent determines from at least one candidate resource transfer tool based on at least one of the following: the user's session context, detection information of the current environment, and user profile features. The user profile features include, but are not limited to, one or more of the following: user preference information for resource transfer tools, user's age, occupation, and user's historical payment records. The detection information of the current environment includes, but is not limited to, one or more of the following: the user's current geographical location, environmental image, and environmental noise. The current geographical location can reflect whether the user is currently in a relatively private and safe environment or a relatively public and unsafe environment; the environmental image can reflect the number of other people in the current payment scenario, and whether the current payment scenario is taking place inside a vehicle, indoors, or outdoors; the environmental noise can reflect whether the current payment environment is relatively quiet or relatively noisy.

[0126] For example, the second agent constructs prompts based on the user's conversation context and user profile features; these prompts are then input into a large model to obtain the first resource transfer tool output by the large model. The user's conversation context can include one or more user commands. By considering not only the user's conversation context but also their profile features when determining the first resource transfer tool using the large model, the dynamically determined first resource transfer tool can not only match the user's current intent but also take into account the user's payment preferences, further improving the user's service experience for shopping and paying via wearable devices.

[0127] For example, the second intelligent agent constructs prompt words based on the user's conversation context, current environment detection information, and user profile features; these prompt words are then input into a large model to obtain the first resource transfer tool output by the large model. Determining the first resource transfer tool in this way allows for a more accurate identification of the appropriate resource transfer tool for the current user and scenario, improving the intelligence, flexibility, and security of users placing orders and making payments based on wearable devices.

[0128] In step 312, the intelligent agent sends order information and first resource transfer tool information to the wearable device via the mobile terminal.

[0129] Specifically, the first intelligent agent can send the order information to the wearable device, and the second intelligent agent can send the information of the first resource transfer tool to the wearable device.

[0130] In step 313, the wearable device displays order information and information about the first resource transfer tool.

[0131] In step 314, the user issues a resource transfer instruction. This instruction can be a voice command, a click or touch of a designated interactive control on the wearable device, or a specific gesture. For example, the resource transfer instruction could be natural language such as the user saying "go to payment," "go to pay," or "confirm payment."

[0132] In step 315, the wearable device transmits a resource transfer instruction to the intelligent agent via a mobile terminal. For example, in this step, the wearable device transmits a resource transfer instruction to a second intelligent agent.

[0133] In step 316, the agent performs resource transfer for the order. For example, in this step, the second agent can perform resource transfer by calling the server of the second application.

[0134] In other embodiments, instead of performing resource transfer through steps 315 and 316, the wearable device can transmit a resource transfer instruction to the server of the second application to trigger the server of the second application to perform resource transfer.

[0135] In step 317, the intelligent agent sends the resource transfer result to the wearable device via a mobile terminal. For example, the second intelligent agent sends the resource transfer result to the wearable device via a mobile terminal.

[0136] In step 318, the wearable device displays the resource transfer results.

[0137] In this embodiment of the disclosure, the above process realizes a convenient, secure, and flexible shopping and payment process based on wearable devices, mobile terminals, and smart agents, which meets the user's needs for easy, fast, and secure search and purchase of the same products without requiring users to perform cumbersome manual search, application switching, and other operations, further improving the user's service experience.

[0138] Figure 4 These are schematic diagrams illustrating the interactive interface presented in an information interaction method according to some embodiments of this disclosure. For example... Figure 4 As shown, the information interaction method is executed by the smart glasses, and the method includes steps 41 to 45.

[0139] In step 41, as Figure 4 As shown in (a), the shooting page is displayed after the user activates the target application in the smart glasses.

[0140] In step 42, in response to the user's gaze duration exceeding a set duration, the smart glasses capture a first image. The smart glasses then transmit the first image to the target application's server. After determining a first item of interest to the user in the first image based on the first image and other information, the server retrieves a second item matching the first item from the merchant system and sends the information of the second item to the smart glasses.

[0141] In step 43, as Figure 4 As shown in (b), the smart glasses display information about a second item. For example, the second item is a shirt from brand A. Furthermore, during the user's interaction with the smart glasses, operation guidance information can be displayed or played via voice to assist the user in performing the next step.

[0142] In step 44, as Figure 4 As shown in (c), after the user says "Place an order for me", the order information for the second item and the recommended payment method (i.e., "First Resource Transfer Tool") are displayed.

[0143] In step 45, as Figure 4 As shown in (d), after the user says "Confirm payment", the payment result for the order is displayed.

[0144] In this embodiment of the disclosure, the above process realizes the shopping and payment process based on wearable devices, which meets the user's needs for easy, fast and secure search and purchase of the same products, without requiring users to perform cumbersome manual search, application switching and other operations, and further improves the user's service experience.

[0145] Figure 5 This is a schematic diagram of the structure of smart glasses according to some embodiments of this disclosure. For example... Figure 5 As shown, the smart glasses 50 are used to perform the information interaction method as described above.

[0146] In this embodiment of the disclosure, the smart glasses meet the user's need to search for and purchase the same product in a simple, fast and secure way, without requiring the user to perform cumbersome manual search, application switching and other operations, thus further improving the user's service experience.

[0147] Figure 6 This is a structural schematic diagram of a wearable device according to other embodiments of this disclosure. For example... Figure 6 As shown, the wearable device 60 includes a memory 61 and a processor 62 coupled to the memory 61. The memory 61 is used to store instructions for executing embodiments of the information interaction method. The processor 62 is configured to execute the information interaction method in any of the embodiments of this disclosure based on the instructions stored in the memory 61.

[0148] Figure 7 This is a schematic diagram of the structure of an information interaction system according to some embodiments of this disclosure. For example... Figure 7 As shown, the information interaction system includes a wearable device 71, a mobile terminal 72, and an intelligent agent 73. In this embodiment of the disclosure, the information interaction method described in the previous embodiments can be executed by cooperating with the wearable device 71, the mobile terminal 72, and the intelligent agent 73.

[0149] In this embodiment of the disclosure, the above system meets the user's need to search for and purchase the same product in a simple, fast and secure way, without requiring the user to perform cumbersome manual search, application switching and other operations, thus further improving the user's service experience.

[0150] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations thereof, can be implemented by computer-readable program instructions.

[0151] These computer-readable program instructions are provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable device to produce a machine, such that execution of the instructions by the processor produces means for implementing the functions specified in one or more boxes of the flowchart and / or block diagram.

[0152] These computer-readable program instructions may also be stored in a computer-readable storage medium. These instructions cause a computer to work in a particular manner to produce an article of manufacture, including instructions that implement the functions specified in one or more boxes in a flowchart and / or block diagram.

[0153] This disclosure may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.

[0154] The information interaction method and wearable device according to this disclosure have now been described in detail. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein based on the above description.

Claims

1. An information interaction method, executed by a wearable device, comprising: A first image is captured, the first image including a first item seen by the user; Display information about the second item, which is an item in the merchant system that matches the first item; In response to the user's first instruction, the order information for the second item and the information of the first resource transfer tool are displayed, the first instruction indicating that an order for the second item be placed; In response to the user's second instruction, the resource transfer result of the order is displayed, wherein the second instruction indicates that the resource transfer of the order is performed.

2. The method according to claim 1, wherein, The step of displaying the resource transfer result of the order in response to the user's second instruction includes: Upon receiving the user's second instruction, verification guidance information is displayed and / or played via voice, the verification guidance information including instructions for the user to perform a specified operation; In response to the user performing the specified operation, the resource transfer result of the order is displayed.

3. The method according to claim 2, wherein, The specified operation is to read aloud the specified information. The response to the user executing the specified operation, displaying the resource transfer result of the order, includes: During the period when the user reads the specified information, the user's first voice information is collected and transmitted to the intelligent agent so that the intelligent agent can perform voiceprint verification on the user based on the first voice information. The specified information is dynamically determined by the intelligent agent. If the voiceprint verification is successful, the resource transfer result of the order will be displayed.

4. The method according to claim 3, wherein, The step of collecting the user's first voice information and transmitting the first voice information to the intelligent agent, so that the intelligent agent can perform voiceprint verification on the user based on the first voice information, includes: During the collection of the first voice information through the wearable device, the mobile terminal is also triggered to collect the user's third voice information; The mobile terminal sends the first voice information and the third voice information to the intelligent agent, so that the intelligent agent can verify the user's voiceprint based on the first voice information and the third voice information.

5. The method according to claim 3, wherein, The display and / or voice playback of verification guidance information is executed when the user's identity verification by the intelligent agent fails based on at least one risk control decision factor, wherein the at least one risk control decision factor includes previously collected voiceprint information of the user; or, The display and / or voice playback verification guidance information is executed when the intelligent agent determines that voiceprint verification cannot be performed based on the previously collected second voice information of the user, or when voiceprint verification based on the second voice information fails.

6. The method according to claim 5, wherein, The step of displaying the resource transfer result of the order in response to the user's second instruction further includes: Upon receiving the user's second instruction and successfully verifying the user's identity based on at least one risk control decision factor, the resource transfer result of the order is displayed; or, Upon receiving the user's second instruction and successfully verifying the voiceprint through the intelligent agent based on the user's previously collected second voice information, the resource transfer result of the order is displayed.

7. The method according to claim 1, wherein, The step of displaying the resource transfer result of the order in response to the user's second instruction includes: In response to the user's second instruction, a resource transfer notification message is sent to the intelligent agent. Upon receiving the notification message, the intelligent agent verifies the user based on a target verification method, and after successful verification, sends the resource transfer result of the order to the wearable device. The resource transfer result of the order is displayed. The target verification method includes one or more of the following: voice verification, voiceprint verification, face verification, fingerprint verification, iris verification, gesture verification, SMS verification code verification, and password verification.

8. The method according to claim 1, wherein, The step of displaying the resource transfer result of the order in response to the user's second instruction includes: In response to a third instruction from the user, information about the second resource transfer tool is displayed, the third instruction indicating a switch to the resource transfer tool; In response to the second instruction, the resource transfer result of the order is displayed.

9. The method according to claim 1, further comprising: During the display of at least one of the information on the second item, the order information for the second item, and the information on the first resource transfer tool, operation guidance information is displayed and / or played via voice to assist the user in performing the next operation.

10. The method according to claim 1, wherein, The information displayed for the second item includes: Display a second image, which contains multiple items from the first image and the numbers of the multiple items; In response to a fourth instruction from the user, information about a second item is displayed, wherein the fourth instruction indicates the number information of the first item selected by the user from the plurality of items.

11. The method according to claim 1, wherein, The acquisition of the first image includes: In response to the user's first action, a shooting page is displayed, the first action being used to invoke the target application; In response to the user's second action, a first image is acquired based on the target application, the second action being used to trigger the acquisition.

12. The method according to any one of claims 1 to 11, wherein, The wearable device is smart glasses, and the information displayed for the second item includes: The first screen of the smart glasses displays first information about the second item, and the second screen of the smart glasses displays second information about the second item, wherein the first information is different from the second information; or The information display page for the second item is displayed. The display page includes a background and an image of the second item, which is presented in a three-dimensional effect.

13. The method according to any one of claims 1 to 11, wherein, The first resource transfer tool is determined by the intelligent agent from at least one candidate resource transfer tool based on at least one of the user's session context, current environment detection information, and user profile features, upon instruction or triggering by the wearable device.

14. A wearable device, comprising: A module for performing the information interaction method as described in any one of claims 1 to 13.

15. A wearable device, comprising: Memory; as well as A processor coupled to the memory, the processor being configured to execute the information interaction method as described in any one of claims 1 to 13 based on instructions stored in the memory.

16. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the information interaction method as described in any one of claims 1 to 13.

17. A computer program product having stored computer program instructions thereon, which, when executed by a processor, implement the information interaction method as described in any one of claims 1 to 13.

Images