Visual payment method, device, equipment and system based on wearable equipment

Abstract

The invention discloses a visual payment method, device, equipment and system based on wearable equipment, and belongs to the technical field of payment. The method comprises the steps that multimedia data collected by wearable equipment of a user are acquired, the multimedia data comprise image data or comprise image data and voice data, and the image data are obtained by tracking eyeball movement of the user through the wearable equipment, determining a user gazing area and shooting the user gazing area; analyzing the multimedia data, extracting to obtain transaction information, and obtaining recommended payment mode information based on the transaction information and a pre-constructed payment mode recommendation model, the payment mode recommendation model being obtained by training based on historical transaction information of the sample user; according to the recommended payment mode information and the transaction information, a payment message is generated, and the payment message is used for initiating payment. According to the embodiment of the invention, the method can adapt to variable payment scenes, and improves the payment flexibility.

Description

Technical Field

[0001] The present application belongs to the field of payment technology, and in particular, relates to a visual payment method, device, equipment and system based on a wearable device. Background Art

[0002] With the continuous development of mobile payment, various new payment methods continue to emerge, such as barcode payment, Near Field Communication (NFC) payment, face payment, etc. The payment scenarios are complex and changeable, and users need to choose the appropriate payment method according to the payment scenario. For example, if the merchant's payment barcode is set in the payment scenario, the user needs to scan the payment barcode through the application in the user terminal to obtain the corresponding information to initiate payment; if the payment scenario supports NFC payment, the user needs to have a user terminal with a built-in NFC module to communicate with the acceptance terminal in close proximity to realize payment; if the payment scenario supports face payment, it is necessary to collect the user's face image through a camera device to verify the payment, but the collection of face images has high requirements for shooting angles, lighting, etc. However, all of the above payment methods need to rely on specific biometrics, dedicated hardware or environmental conditions, and have poor flexibility, and lack a payment method that can adapt to changing payment scenarios. Summary of the invention

[0003] The embodiments of the present application provide a visual payment method, apparatus, device and system based on a wearable device, which can adapt to changing payment scenarios and improve payment flexibility.

[0004] In a first aspect, an embodiment of the present application provides a visual payment method based on a wearable device, comprising: obtaining multimedia data collected by a user's wearable device, the multimedia data including image data, or image data and voice data, the image data being obtained by the wearable device tracking the user's eye movement, determining the user's gaze area and photographing the user's gaze area; parsing the multimedia data, extracting transaction information, and obtaining recommended payment method information based on the transaction information and a pre-built payment method recommendation model, the payment method recommendation model being trained based on historical transaction information of sample users; generating a payment message based on the recommended payment method information and the transaction information, the payment message being used to initiate payment.

[0005] In a second aspect, an embodiment of the present application provides a visual payment device based on a wearable device, including: a data acquisition module, used to acquire multimedia data collected by a user's wearable device, the multimedia data including image data, or image data and voice data, the image data being obtained by the wearable device tracking the user's eye movement, determining the user's gaze area and photographing the user's gaze area; a model processing module, used to parse the multimedia data, extract transaction information, and obtain recommended payment method information based on the transaction information and a pre-built payment method recommendation model, the payment method recommendation model being trained based on historical transaction information of sample users; a message generation module, used to generate a payment message based on the recommended payment method information and transaction information, the payment message being used to initiate payment.

[0006] In a third aspect, an embodiment of the present application provides a visual payment device based on a wearable device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the visual payment method based on the wearable device of the first aspect is implemented.

[0007] In a fourth aspect, an embodiment of the present application provides a visual payment system based on a wearable device, comprising: a wearable device for collecting multimedia data, the multimedia data including image data, or image data and voice data, the image data being obtained by the wearable device tracking the user's eye movement, determining the user's gaze area and photographing the user's gaze area; a visual payment processing platform, which is communicatively connected to the wearable device, and is used to execute the visual payment method based on the wearable device of the first aspect.

[0008] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the visual payment method based on a wearable device of the first aspect is implemented.

[0009] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the visual payment method based on a wearable device of the first aspect.

[0010] The embodiments of the present application provide a visual payment method, device, equipment and system based on wearable devices, which can extract transaction information based on the image data of the user's gaze area obtained by the wearable device tracking the user's eye movement, the user's voice data collected by the wearable device, etc., and obtain recommended payment method information representing the recommended payment method through a payment method recommendation model based on the extracted transaction information, generate a payment message based on the recommended payment method information and the transaction information, and initiate payment through the payment message. Wearable devices initiate payment through natural methods such as gaze and voice, which will greatly reduce the limitations of specific biometrics, dedicated hardware, and environmental conditions, and can realize the acceptance of full-scene payment, thereby adapting to changing payment scenarios and improving payment flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] Figure 1 A flowchart of a visual payment method based on a wearable device provided in one embodiment of the present application;

[0013] Figure 2 A flowchart of an example of a visual payment process provided in an embodiment of the present application;

[0014] Figure 3 A schematic diagram of an example of a visual payment architecture provided in an embodiment of the present application;

[0015] Figure 4 A schematic diagram of an example of a link in the visual payment method provided in an embodiment of the present application;

[0016] Figure 5 A flowchart of a visual payment method based on a wearable device provided in another embodiment of the present application;

[0017] Figure 6 A schematic diagram of an example of combining basic feature information and payment preference information provided in an embodiment of the present application;

[0018] Figure 7 A flowchart of a visual payment method based on a wearable device provided in another embodiment of the present application;

[0019] Figure 8 A schematic diagram of an example of learning and training of a payment method recommendation model provided in an embodiment of the present application;

[0020] Fig. 9A schematic diagram of the structure of a visual payment device based on a wearable device provided in one embodiment of the present application;

[0021] Fig.10 A schematic diagram of the structure of a visual payment device based on a wearable device provided in one embodiment of the present application;

[0022] Fig.11 A schematic diagram of the structure of a visual payment system for a wearable device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0023] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are intended only to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by showing examples of the present application. It should be noted that the acquisition, storage, use, and processing of information and data in the embodiments of the present application are authorized by users or relevant agencies, in compliance with the relevant provisions of national laws and regulations.

[0024] With the continuous development of mobile payment, various new payment methods continue to emerge, such as barcode payment, NFC payment, face payment, etc. Payment scenarios are complex and changeable, and users need to choose the appropriate payment method according to the payment scenario. For example, if the merchant's payment barcode is set in the payment scenario, the user needs to scan the payment barcode through the application in the user terminal to obtain the corresponding information to initiate payment; if the payment scenario supports NFC payment, the user needs to use the user terminal with a built-in NFC module to communicate with the acceptance terminal in close proximity to realize payment; if the payment scenario supports face payment, it is necessary to collect the user's face image through a camera device to verify the payment, but the collection of face images has high requirements for shooting angles, lighting, etc. However, all of the above payment methods need to rely on specific biometrics, dedicated hardware or environmental conditions, and have poor flexibility. There is a lack of a payment method that can adapt to changing payment scenarios.

[0025] The present application provides a visual payment method, device, equipment, system, medium and program product based on wearable devices, which can extract transaction information based on the image data of the user's gaze area obtained by tracking the user's eye movement, the user's voice data collected by the wearable device, etc., and obtain the recommended payment method information through the payment method recommendation model that can output the recommended payment method according to the extracted transaction information, and generate a payment message according to the recommended payment method information and the transaction information, so as to send the payment message to the payment processing platform to initiate payment. On the one hand, the process does not require manual operation by the user, and the payment can be initiated through natural interaction methods such as gaze and voice, which can save manual operation steps such as scanning and inputting. On the other hand, the wearable device initiates payment through natural methods such as gaze and voice, which is greatly reduced by the limitations of specific biometrics, dedicated hardware, and environmental conditions. With the concept of "function is there, structure is completely absent", the acceptance of full-scene payment is realized, thereby improving the flexibility of payment, shortening the time spent on payment, and improving payment efficiency.

[0026] The following describes the wearable device-based visual payment method, device, equipment, system, medium and program product provided in this application.

[0027] The first aspect of the present application provides a visual payment method based on a wearable device, which can be executed by a visual payment processing platform, which can include at least one of a visual payment device, equipment, and system based on a wearable device. The visual payment processing platform can be implemented as the wearable device itself, the background system of the payment application, the payment management system, etc., or it can be implemented as more than two of the wearable device itself, the background system of the payment application, and the payment management system, that is, the wearable device itself, the background system of the payment application, and the payment management system can interact with each other to jointly implement the visual payment method based on the wearable device of the embodiment of the present application. Figure 1 A flowchart of a visual payment method based on a wearable device provided in one embodiment of the present application, such as Figure 1 As shown, the visual payment method based on the wearable device may include steps S101 to S103.

[0028] In step S101, multimedia data collected by the wearable device of the user is obtained.

[0029] Wearable devices may include but are not limited to smart glasses, smart headphones and other devices. Wearable devices have the function of collecting images and voices. After the user wears the wearable device, the wearable device can collect multimedia data around the user. The multimedia data may include image data, or the multimedia data may include image data and voice data. The image data is obtained by the wearable device tracking the user's eye movement, determining the user's gaze area and photographing the user's gaze area. The user's gaze area is the area where the user is gazed, determined by the wearable device by tracking the user's eye movement. The wearable device can photograph the user's gaze area to obtain image data. In the process of using the wearable device, the wearable device can receive the user's voice command and collect voice data.

[0030] In step S102, the multimedia data is parsed to extract the transaction information, and based on the transaction information and a pre-built payment method recommendation model, the recommended payment method information is obtained.

[0031] The multimedia data may be parsed to extract transaction information therefrom. The transaction information may include but is not limited to payment information, order information, etc. The payment information may include but is not limited to user information, merchant information, etc.

[0032] By parsing the image data, the target object can be identified from the image data. The target object may include but is not limited to one or more of the images such as a graphic code, a merchant face image, a merchant name, a bill, a price tag, etc.; according to the target object, the merchant information matching the target object is extracted. The graphic code may include the merchant's payment code. The identified graphic code can be scanned and decoded using a graphic code decoder to obtain the information contained in the graphic code. The graphic code decoder may include but is not limited to a decoder using Zebra Crossing, i.e., ZXing deep learning. For example, if the graphic code contains merchant information such as a merchant logo, the merchant information such as the merchant logo can be extracted as transaction information; if the graphic code includes a Uniform Resource Locator (URL), the URL can be parsed to obtain merchant information such as the merchant logo as transaction information. The merchant information database can store the correspondence between merchant identity and merchant information. After the merchant face image is identified from the image data, the merchant identity can be determined based on the merchant face image through a deep learning face recognition algorithm such as the FaceNet algorithm, thereby obtaining the merchant information; the merchant information database can store the correspondence between merchant face samples and merchant information. After the merchant face image is identified from the image data, the merchant face image can be used to match the merchant face samples in the merchant information database to obtain matching merchant face samples, and then obtain the merchant information corresponding to the matching merchant face samples. The merchant information database may include the correspondence between merchant names and merchant information. The images captured by the wearable device may include merchant signs, counters, and various furnishings in the merchant. If there is a merchant name, the merchant name can be identified from the merchant signs, counters, and other furnishings in the image through optical character recognition (OCR) technology, thereby obtaining the merchant information corresponding to the merchant name from the merchant information database as transaction information. The specific type of OCR technology is not limited here. The identified merchant name can also be combined with the location information of the wearable device, the bill information collected by the wearable device, etc., to determine the corresponding merchant information in the merchant information database. OCR technology combined with natural language processing (NLP) technology can be used to extract key information from the identified bills, such as transaction time, transaction amount, transaction details, merchant name, etc. as transaction information. Convolutional Recurrent Neural Network (CRNN) can also be used to extract structured data from bills and other documents to obtain transaction information. The merchant name extracted from the bill can also be used to determine the merchant information corresponding to the merchant name in the merchant information database as transaction information.OCR technology combined with NLP technology can be used to extract information such as product name, product price, merchant name, etc. from the identified price tag. The merchant information database can store the corresponding relationship between the product information and merchant information of each merchant, and the merchant information corresponding to the product information can be obtained from the merchant information database as transaction information. In some examples, the wearable device can also provide location information. The location information provided by the wearable device during the transaction process can represent the geographical location of the merchant. The location information can include but is not limited to Global Positioning System (GPS) data; the location information can be used to query the list of merchants within a preset distance from the location indicated by the location information, and the merchants can be screened to obtain merchant information; the location information can also be combined with the identified merchant name, billing information, etc. to determine the merchant, so as to obtain the merchant information of the merchant; the merchant information database can store the merchant's geographical location information, and can also use a geographical location matching algorithm to match the location information with the merchant's geographical location information in the merchant information database, so as to determine the merchant, and then obtain the merchant information as transaction information. For voice data, the voice data can be first converted into text data, and then the transaction information in the text data can be extracted using a semantic understanding algorithm. The transaction amount, product information, and product note information can be extracted from the voice data; for example, if a user says "pay 50 yuan to buy a T-shirt" by voice, the transaction amount can be extracted as 50 yuan and the product name can be T-shirt. The input of the payment method recommendation model is information that can reflect the payment environment and payment status, and the payment method recommendation model can output recommended payment method information. Both the payment environment and the payment status are used to characterize the payment scenario, which may include but are not limited to the user's available discount information, the user's payment preference, the transaction amount, etc. The recommended payment method information is characterized as the payment method recommended for this transaction. The number of recommended payment methods represented by the recommended payment method information is not fixed here, and can be one or more. The payment method can represent the type of payment card used for payment, the payment card used for payment, the discount combination, etc. Different payment methods may include payment using different types of payment cards, payment using different payment cards, payment using different discount combinations, etc. Other information associated with the transaction information can be further obtained based on the transaction information. Other information associated with the transaction information may include but is not limited to payment preference information, transaction discount information, etc. Payment preference information can represent the user's preference for payment methods. Transaction discount information can represent the discount items that can be used in the transaction, etc. The discount items may include electronic coupons, full discount rules, etc., which are not limited here. Based on other information associated with the transaction information, information that can represent the payment environment and payment status of this transaction can be obtained, and the information representing the payment environment and payment status of this transaction can be input into the payment method recommendation model to obtain the recommended payment method information output by the payment method recommendation model. The payment method recommendation model is trained based on the historical transaction information of sample users.Information that can characterize the payment environment and payment status of historical transactions can be obtained based on the historical transaction information of sample users. The information that characterizes the payment environment and payment status of historical transactions and the payment methods preferred by sample users are used as training samples for model training, thereby obtaining a payment method recommendation model.

[0033] In step S103, a payment message is generated according to the recommended payment method information and the transaction information.

[0034] At least part of the transaction information required to generate the payment message can be directly extracted from the multimedia data. In some examples, part of the transaction information can also be obtained based on the transaction information extracted from the multimedia data. For example, if the obtained transaction information includes a merchant identifier, the merchant account can be obtained according to the merchant identifier, and the merchant account can also be used as transaction information. After obtaining the recommended payment method information and transaction information, the recommended payment method information and transaction information are filled into each field of the payment message according to the message format to generate the payment message. The payment message may include but is not limited to the transaction amount, merchant account, coupon ID, order note information, payment card information, merchant information, commodity information, etc. In some examples, the transaction information may include information about the payment institution involved in the transaction. Different payment institutions, such as different clearing institutions, different acquiring institutions, etc., may have different requirements for the message format. The corresponding relationship between the payment institution and the message format can be established in advance, and the corresponding message format can be determined according to the transaction information, and the payment message is generated according to the corresponding message format.

[0035] The payment message is used to initiate payment, integrating multiple dimensions such as user intention, product information, and preferential information. Before or after the payment message is generated, the user who initiates the payment can be authenticated, and the security risk verification of this payment can also be performed. The identity verification may include identity verification using biometrics and other forms of identity verification. If the user's identity verification fails and / or the security risk verification fails, the current payment is blocked and the reason for the payment failure is fed back to the user. If the user's identity verification passes and the security risk verification passes, the current payment continues, the user's account is deducted, a payment success certificate is issued, and the payment completion information is visually displayed to the user through the wearable device. After the payment message is generated, the visual payment device and device based on the wearable device can send the payment message to the payment processing platform so that the payment processing platform completes the payment according to the payment message; in the case where the visual payment device and device based on the wearable device that generates the payment message does not include a wearable device, the visual payment device and device based on the wearable device can also send the payment message to the wearable device, and the wearable device sends the payment message to the payment processing platform so that the payment processing platform completes the payment according to the payment message. The payment platform may include but is not limited to the payment management system, the backend system of the payment application, etc. For example, Figure 2A flowchart of an example of a visual payment process provided in an embodiment of the present application, such as Figure 2 As shown, the visual payment process may include steps a1 to a8.

[0036] In step a1, the wearable device captures image data and voice commands and sends them to the payment management system.

[0037] In step a2, the payment management system analyzes the image data and voice command, obtains transaction information and feeds it back to the wearable device. The transaction information includes merchant information.

[0038] In step a3, the wearable device obtains recommended payment method information based on the transaction information and sends the recommended payment method information to the payment management system.

[0039] In step a4, the wearable device makes payment.

[0040] In step a5, the payment management system generates a payment message based on the transaction information and the recommended payment method information, and sends the payment message to the payment application backend system to initiate payment.

[0041] In step a6, the payment management system sends a payment collection notification to the merchant device.

[0042] In step a7, the payment management system sends a payment notification to the wearable device.

[0043] In step a8, the wearable device may initiate a payment query to the payment management system.

[0044] For the specific contents of the above steps a1 to a8, please refer to the relevant descriptions in the embodiments of the present application and will not be repeated here.

[0045] In the embodiment of the present application, the visual payment architecture based on wearable devices may include a scene layer and a large model layer. Figure 3 A schematic diagram of an example of a visual payment architecture provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the visual payment architecture may include a scene layer 21 and a large model layer 22 .

[0046] The scenario layer 21 includes the participants in the payment scenario and the links of transaction information processing, user identity verification, security risk verification, etc. The scenario layer 21 includes the user side 211 and the merchant side 212. The user side may include wearable devices such as smart glasses, smart headphones, smart watches, and may also include electronic devices such as mobile phones. The merchant side 212 may provide graphic codes, facial images, bills, merchant names, location information, etc. that can be collected by wearable devices.

[0047] The large model layer 22 can use the pre-trained large model to complete the multimodal data understanding, voice mapping, payment message generation and other functions transmitted from the scene layer. Since the data transmitted by the wearable device includes data of multiple modes such as image data and voice data, it is necessary to pre-train the language model, visual model, etc., so as to uniformly map the multimodal data to the semantic space, thereby integrating data of different modes and forming the same understanding of user intentions and commodity attributes. This part of the function can be achieved through the multimodal semantic understanding large model 221. Since the visual payment architecture focuses on payment transactions, the language model can be trained on the basis of the payment transaction corpus using the self-supervised learning method, and the knowledge in the payment transaction field can be embedded in the payment transaction knowledge large model 222. The large model layer 22 can fill the acquired transaction information and recommended payment method information into the fields of the payment message to generate the payment message, and the large model layer 22 can flexibly adapt the message format of different payment scenarios. The large model layer 22 can also detect the security risks of the payment message through the anomaly detection large model 223 trained by the comparative learning method, and give real-time warnings for suspected abnormal payments to prevent payment risks.

[0048] The scene layer 21 and the large model layer 22 can also realize other functions such as providing recommended payment method information, which are not elaborated here.

[0049] The visual payment method based on a wearable device in the embodiment of the present application may include multiple steps. To facilitate understanding of the overall process, the multiple steps of the visual payment method based on a wearable device are explained below through a schematic diagram. Figure 4 A schematic diagram of an example of a link in the visual payment method provided in an embodiment of the present application, such as Figure 4 As shown, the visual payment method based on wearable devices may include a demand identification link 31, an information analysis link 32, a security verification link 33 and a payment link 34. Multiple links may involve users 35, wearable devices 36 and multiple functional units. The functional units include a multimodal semantic understanding unit 371, a payment transaction knowledge unit 372, a payment message filling unit 373 and an anomaly detection unit 374.

[0050] In the demand identification link 31, the user 35 performs visual recognition and voice input, and the wearable device 36 can use eye tracking technology to achieve visual recognition, and record and push the voice input. The wearable device 36 uses the eye tracking technology to obtain image data and the voice data obtained by voice processing to transmit to the multimodal semantic understanding unit 371. The multimodal semantic understanding unit 371 performs feature extraction and image analysis on the image data, and extracts transaction information such as the merchant's payment account number. The image analysis can also be assisted by NLP semantic analysis to extract transaction information such as transaction amount and order note information; the multimodal semantic understanding unit 371 performs voice-to-text conversion on the voice data, and extracts transaction information such as transaction amount and order note information with the help of NLP semantic analysis.

[0051] In the information analysis link 32, the payment transaction knowledge unit 372 can obtain transaction information from the multimodal semantic understanding unit 371, map the merchant information with the transaction information, and perform available discount calculations to obtain recommended payment method information, and transmit the recommended payment method information, merchant information, etc. to the wearable device 36. The wearable device 36 displays the recognition result, which may include the recommended payment method information and merchant information. The user 35 can confirm the recognition result displayed by the wearable device 36. After confirming the recognition result, a payment message can be generated by the payment message filling unit 373.

[0052] In the security verification link 33, the abnormality detection unit 374 can perform abnormal risk detection on the payment message. The user 35 can provide biometric features such as facial images and voiceprints. The wearable device 36 collects the biometric features of the user 35 and transmits the biometric features to the abnormality detection unit 374. The abnormality detection unit 374 can perform security risk verification and identity authentication based on the biometric features.

[0053] In the payment link 34, if the abnormal risk detection, security risk verification and identity authentication are all passed, the wearable device 36 initiates the deduction and notifies the user of the transaction result 35. The transaction result includes payment success or payment failure. If the payment fails, the reason for the payment failure can also be fed back and analyzed.

[0054] In the embodiment of the present application, the transaction information can be extracted based on the image data of the user's gaze area obtained by the wearable device tracking the user's eye movement, the user's voice data collected by the wearable device, etc., and the recommended payment method information representing the recommended payment method can be obtained through the payment method recommendation model based on the extracted transaction information, and a payment message is generated based on the recommended payment method information and the transaction information, and the payment is initiated through the payment message. Wearable devices initiate payments through natural methods such as gaze and voice, which will greatly reduce the limitations of specific biometrics, dedicated hardware, and environmental conditions, and can realize the acceptance of full-scene payments, thereby adapting to changing payment scenarios and improving payment flexibility.

[0055] In some embodiments, user identity authentication may be performed on the user using the wearable device first. If the user identity authentication is passed, a payment message is generated based on the recommended payment method information and transaction information, thereby ensuring that the user using the wearable device is a legitimate user and preventing the wearable device from being stolen.

[0056] User identity authentication may include one or more of the following: obtaining the user's biometrics collected by the wearable device, and performing user identity authentication based on the biometrics and the user's registered biometric samples; obtaining device information of an electronic device that is in a communication connection with the wearable device, and performing user identity authentication based on the device information and pre-registered binding device information; obtaining the user's eye movements collected by the wearable device, and performing user identity authentication based on the eye movements and pre-set eye verification movements; obtaining the user's behavior information collected by the wearable device, and performing user identity authentication based on the behavior information and pre-obtained user behavior characteristic conditions.

[0057] Biometrics may include but are not limited to voiceprints, irises, fingerprints, etc. For example, if the biometrics include voiceprints, a sound receiving device and a voiceprint recognition algorithm may be pre-embedded in the wearable device. The sound receiving device may be implemented as a highly sensitive microphone or other device to match the biometrics with the user's registered voiceprint sample. If the match is successful, it can be determined that the user's identity verification has been passed, so as to ensure that only the voice of the pre-registered user can unlock the payment function of the wearable device and perform subsequent payment operations. The voiceprint recognition algorithm can continuously optimize the recognition accuracy through machine learning, so that it can distinguish subtle voice differences and reduce the possibility of imitating voice to pass user identity verification.

[0058] The user can bind his electronic device with the wearable device in advance, and use the electronic device bound to the wearable device as a trusted device. The electronic device can be a device other than the wearable device used for payment, such as a mobile phone, smart watch, smart bracelet, etc. The bound device information includes the device information of the electronic device bound to the wearable device. During the user identity authentication process, the device information of the electronic device that is in communication connection with the wearable device at that time can be obtained. The specific method of the communication connection is not limited here. For example, the communication connection can be implemented as but not limited to a Bluetooth connection. The obtained device information is compared with the bound device information pre-registered by the user. If the obtained device information is consistent with the bound device information, it can be determined that the user identity authentication is passed. In other words, the payment operation is allowed only when the wearable device is in a communication connection state with a registered trusted electronic device of the user. The communication connection state here is a real-time communication connection state that maintains an active connection. Whether the wearable device is in a communication connection state with the electronic device can be determined by real-time connection detection, thereby further preventing the wearable device from being stolen by others.

[0059] Users can also preset eyeball verification actions. When payment is required, the user's eyeball movements are collected. Whether the user's identity verification is passed is determined by whether the collected eyeball movements match the preset eyeball verification actions. Eyeball verification actions can be some specific eyeball movement directions or attention orientation sequences. For example, the eyeball verification action can be "up, up, down, down". If the collected user's eyeball movements are also "up, up, down, down", it can be determined that the user's identity verification is passed. Allow users to define the length, complexity and verification frequency of eyeball verification actions according to their own habits and needs, so as to achieve the security of verification and the convenience of user use. In order to further improve the accuracy of user identity verification, machine learning algorithms can be used to conduct in-depth analysis of the user's eyeball movement patterns, learn the difference between natural eyeball movements and deliberately imitated eyeball movements, so as to be able to identify and distinguish natural eyeball movements from deliberately imitated eyeball movements, and further reduce the possibility of illegal users simulating user eyeball movements to pass user identity verification. In some examples, when a user authenticates their identity through eye movements, the wearable device can provide prompt information to the user to guide the user to complete the correct eye movement in real time. For example, the wearable device can prompt the user to complete the correct eye movement through vibration prompts, sound prompts, or virtual visual prompts, thereby improving the accuracy of user authentication and user experience. Authentication through eye movements utilizes the user's unique physiological characteristics and memory ability, which can improve the security of user authentication.

[0060] The user's behavior information may characterize the behavior characteristics of the user's identity authentication, and may include but is not limited to the number of times the user performs identity authentication within a predetermined time period, the duration of the user's identity authentication, etc. The behavior characteristic conditions include the conditions that the behavior information should meet to pass the user identity authentication. For example, the behavior characteristic conditions may include but are not limited to: the number of times the user identity authentication is performed within a predetermined time period is within a preset number of times, the duration of the user identity authentication is within a preset duration range, etc. If the user's behavior information meets the behavior characteristic conditions, it can be determined that the user identity authentication is passed; if the user's behavior information does not meet the behavior characteristic conditions, it can be determined that the user identity authentication is not passed.

[0061] It should be noted that the above-mentioned user identity verification methods can also be used in combination to further improve the accuracy of user identity verification. In the event that user identity verification fails, an additional security verification process can be triggered or the payment function can be temporarily locked.

[0062] In some embodiments, unified user identification across platforms and devices can also be achieved based on a card management platform for managing the user's payment card. The user completes user identity authentication on the card management platform through the wearable device, and a seamless experience of binding the wearable device with the user's payment card can be achieved. A binding request can be sent to the card management platform, and the binding request includes user information; when the user indicated by the user information is a registered user of the card management platform, a payment card list sent by the card management platform is received, and the payment card list includes the card information of the user's payment card indicated by the user information, and the payment card list can indicate all payment cards that the user has that support binding; in response to the selection input for the payment card list, the payment card indicated by the selection input is bound to the wearable device. The selection input can select the card information of one payment card or the card information of multiple payment cards, thereby realizing one-click binding of the user's payment card, without the user having to enter information such as the card number and verification code, and simplifying the binding process of the payment card and the wearable device. During the binding process, the user can also be authenticated to ensure the security of the binding operation and prevent the user's payment card from being stolen. Identity authentication may include but is not limited to device fingerprint verification, verification code verification, biometric verification, and other methods.

[0063] In some embodiments, payment card binding of wearable devices can realize cross-platform synchronization and centralized management of payment card information. For example, real-time synchronization of payment card information between multiple platforms such as wearable device platform, application platform, web platform, etc. can be realized. Card information can also be viewed, edited and managed through wearable devices to improve user experience. According to the information query instruction sent by the wearable device, the transaction information, bills, etc. of the user's historical transactions can be fed back to the wearable device so that the user can view the transaction information of the user's historical transactions through the wearable device. Reminder information can also be set in advance, and when the reminder time limit is reached, the reminder information is sent to the wearable device.

[0064] In some embodiments, associated transaction discount information, user payment preference information for products, etc. can be obtained based on transaction information, and recommended payment method information can be obtained by combining transaction discount information, payment preference information, etc. to provide users with recommended payment methods from multiple aspects. Figure 5 This is a flowchart of a visual payment method based on a wearable device provided in another embodiment of the present application. Figure 5 and Figure 1 The difference is that Figure 1 Step S102 in the above example can be specifically broken down into the following steps: Figure 5 Steps S1021 to S1025 in .

[0065] In step S1021, the multimedia data is parsed to extract transaction information.

[0066] In step S1022, based on the transaction information, transaction preferential information associated with the transaction information is obtained, and based on the transaction preferential information associated with the transaction information, preferential combination information whose preferential amount meets the preset conditions is obtained.

[0067] The transaction information may include payment card information, and the transaction preferential information associated with the associated information may be obtained according to the payment card information. The transaction preferential information may include, but is not limited to, preferential information such as full reduction rules, electronic coupons, electronic red envelopes, and points corresponding to each payment card of the user. The transaction preferential information corresponding to the payment cards available to the user may be traversed to form different preferential combinations. A payment method may correspond to at least one preferential combination, and the preferential amount of each preferential combination is calculated. A preferential combination whose preferential amount meets the preset conditions is selected from multiple preferential combinations, and preferential combination information is obtained. The preferential combination information is used to characterize the preferential combination. In some examples, the preset conditions may include preferential combination information of the preferential combination with the largest preferential amount under each payment method, that is, the preferential combination information with the largest preferential amount corresponding to each payment method is to be obtained. Specifically, the preferential combination information whose preferential amount meets the preset conditions corresponding to each payment method may be obtained through a dynamic planning table. The dynamic planning table may be initialized, and then the dynamic planning table may be filled according to the transaction amount and the preferential combination information, so as to obtain the preferential combination information whose preferential amount meets the preset conditions.

[0068] In step S1023, payment preference information associated with the transaction information is obtained according to the transaction information.

[0069] The user's choice of payment method is closely related to the type of goods purchased by the user, which may include physical goods or services. For example, when users take the bus or subway, they tend to use payment method A1; when users buy daily necessities, they tend to use payment method A2; when users buy household appliances offline, they tend to use payment method A3. The embodiment of the present application can obtain the user's preference for the payment method corresponding to the type of goods purchased by the user this time based on the transaction information. The payment preference information associated with the transaction information can characterize the user's preference for the payment method used for the type of goods purchased this time. The payment preference information can be obtained based on the user's total payment preference and the payment preference corresponding to the type of goods. The payment preference information can be used as a payment preference profile of the user.

[0070] In some examples, the transaction information may include user information and product information. User information is used to characterize the user, for example, the user information may include a user ID, etc. Product information is used to characterize the product, for example, the product information may include a product name, etc. A user latent vector corresponding to the user information and a product latent vector corresponding to the product information may be obtained; an inner product operation is performed on the user latent vector corresponding to the user information and the product latent vector corresponding to the product information to obtain payment preference information associated with the transaction information.

[0071] The user latent vector is used to characterize the user's preference for payment methods, and the commodity latent vector is used to characterize the preference for payment methods for various types of commodities. Different users have different corresponding user latent vectors, and different commodity types have different corresponding commodity latent vectors. The user latent vector corresponding to the user information is the user latent vector corresponding to the user indicated by the user information. The commodity latent vector corresponding to the commodity information is the commodity latent vector corresponding to the commodity type represented by the commodity information. The user latent vector of each user and the commodity latent vector corresponding to various commodity types can be generated in advance based on the historical transaction information of each user, so that in the visual payment process, the user latent vector of the user making this payment and the commodity latent vector corresponding to the commodity type involved can be directly called. It is also possible to obtain the user's historical transaction information after determining the user according to the user information in the visual payment process, and directly calculate the user latent vector of the user and the commodity latent vector corresponding to the commodity type involved.

[0072] Specifically, the user's co-occurrence matrix can be generated based on the user's historical transaction information. The user's co-occurrence matrix is ​​used to characterize the user's preference for payment methods for different types of goods; the user's co-occurrence matrix is ​​decomposed into a user latent vector matrix and a product latent vector matrix. The user latent vector matrix includes user latent vectors, and the product latent vector matrix includes product latent vectors. The user's co-occurrence matrix is ​​used to characterize the payment methods preferred by users to purchase goods of different types during historical transactions. Each row of the co-occurrence matrix represents a user, and each column represents a type of product. The value of the element in the co-occurrence matrix is ​​the identifier of the payment method of the user represented by the row for the product of the type represented by the column. The identifier of the payment method can be obtained by encoding, for example, the identifier of the payment method can be obtained using a one-hot encoding algorithm. The relationship between the co-occurrence matrix and the user latent vector matrix and the product latent vector matrix can satisfy the following formula (1):

[0073] R≈PQ T (1)

[0074] Among them, R is the co-occurrence matrix; P is the user latent vector matrix, and P∈R m*k , m is the number of users, k is the dimension of the latent vector; Q is the product latent vector matrix, and Q∈R n*k , m is the number of commodity types, Q T is the transposed matrix of the product latent vector matrix.

[0075] By decomposing the co-occurrence matrix, users and merchant types can be located in the representation space of latent vectors. The closer the user latent vector is to the product latent vector, the closer the distance between the latent vectors is. The loss function can be used to determine whether the user latent vector matrix and the product latent vector matrix obtained by decomposing the co-occurrence matrix are accurate. In some examples, the user latent vector matrix and the product latent vector matrix can be iterated by stochastic gradient descent to reduce the loss function until the loss function meets the preset loss condition, thereby obtaining accurate user latent vector matrix and product latent vector matrix. A first latent vector matrix and a second latent vector matrix may be randomly generated; according to the user's co-occurrence matrix, the first latent vector matrix, the second latent vector matrix and a preset learning rate parameter, the first latent vector matrix and the second latent vector matrix are gradient updated to obtain an updated latent vector matrix; based on the user's co-occurrence matrix and the updated latent vector matrix, a loss function is obtained; if the loss function satisfies a preset loss condition, the updated latent vector matrix is ​​determined as a user latent vector matrix and a commodity latent vector matrix; if the loss function does not satisfy the preset loss condition, the updated latent vector matrix is ​​gradient updated based on the user's co-occurrence matrix, the updated latent vector matrix and the preset learning rate parameter until the loss function of the updated updated latent vector matrix satisfies the preset loss condition, and the updated updated latent vector matrix is ​​determined as a user latent vector matrix and a commodity latent vector matrix. The updated latent vector matrix obtained after updating the first latent vector matrix and the second latent vector matrix includes a user updated latent vector matrix corresponding to the first latent vector matrix and a merchant updated latent vector matrix corresponding to the second latent vector matrix. The preset loss condition may include that the loss function reaches a minimum value or the loss function is less than a preset loss value, which is not limited here. The gradient update process of the latent vector matrix can be shown as follows (2) to (5), and the loss function can be shown as follows (6):

[0076]

[0077]

[0078] Among them, P μ ′ is the user latent vector matrix after gradient update; if P μ is the first latent vector matrix, then P μ ′ is the user's updated latent vector matrix; if P μ Update the latent vector matrix for the user, then P μ ′ is the updated user updated latent vector matrix; Q′ t is the product latent vector matrix after gradient update; if Q t is the second latent vector matrix, then Q′ t Update the latent vector matrix for the product; if Q t Update the latent vector matrix for the product, then Q′ tUpdate the latent vector matrix for the updated product; η is the learning rate parameter; L is the loss function; R μ,t is the co-occurrence matrix; λ is the regularization parameter. The loss function has a regularization term to prevent overfitting.

[0079] After obtaining the user latent vector matrix and the merchant latent vector matrix, the user latent vector corresponding to the user information and the commodity latent vector corresponding to the commodity information can be obtained. The payment preference information can be obtained based on the following formula (7):

[0080]

[0081] in, P is the payment preference vector of user μ for purchasing goods of type t; μ is the user latent vector of user μ; Q t is the commodity latent vector of commodity type t. The payment preference information can be implemented as at least some elements in the payment preference vector. Each element in the payment preference vector can represent the weight of each payment method used by the user to purchase commodities of the commodity type. The information of the first M payment methods from high to low weights can be used as the payment preference information. M is a positive integer, which can be 1 or an integer greater than 1.

[0082] In step S1024, a transaction state vector is generated according to the preferential combination information and the payment preference information.

[0083] The transaction state vector is obtained by converting the preferential combination information and the payment preference information. The transaction state vector can reflect the current payment environment and payment status. In addition to being related to the preferential combination information and the payment preference information, the transaction state vector can also be related to the historical transaction information, that is, the transaction state vector is generated based on the preferential combination information, the payment preference information and the historical transaction information.

[0084] In step S1025, the transaction state vector is input into the payment method recommendation model. The payment method recommendation model uses a neural network to generate a value parameter of the payment method based on the transaction state vector. Based on the value parameter, N payment methods are selected from a plurality of payment methods to obtain recommended payment method information.

[0085] The payment method recommendation model includes a neural network. The input layer of the neural network can accept a state vector. After being processed by the multiple hidden layers of the neural network, the value parameter output by the output layer of the neural network is obtained. The value parameter can represent the value of the action of selecting a payment method to the payment environment and payment state represented by the transaction state vector. The value parameter can be realized as the Q value of the Q value function.

[0086] According to the value parameter, N payment methods can be selected from multiple payment methods, and the information of the N payment methods can be used as the recommended payment method information. N is a positive integer; if N=1, one payment method is recommended to the user; if N>1, multiple payment methods are recommended to the user. In some examples, the first N payment methods can be selected from multiple payment methods in the order of value parameters from high to low to obtain the recommended payment method information. That is, the N payment methods with the highest value are recommended to the user. In other examples, according to the preset selection probability and value parameter, N payment methods are selected from multiple payment methods to obtain the recommended payment method information. The selection probability is the probability that the selected payment method is the first N payment methods from high to low value parameters. According to the preset selection probability and value parameter, N payment methods are selected from multiple payment methods, that is, under the possibility of selection probability, the information of the first N payment methods from high to low value parameters will be used as the recommended payment method information, and under the possibility of the difference between 1 and the selection probability, the information of the payment methods in N will be randomly selected as the recommended payment method information. By using the selection probability, a payment method can be randomly selected with a certain probability for exploration, which can reasonably expand the user's payment method selection surface and further improve the flexibility of payment. The recommended payment method information obtained according to the selection probability can also be used as sample data to further train the payment method recommendation model, improve the exploration degree of the payment method recommendation model training, and further improve the comprehensiveness and accuracy of the payment method recommendation model.

[0087] In some embodiments, in addition to obtaining the user's payment preference information, the user's basic feature information can also be obtained, and the association learning rule algorithm is used to learn the association rules between the user's payment preference information and the user's basic feature information, so that the payment preference information and the basic feature information can participate in each other's update, making the payment preference information and the basic feature information more accurate and more in line with user needs.

[0088] The basic feature information of the user can be obtained based on the user's historical transaction information, and the user can be clustered based on the basic feature information to obtain the user's location information in the clustering space; the user's feature graph network can be constructed based on the user's payment preference information; based on the pre-learned association between the feature graph network and the clustering space, the user's location information in the clustering space is updated using the user's feature graph network; the user's payment preference information is updated using the payment preference information of the user's neighboring users in the clustering space, and the distance between the location of the neighboring users in the clustering space and the user's location is within a preset distance range.

[0089] The basic characteristic information of users can represent the shopping habits and personal information of users, so as to mine the hidden characteristics of users. The basic characteristic information of users can be used as the basic portrait of users. The basic information of users, the information of users' consumption behavior and the information of users' payment behavior can be obtained from the historical transaction information of users, and the obtained information can be converted into vectors. The vectors of various information can form a vector set or a comprehensive vector, and the vector set or the comprehensive vector can be used as the basic characteristic information of users. The basic information of users can include but is not limited to the age of users, the gender of users, the region where users are located, the credit rating information of users, the information of merchants preferred by users, etc. The information of merchants preferred by users can represent merchants whose number of transactions with users exceeds the preset number within a predetermined time period. The information of users' consumption behavior can include but is not limited to price sensitivity information, commodity type preference information, commodity brand preference information, etc. Price sensitivity information can represent the degree of users' reaction to changes in commodity prices. The commodity type preference information can represent the user's purchase preference for commodity types. The commodity brand preference information can represent the user's purchase preference for commodity brands. The information of users' payment behavior can include but is not limited to payment frequency information, payment amount information, payment time, payment channel information, etc. Since the data types of various historical transaction information of users are different, such as continuous type information, classification type information and other types of information, it is necessary to convert this information into a vector format for easy processing. For example, for continuous type information, the information can be mean normalized, and the original data set of the information can be normalized to a data set with a mean of 0 and a variance of 1, and the data set can be used as a vector. For another example, for classification type information, such as user gender, user credit rating information, etc., embedding technology, namely the Embedding method, can be used for conversion to obtain the corresponding vector. For another example, for other types of information, such as sequence information representing payment behavior, attribute label information, etc., encoding technology can be used to convert the information into a numerical vector. The encoding technology is not limited here. For example, multi-hot encoding technology, namely multi-hot encoding technology, can be used.

[0090] Using a clustering algorithm, users are clustered in a clustering space based on their basic feature information. When the clustered user clusters are obtained, the location information of each user in the clustering space can also be obtained. The location information can represent the location of the user in the clustering space. The type of clustering algorithm used is not limited here.

[0091] The payment preference information of each user may correspond to the feature graph network of the user. The feature graph network may characterize the payment preference information of the user. In some examples, the feature graph network may include a master node and a slave node, the master node characterizes the user, the slave node characterizes the commodity type and / or the payment method, the edge between the master node and the slave node may characterize the payment method preferred by the user for purchasing the commodity type and / or the degree of preference of the user for the payment method, and the edge between the slave nodes may characterize the degree of preference for the payment method for the commodity type purchased. In some examples, the feature graph network of the user may also be constructed based on the payment preference information of the user and the basic feature information of the user, and correspondingly, the slave nodes in the feature graph network may also characterize the basic feature information of the user.

[0092] The association learning rule algorithm can be used in advance to learn the association relationship between the feature graph network and the location information of the user in the cluster space. Through the association relationship between the feature graph network and the location information of the user in the cluster space, the user's feature graph network is used to update the location information of the user in the cluster space. After the location information is updated, the user's position in the cluster space will change, and correspondingly, the distance between different users in the cluster space will also change. The closer the distance between the users in the cluster space, the more similar the users are. The characteristics of the user's neighboring users have a high degree of similarity with the characteristics of the user. The payment preference information of the neighboring users can be used to update the payment preference information of the user. For example, the update weight can be set according to the distance between the user and the neighboring users in the cluster space. The closer the distance, the greater the update weight. The payment preference information and weight of the neighboring users are used to update the payment preference information of the user. Similarly, the payment preference information of the user can also be used to update the payment preference information of the neighboring users. The updated payment preference information of the user is more comprehensive and accurate. The payment preference information used to generate the transaction state vector in the above embodiment may include the payment preference information of the user updated by using the payment preference information of the neighboring users.

[0093] In some examples, users in the cluster space may include a first user and a second user. The first user may include a user whose registration duration does not exceed a preset registration duration, and the first user may be regarded as a new user. The second user may include a user whose registration duration exceeds a preset registration duration, and the second user may be regarded as an old user. In the case where the first user's neighboring users include the second user, the payment preference information of the second user, i.e., the new user, may be used to update the payment preference information of the first user, i.e., the old user. In the case where the second user's neighboring users include the first user, the payment preference information of the first user, i.e., the old user, may be used to update the payment preference information of the second user, i.e., the new user. In the case where there are fewer information records of the new user, the payment preference of the new user may be inferred, thereby improving the accuracy and suitability of the payment method recommended for the new user.

[0094] For example, Figure 6A schematic diagram of an example of combining basic feature information and payment preference information provided in an embodiment of the present application, such as Figure 6 As shown, the user's historical transaction data is divided into basic feature information and payment preference information from the user dimension and payment dimension, the clustering algorithm is used to classify the users based on the basic feature information, and the association learning rule algorithm is used to learn the association relationship between the user's location information in the clustering space and the feature graph network corresponding to the payment preference information, so that the users' payment preference information can assist each other in updating each other, thereby realizing the inference of the payment preferences of new users and the update of the payment preferences of old users.

[0095] In some embodiments, model training may be performed based on historical transaction information of sample users to obtain a payment method recommendation model and put it into use. Figure 7 A flowchart of a visual payment method based on a wearable device provided in another embodiment of the present application, Figure 7 and Figure 1 The difference is that Figure 7 The wearable device-based visual payment method shown may also include steps S104 to S107.

[0096] In step S104, based on the historical transaction information of the sample user, transaction preferential information associated with the historical transaction information is obtained, and based on the transaction preferential information associated with the historical transaction information, preferential combination information is obtained.

[0097] In step S105, payment preference information of the sample user is obtained based on historical transaction information.

[0098] In step S106, a sample state vector is generated according to the preferential combination information and the payment preference information of the sample user.

[0099] The sample state vector can represent the payment state of the sample user. The specific contents of the preferential combination information, payment preference information and sample state vector obtained in the above steps S104 to S106 can be referred to the relevant contents in the above embodiments, which will not be repeated here.

[0100] In step S107, the initial recommendation model including the neural network is trained based on the sample state vector so that the value parameters predicted by the value function of the neural network meet the value prediction conditions, thereby obtaining a payment method recommendation model.

[0101] The payment environment, payment status, etc. used by the initial recommendation model can be defined first. The payment environment represents the user's payment scenario, which may include but is not limited to the transaction discount information corresponding to the user's payment method, the user's basic feature information, and historical transaction information. The payment status represents the description of the payment scenario, which may include but is not limited to the transaction amount, transaction discount information, and the user's payment preference information. During each payment process, the payment status can be updated according to the user's choice of payment method in the payment environment, thereby calculating the value parameter of the user's behavior of choosing a payment method. The value parameter here can also be regarded as a reward value.

[0102] The initial recommendation model can be a deep reinforcement learning model. The initial recommendation model can create a neural network and use the neural network to approximate value parameters, so that the value parameters obtained by the trained initial recommendation model gradually approach the real value parameters, that is, the difference between the value parameters obtained by the trained initial recommendation model and the real value parameters is within a preset error range, thereby enabling the initial recommendation model to accurately evaluate the value parameters of the payment status and payment method selection behavior.

[0103] Specifically, the sample state vector can be input into the neural network to obtain the value parameters of multiple payment methods; according to the gap between the value parameters and the measured value parameters, the network parameters of the neural network are updated, and sample data are randomly selected from the memory bank to train the initial recommendation model until the gap between the value parameters and the measured value parameters meets the value prediction conditions, thereby obtaining a payment method recommendation model.

[0104] The sample state is input into the input layer of the neural network. After being processed by multiple hidden layers in the neural network, the output layer in the neural network can output the value parameters corresponding to each payment method. The network parameters of the neural network can be updated through experience playback and value parameter learning, so that the value parameters output by the neural network are closer and closer to the measured value parameters, until the difference between the value parameters output by the neural network and the measured value parameters meets the value prediction conditions. The measured value parameters are real value parameters. The value prediction conditions can be set according to scenarios, needs, experience, etc. For example, the value prediction conditions may include that the difference between the value parameters and the measured value parameters is within a preset deviation range.

[0105] The sample data in the memory bank includes paired sample state vectors and sample payment method information. The sample payment method information is obtained according to the output of the initial recommendation model obtained by using the sample state vector as the input of the initial recommendation model during the training process. The output of the initial recommendation model is related to the preset selection probability. During the model training process, the sample payment method information output by the initial recommendation model or the updated initial recommendation model and the input sample state vector can be stored in the memory bank as candidate training samples. Some candidate training samples can be randomly selected from the memory bank for model training to achieve experience playback, reduce the correlation between the training samples used in the training model, and improve the stability of model training. The sample payment method information output by the initial recommendation model or the updated initial recommendation model is the information of the payment method output according to the selection probability. The initial recommendation model or the updated initial recommendation model can obtain the value parameters corresponding to multiple payment methods under the condition that the sample state vector is used as the input, and can be sorted from high to low according to the value parameters, and combined with the selection probability, the output payment method information is determined. The selection probability is the probability that the output sample payment method information is the information of the top N payment methods with value parameters from high to low. That is to say, the initial recommendation model or the updated initial recommendation model has the possibility of outputting the information of the top N payment methods with value parameters from high to low with the selection probability, and has the possibility of randomly outputting the information of N payment methods among multiple payment methods with the difference of 1 and the selection probability.

[0106] For example, Figure 8 A schematic diagram of an example of learning and training of a payment method recommendation model provided in an embodiment of the present application is shown as follows: Figure 8 As shown in the figure, the Lagrangian function as a constraint condition can be solved according to the available card benefit information, coupon information and points information corresponding to the user's payment method, so as to obtain the preferential combination information with the maximum preferential value corresponding to each payment method. The user's co-occurrence matrix is ​​decomposed to obtain the user's latent vector and the product's latent vector, and then the user's payment preference information is obtained according to the user's latent vector and the product's latent vector. Combining the preferential combination information and the payment preference information, the payment method recommendation model can be trained through reinforcement learning of the state, behavior, reward, i.e., value parameter, and payment environment.

[0107] A second aspect of the present application provides a visual payment device based on a wearable device. Fig. 9 A schematic diagram of the structure of a visual payment device based on a wearable device provided in one embodiment of the present application, such as Fig. 9 As shown, the wearable device-based visual payment apparatus 400 may include a data acquisition module 401, a model processing module 402 and a message generation module 403.

[0108] The data acquisition module 401 can be used to acquire multimedia data collected by the user's wearable device. The multimedia data includes image data, or image data and voice data. The image data is obtained by the wearable device tracking the user's eye movement, determining the user's gaze area and photographing the user's gaze area.

[0109] The model processing module 402 can be used to parse multimedia data, extract transaction information, and obtain recommended payment method information based on the transaction information and a pre-built payment method recommendation model, where the payment method recommendation model is trained based on historical transaction information of sample users.

[0110] The message generation module 403 may be used to generate a payment message according to the recommended payment method information and the transaction information, and the payment message is used to initiate payment.

[0111] In some embodiments, the model processing module 402 can be specifically used to: identify a target object from image data, where the target object includes one or more of a graphic code, a merchant face image, a merchant name, a bill, and a price tag; and extract merchant information that matches the target object based on the target object.

[0112] In some embodiments, the model processing module 402 can be specifically used to: obtain transaction discount information associated with the transaction information based on the transaction information, and obtain discount combination information whose discount amount meets preset conditions based on the transaction discount information associated with the transaction information; obtain payment preference information associated with the transaction information based on the transaction information; generate a transaction state vector based on the discount combination information and the payment preference information; input the transaction state vector into the payment method recommendation model, generate a value parameter of the payment method based on the transaction state vector by using a neural network through the payment method recommendation model, and select N payment methods from a plurality of payment methods based on the value parameters to obtain recommended payment method information, where N is a positive integer.

[0113] In some examples, the transaction information includes user information and product information. The model processing module 402 can be specifically used to: obtain a user latent vector corresponding to the user information and a product latent vector corresponding to the product information, the user latent vector is used to represent the user's preference for the payment method, and the product latent vector is used to represent the preference for the payment method for each type of product purchased; perform an inner product operation on the user latent vector corresponding to the user information and the product latent vector corresponding to the product information to obtain the payment preference information associated with the transaction information.

[0114] In some examples, the model processing module 402 can be specifically used to: select the top N payment methods from a plurality of payment methods in order of value parameters from high to low to obtain recommended payment method information; or, select N payment methods from a plurality of payment methods in order of preset selection probabilities and value parameters to obtain recommended payment method information, where the selection probability is the probability that the selected payment method is the top N payment methods in order of value parameters from high to low.

[0115] In some embodiments, the message generation module 403 may be specifically used to generate a payment message according to the recommended payment method information and transaction information when the user identity is authenticated.

[0116] User identity authentication may include one or more of the following: obtaining the user's biometrics collected by the wearable device, and performing user identity authentication based on the biometrics and the user's registered biometric samples; obtaining device information of an electronic device that is in a communication connection with the wearable device, and performing user identity authentication based on the device information and pre-registered binding device information; obtaining the user's eye movements collected by the wearable device, and performing user identity authentication based on the eye movements and pre-set eye verification movements; obtaining the user's behavior information collected by the wearable device, and performing user identity authentication based on the behavior information and pre-obtained user behavior characteristic conditions.

[0117] In some embodiments, the visual payment apparatus 400 of the wearable device further includes a sending module, a receiving module and a binding module.

[0118] The sending module can be used to send a binding request to the card management platform, where the binding request includes user information.

[0119] The receiving module can be used to: when the user indicated by the user information is a registered user of the card management platform, receive a payment card list sent by the card management platform, the payment card list including card information of the payment card of the user indicated by the user information.

[0120] The binding module may be configured to: in response to a selection input for the payment card list, bind the payment card indicated by the selection input with the wearable device.

[0121] In some embodiments, the visual payment device 400 of the wearable device may further include a latent vector acquisition module. The latent vector acquisition module may be used to: generate a user's co-occurrence matrix based on the user's historical transaction information, the user's co-occurrence matrix is ​​used to characterize the user's preference for payment methods of different types of goods; decompose the user's co-occurrence matrix into a user latent vector matrix and a commodity latent vector matrix, the user latent vector matrix includes user latent vectors, and the commodity latent vector matrix includes commodity latent vectors.

[0122] In some examples, the latent vector acquisition module can be specifically used to: randomly generate a first latent vector matrix and a second latent vector matrix; perform gradient update on the first latent vector matrix and the second latent vector matrix according to the user's co-occurrence matrix, the first latent vector matrix, the second latent vector matrix and a preset learning rate parameter to obtain an updated latent vector matrix; obtain a loss function based on the user's co-occurrence matrix and the updated latent vector matrix; if the loss function satisfies the preset loss condition, determine the updated latent vector matrix as the user latent vector matrix and the product latent vector matrix; if the loss function does not meet the preset loss condition, perform gradient update on the updated latent vector matrix based on the user's co-occurrence matrix, the updated latent vector matrix and the preset learning rate parameter, until the loss function of the updated updated latent vector matrix meets the preset loss condition, and determine the updated updated latent vector matrix as the user latent vector matrix and the product latent vector matrix.

[0123] In some embodiments, the visual payment device 400 of the wearable device may include a model training module. The model training module may be used to: obtain transaction preferential information associated with the historical transaction information according to the historical transaction information of the sample user, and obtain preferential combination information according to the transaction preferential information associated with the historical transaction information; obtain payment preference information of the sample user according to the historical transaction information; generate a sample state vector according to the preferential combination information and the payment preference information of the sample user; train an initial recommendation model including a neural network based on the sample state vector so that the value parameter predicted by the value function of the neural network meets the value prediction condition, and obtain a payment method recommendation model.

[0124] In some examples, the model training module can be specifically used to: input the sample state vector into the neural network to obtain value parameters of multiple payment methods; update the network parameters of the neural network according to the gap between the value parameters and the measured value parameters, and randomly select sample data from the memory bank to train the initial recommendation model until the gap between the value parameters and the measured value parameters meets the value prediction conditions, thereby obtaining a payment method recommendation model, wherein the sample data in the memory bank includes paired sample state vectors and sample payment method information, and the sample payment method information is obtained according to the output of the initial recommendation model obtained by using the sample state vector as the input of the initial recommendation model during the training process, and the output of the initial recommendation model is related to a preset selection probability.

[0125] In some embodiments, the visual payment device 400 of the wearable device may further include an association update module. The association update module may be used to: obtain basic feature information of the user based on the historical transaction information of the user, cluster the user based on the basic feature information of the user, and obtain the location information of the user in the cluster space; construct a feature graph network of the user based on the payment preference information of the user; update the location information of the user in the cluster space using the feature graph network of the user based on the association relationship between the pre-learned feature graph network and the location relationship of the user in the cluster space; update the payment preference information of the user using the payment preference information of the neighboring users of the user in the cluster space, and the distance between the location of the neighboring users in the cluster space and the location of the user is within a preset distance range.

[0126] It should be noted that the visual payment device 400 based on a wearable device is a device corresponding to the above-mentioned visual payment method based on a wearable device. All implementation methods in the above-mentioned method embodiment are applicable to the embodiments of the device and can achieve the same technical effects, which will not be repeated here.

[0127] The third aspect of the present application also provides a visual payment device based on a wearable device. Fig.10 A schematic diagram of the structure of a visual payment device based on a wearable device provided in an embodiment of the present application, such as Fig.10 As shown, the wearable device-based visual payment device 500 includes a memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0128] In some examples, the processor 502 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0129] The memory 501 may include a read-only memory (ROM), a random access memory (RAM), a disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the visual payment method based on the wearable device in the embodiment of the present application.

[0130] The processor 502 runs a computer program corresponding to the executable program code by reading the executable program code stored in the memory 501, so as to implement the visual payment method based on the wearable device in the above embodiment.

[0131] In some examples, the wearable device-based visual payment device 500 may also include a communication interface 503 and a bus 504. Fig.10 As shown, the memory 501, the processor 502, and the communication interface 503 are connected via a bus 504 and communicate with each other.

[0132] The communication interface 503 is mainly used to realize the communication between the modules, devices, units and / or equipment in the embodiment of the present application. The communication interface 503 can also be used to access input devices and / or output devices.

[0133] The bus 504 includes hardware, software, or both, coupling the components of the wearable device-based visual payment device 500 to each other. By way of example and not limitation, the bus 504 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a Memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-E) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of the above. Where appropriate, the bus 504 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0134] A fourth aspect of the present application provides a visual payment system based on a wearable device. Fig.11A schematic diagram of the structure of a visual payment system for a wearable device provided in one embodiment of the present application, such as Fig.11 As shown, the visual payment system for wearable devices may include a wearable device 61 and a visual payment processing platform 62 .

[0135] The wearable device 61 can be used to collect multimedia data. The multimedia data includes image data, or includes the image data and voice data. The image data is obtained by the wearable device tracking the user's eye movement, determining the user's gaze area and photographing the user's gaze area.

[0136] The visual payment processing platform 62 can be communicatively connected with the wearable device 61 to execute the visual payment method based on wearable devices in the above embodiment, and can achieve the same technical effect. For specific contents, please refer to the relevant description in the above method embodiment, which will not be repeated here.

[0137] In a fifth aspect, the present application provides a computer-readable storage medium, on which a computer program instruction is stored. When the computer program instruction is executed by a processor, the visual payment method based on a wearable device in the above embodiment can be implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Among them, the above-mentioned computer-readable storage medium may include a non-transitory computer-readable storage medium, such as a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc., which is not limited here.

[0138] The sixth aspect of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the visual payment method based on wearable devices in the above-mentioned embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0139] It should be clear that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. For device embodiments, equipment embodiments, system embodiments, computer-readable storage medium embodiments, and computer program product embodiments, the relevant parts can refer to the description part of the method embodiment. The present application is not limited to the specific steps and structures described above and shown in the figures. Those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application. In addition, for the sake of simplicity, a detailed description of known method technologies is omitted here.

[0140] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (system) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.

[0141] Those skilled in the art should understand that the above embodiments are exemplary rather than restrictive. Different technical features appearing in different embodiments can be combined to achieve beneficial effects. Based on the study of the drawings, the specification and the claims, those skilled in the art should be able to understand and implement other variations of the disclosed embodiments. In the claims, the term "comprising" does not exclude other devices or steps; the quantifier "one" does not exclude multiple; the terms "first" and "second" are used to indicate names rather than to indicate any specific order. Any figure marks in the claims should not be understood as limiting the scope of protection. The functions of multiple parts appearing in the claims can be implemented by a single hardware or software module. The appearance of certain technical features in different dependent claims does not mean that these technical features cannot be combined to achieve beneficial effects.

Claims

1. A visual payment method based on wearable devices, characterized in that: include: Acquire multimedia data collected by a wearable device of a user, wherein the multimedia data includes image data, or includes the image data and voice data, wherein the image data is obtained by the wearable device tracking the eye movement of the user, determining the user's gaze area, and photographing the user's gaze area; Parsing the multimedia data to extract transaction information, and obtaining recommended payment method information based on the transaction information and a pre-built payment method recommendation model, wherein the payment method recommendation model is trained based on historical transaction information of sample users; A payment message is generated according to the recommended payment method information and the transaction information, and the payment message is used to initiate payment.

2. The method according to claim 1, characterized in that The transaction information includes merchant information; the step of parsing the multimedia data to extract the transaction information includes: Identify a target object from the image data, wherein the target object includes one or more of a graphic code, a merchant face image, a merchant name, a bill, and a price tag; According to the target object, the merchant information matching the target object is extracted.

3. The method according to claim 1, characterized in that The obtaining of the recommended payment method information based on the transaction information and the pre-built payment method recommendation model includes: According to the transaction information, obtain transaction preferential information associated with the transaction information, and according to the transaction preferential information associated with the transaction information, obtain preferential combination information whose preferential amount meets the preset conditions; Obtaining payment preference information associated with the transaction information according to the transaction information; generating a transaction state vector according to the preferential combination information and the payment preference information; The transaction state vector is input into the payment method recommendation model, and the payment method recommendation model uses a neural network to generate a value parameter of the payment method according to the transaction state vector. According to the value parameter, N payment methods are selected from a plurality of payment methods to obtain the recommended payment method information, where N is a positive integer.

4. The method according to claim 3, characterized in that The transaction information includes user information and product information; The obtaining, according to the transaction information, payment preference information associated with the transaction information includes: Obtain the user latent vector corresponding to the user information and the product latent vector corresponding to the product information. The user latent vector is used to represent the user's preference for the payment method, and the product latent vector is used to represent the preference for the payment method for each type of product purchased; An inner product operation is performed on the user latent vector corresponding to the user information and the commodity latent vector corresponding to the commodity information to obtain payment preference information associated with the transaction information.

5. The method according to claim 3, characterized in that: The selecting N payment methods from the multiple payment methods according to the value parameter to obtain the recommended payment method information includes: Selecting the top N payment methods from the multiple payment methods in descending order of the value parameters to obtain the recommended payment method information; or, According to the preset selection probability and the value parameter, N payment methods are selected from multiple payment methods to obtain the recommended payment method information, and the selection probability is the probability that the selected payment method is the top N payment methods with the value parameter from high to low.

6. The method according to claim 1, characterized in that The generating a payment message according to the recommended payment method information and the transaction information includes: In the case of passing the user identity authentication, generating the payment message according to the recommended payment method information and the transaction information; User authentication may include one or more of the following: Obtaining biometric features of the user collected by the wearable device, and performing user identity authentication based on the biometric features and a registered biometric sample of the user; Acquire device information of an electronic device that is in a communication connection state with the wearable device, and perform user identity authentication based on the device information and pre-registered binding device information; Acquire the eye movement of the user collected by the wearable device, and perform user identity authentication according to the eye movement and a preset eye verification action; The user's behavior information collected by the wearable device is obtained, and the user's identity verification is performed according to the behavior information and the user's behavior characteristic conditions obtained in advance.

7. The method according to claim 1, characterized in that Also includes: Sending a binding request to a card management platform, wherein the binding request includes user information; In a case where the user indicated by the user information is a registered user of the card management platform, receiving a payment card list sent by the card management platform, the payment card list including card information of the payment card of the user indicated by the user information; In response to a selection input for the payment card list, the payment card indicated by the selection input is bound to the wearable device.

8. The method according to claim 1, characterized in that Also includes: Generate a user's co-occurrence matrix based on the user's historical transaction information. The user's co-occurrence matrix is ​​used to characterize the user's preference for payment methods for different types of goods. The user co-occurrence matrix is ​​decomposed into a user latent vector matrix and a product latent vector matrix. The user latent vector matrix includes user latent vectors, and the product latent vector matrix includes product latent vectors.

9. The method according to claim 8, characterized in that The step of decomposing the user co-occurrence matrix into a user latent vector matrix and a product latent vector matrix includes: Randomly generate a first latent vector matrix and a second latent vector matrix; According to the user's co-occurrence matrix, the first latent vector matrix, the second latent vector matrix, and a preset learning rate parameter, gradient updating is performed on the first latent vector matrix and the second latent vector matrix to obtain an updated latent vector matrix; Obtaining a loss function based on the user's co-occurrence matrix and the updated latent vector matrix; If the loss function satisfies a preset loss condition, determining the updated latent vector matrix as a user latent vector matrix and a product latent vector matrix; If the loss function does not meet the preset loss condition, the updated latent vector matrix is ​​gradient updated based on the user's co-occurrence matrix, the updated latent vector matrix and a preset learning rate parameter until the loss function of the updated latent vector matrix meets the preset loss condition, and the updated latent vector matrix is ​​determined as the user latent vector matrix and the product latent vector matrix.

10. The method according to claim 1, characterized in that Before acquiring the multimedia data uploaded by the wearable device of the user, the method further includes: According to the historical transaction information of the sample user, obtaining transaction preferential information associated with the historical transaction information, and according to the transaction preferential information associated with the historical transaction information, obtaining preferential combination information; Obtaining payment preference information of sample users based on the historical transaction information; Generate a sample state vector according to the preferential combination information and the payment preference information of the sample user; The initial recommendation model including the neural network is trained based on the sample state vector so that the value parameter predicted by the value function of the neural network meets the value prediction condition, thereby obtaining the payment method recommendation model.

11. The method according to claim 10, characterized in that The initial recommendation model including the neural network is trained based on the sample state vector so that the value parameter predicted by the value function of the neural network meets the value prediction condition, and the payment method recommendation model is obtained, including: Input the sample state vector into the neural network to obtain the value parameters of multiple payment methods; According to the difference between the value parameter and the measured value parameter, the network parameters of the neural network are updated, and sample data are randomly selected from the memory bank to train the initial recommendation model until the difference between the value parameter and the measured value parameter meets the value prediction condition, thereby obtaining the payment method recommendation model. Among them, the sample data in the memory bank includes paired sample state vectors and sample payment method information, and the sample payment method information is obtained according to the output of the initial recommendation model obtained by using the sample state vector as the input of the initial recommendation model during the training process, and the output of the initial recommendation model is related to the preset selection probability.

12. The method according to claim 1, characterized in that Also includes: According to the historical transaction information of the user, the basic feature information of the user is obtained, and the user is clustered based on the basic feature information of the user to obtain the location information of the user in the cluster space; Build a user feature graph network based on the user's payment preference information; Based on the association between the pre-learned feature graph network and the position relationship of the user in the cluster space, the user's feature graph network is used to update the user's position information in the cluster space; The payment preference information of the user is updated by using the payment preference information of the neighboring users of the user in the cluster space, and the distance between the position of the neighboring users in the cluster space and the position of the user is within a preset distance range.

13. A visual payment device based on a wearable device, characterized in that: include: A data acquisition module, used to acquire multimedia data collected by a wearable device of a user, wherein the multimedia data includes image data, or includes the image data and voice data, wherein the image data is obtained by the wearable device tracking the eye movement of the user, determining the user's gaze area, and photographing the user's gaze area; A model processing module, configured to parse the multimedia data, extract transaction information, and obtain recommended payment method information based on the transaction information and a pre-built payment method recommendation model, wherein the payment method recommendation model is trained based on historical transaction information of sample users; The message generation module is used to generate a payment message according to the recommended payment method information and the transaction information, and the payment message is used to initiate payment.

14. A visual payment device based on a wearable device, characterized in that: include: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the wearable device-based visual payment method is implemented as described in any one of claims 1 to 12.

15. A visual payment system based on wearable devices, characterized in that: include: A wearable device for collecting multimedia data, wherein the multimedia data includes image data, or includes the image data and voice data, wherein the image data is obtained by the wearable device tracking the eye movement of the user, determining the user's gaze area, and photographing the user's gaze area; A visual payment processing platform, which is in communication with the wearable device, is used to execute the visual payment method based on the wearable device as described in any one of claims 1 to 12.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the wearable device-based visual payment method as described in any one of claims 1 to 12 is implemented.

17. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the visual payment method based on a wearable device as described in any one of claims 1 to 12.

Images