Shopping processing method, device, robot, storage medium, and program product

Abstract

This disclosure provides a shopping processing method, apparatus, robot, storage medium, and computer program product. The method includes: determining the inventory information of a first item in a target area based on first image information collected by a sensing device; if the inventory information is determined to be lower than an inventory threshold, determining the repurchase time interval of the first item based on the user's historical item purchase information; if the current time is determined to be within the repurchase time interval, generating a repurchase reminder message for the first item and performing reminder processing; and executing an item order payment operation based on user shopping instruction information collected by the sensing device. This disclosure can automatically monitor the inventory of items and trigger repurchase reminders, reducing the occurrence of item shortages and untimely replenishment; it can complete the entire shopping process through a robot, enabling convenient online shopping, reducing the user's learning cost and usage difficulty, and improving the efficiency and reliability of shopping execution.

Description

Technical Field

[0001] This application relates to the field of robotics, and more particularly to a shopping processing method, apparatus, robot, storage medium, and computer program product. Background Technology

[0002] With the development of e-commerce, online shopping has become an important way for users to consume daily. Users can use mobile phones, tablets, and other terminals to shop through e-commerce apps or websites. Currently, users need to actively check the inventory of goods to determine when to purchase them, which can easily lead to oversights or forgetfulness, resulting in untimely restocking and shortages. At the same time, users need to manually complete operations such as product selection and payment when shopping, relying on touch and text input, which presents a high learning curve and difficulty for some users. In addition, users cannot operate smart terminals to complete shopping while doing housework, cooking, or other activities. Summary of the Invention

[0003] This disclosure provides a shopping processing method, apparatus, robot, storage medium, and computer program product.

[0004] According to a first aspect of this disclosure, a shopping processing method is provided, applied to a robot, the method comprising: determining the inventory information of a first item in a target area based on first image information collected by a sensing device; if the inventory information is determined to be lower than an inventory threshold, determining a repurchase time interval for the first item based on the user's historical item purchase information; if the current time is determined to be within the repurchase time interval, generating a repurchase reminder message for the first item, and performing prompt processing based on the repurchase reminder message; and performing an item order payment operation based on user shopping instruction information collected by the sensing device.

[0005] Optionally, the step of executing the item order payment operation based on the user shopping instruction information collected by the sensing device includes: determining the target item that the user needs to purchase based on the user shopping instruction information; generating item order information corresponding to the target item; and executing the item order payment operation based on the item order information.

[0006] Optionally, the target item is the first item, and generating the item order information corresponding to the target item includes: determining the first item identifier information of the first item, the user's address information, and the payment amount information of the first item based on the historical item purchase information; generating the item order information, wherein the item order information includes the first item identifier information, the payment amount information, and the user's address information.

[0007] Optionally, the target item is a second item different from the first item, and generating the item order information corresponding to the target item includes: determining the second item identification information corresponding to the second item based on the user shopping instruction information; and generating the item order information based on the second item identification information.

[0008] Optionally, the user shopping instruction information includes voice instructions; determining the second item identification information corresponding to the second item based on the user shopping instruction information includes: converting the voice instructions to obtain corresponding text instructions; obtaining first keyword information based on the text instructions; matching the user's historical item purchase information with the first keyword information to obtain a first purchased item matching the second item; and determining the second item identification information based on the first purchased item.

[0009] Optionally, if the first purchased item cannot be obtained, at least one guiding content is determined based on the text instruction; information interaction is performed with the user based on the at least one guiding content; second keyword information is obtained based on supplementary information input by the user during the information interaction and the text instruction; the user's historical item purchase information and the second keyword information are matched to determine a second purchased item that matches the second item; and the second item identification information is determined based on the second purchased item.

[0010] Optionally, the user shopping instruction information includes voice instructions and second image information of the second item. The step of determining the second item identification information corresponding to the second item that the user needs to purchase includes: obtaining first item feature information based on the second image information; matching the first item feature information with preset item feature information in the item feature library to determine matching second item feature information; and determining the second item identification information based on the item corresponding to the matching second item feature information.

[0011] Optionally, generating the item order information corresponding to the second item based on the second item identification information includes: obtaining the basic item information of the second item based on the second item identification information and displaying it to the user, wherein the basic item information includes the item name, specifications, and inventory status; upon receiving the user's purchase confirmation instruction and the quantity confirmed by the user, determining the payment amount information based on the second item identification information and the purchase quantity; and generating the item order information, wherein the item order information includes the second item identification information, the payment amount information, and the user's address information.

[0012] Optionally, before performing the item order payment operation, the user's biometric information is collected, and verification is performed based on the biometric information; if the verification is successful, the item order payment operation is performed.

[0013] Optionally, the verification process based on the biometric information includes: obtaining a biometric sample corresponding to the user from a server device based on the user's user identifier and the robot's device identifier; and comparing the biometric information with the biometric sample for verification processing.

[0014] Optionally, an association is established between the user's identifier and the robot's device identifier; upon receiving a biometric recording instruction, the robot is controlled to guide the user to record a biometric sample; the user identifier, the device identifier, and the biometric sample are sent to a preset server device so that the server device establishes an association between the user identifier, the device identifier, and the biometric sample.

[0015] Optionally, the step of performing the item order payment operation based on the item order information includes: submitting the item order information to the payment system; and receiving the order payment operation result information returned by the payment system.

[0016] Optionally, the biometric information includes at least one of voiceprint information, facial information, and iris information.

[0017] Optionally, the robot includes a robot with mobility capabilities; the first image information is collected by the sensing device during the robot's movement within the target area.

[0018] According to a second aspect of this disclosure, a shopping processing apparatus is provided for use with a robot. The apparatus includes: an inventory determination module, configured to determine the inventory information of a first item in a target area based on first image information collected by a sensing device; a repurchase time determination module, configured to determine a repurchase time interval for the first item based on the user's historical item purchase information when the inventory information is determined to be below an inventory threshold; a prompt processing module, configured to generate a repurchase reminder message for the first item when the current time is determined to be within the repurchase time interval, and perform prompt processing based on the repurchase reminder message; and an order execution module, configured to execute an item order payment operation based on user shopping instruction information collected by the sensing device.

[0019] According to a third aspect of this disclosure, a shopping processing apparatus is provided for use in a robot, comprising: a memory; and a processor coupled to the memory, the processor being configured to perform the method as described above based on instructions stored in the memory.

[0020] According to a fourth aspect of this disclosure, a robot is provided, comprising: the shopping processing apparatus as described above.

[0021] According to a fifth aspect of this disclosure, a computer-readable storage medium is provided that stores computer instructions, which, when executed by a processor, implement the steps of the method as described above.

[0022] According to a sixth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of the method described above.

[0023] The shopping processing method, apparatus, robot, storage medium, and computer program products disclosed herein can automatically monitor the inventory of goods and trigger repurchase reminders, reducing the occurrence of shortages and untimely replenishment of goods; they can complete the entire shopping process through robots, enabling convenient online shopping, reducing the learning cost and difficulty of use for users, and improving the efficiency and reliability of shopping execution. Attached Figure Description

[0024] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The accompanying drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other objects and advantages of this disclosure will be further described below with reference to specific embodiments and the accompanying drawings. In the drawings, the same or corresponding technical features or components will be represented by the same or corresponding reference numerals.

[0025] Figure 1 This is a schematic flowchart of some embodiments of the shopping processing method according to the present disclosure;

[0026] Figure 2 This is a schematic diagram of the process of performing an order placement and payment operation for an item according to some embodiments of the shopping processing method of this disclosure;

[0027] Figure 3 This is a schematic diagram of the process for determining second item identification information in some embodiments of the shopping processing method according to this disclosure;

[0028] Figure 4This is a schematic diagram of a process for determining second item identification information by guidance in some embodiments of the shopping processing method according to this disclosure;

[0029] Figure 5 This is a schematic flowchart illustrating the determination of second item identification information in some other embodiments of the shopping processing method according to this disclosure;

[0030] Figure 6 This is a schematic diagram of the process for generating order information for a second item in some embodiments of the shopping processing method according to this disclosure;

[0031] Figure 7 This is a schematic diagram illustrating the process of establishing the association between user identifier, device identifier, and biometric sample in some embodiments of the shopping processing method according to this disclosure;

[0032] Figure 8A This is a schematic diagram of modules according to some embodiments of the shopping processing apparatus of this disclosure. Figure 8B Schematic diagrams of modules according to other embodiments of the shopping processing apparatus according to this disclosure;

[0033] Figure 9 This is a schematic diagram of modules according to some other embodiments of the shopping processing apparatus according to the present disclosure. Detailed Implementation

[0034] Exemplary embodiments of the present disclosure will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the embodiments are described in the specification. However, it should be understood that many implementation-specific settings must be made in carrying out the embodiments to achieve the developer's specific goals, such as complying with constraints related to the device and business, and these constraints may vary depending on the implementation. Furthermore, it should be understood that while development work can be very complex and time-consuming, such development work is merely a routine task for those skilled in the art who benefit from the present disclosure.

[0035] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0036] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0037] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0038] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0039] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0040] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0041] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0042] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0043] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0044] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0045] Furthermore, to avoid obscuring this disclosure with unnecessary detail, only processing steps and / or apparatus structures closely related to at least the solutions according to this disclosure are shown in the accompanying drawings, while other details less relevant to this disclosure are omitted. It should also be noted that similar reference numerals and letters in the drawings indicate similar items, and therefore once an item is defined in one drawing, it need not be discussed again in subsequent drawings.

[0046] The shopping processing method disclosed herein is applied to robots, which can be various types of robots such as service robots. This shopping processing method can be applied in various scenarios such as homes and elderly care facilities; the following explanation uses a home scenario as an example.

[0047] Figure 1 This is a schematic flowchart illustrating some embodiments of the shopping processing method according to this disclosure. Figure 1As shown, the shopping processing method includes steps S101-S104.

[0048] Step S101: Determine the quantity information of the first item in the target area based on the first image information collected by the sensing device.

[0049] Robots can be equipped with sensing devices, communication modules, displays, and other devices; sensing devices can include visual sensing components, voice acquisition modules, and other components; visual sensing components can include cameras, depth cameras, and various sensors.

[0050] The target area can be a pre-defined area within a home, such as the living room or kitchen. The robot can be any type of mobile robot; the initial image information is collected by the robot's sensing devices as it moves within the target area. The initial object can be a mobile phone, milk, clothing, or other similar items.

[0051] Step S102: If the stock information is determined to be lower than the stock threshold, determine the repurchase time range of the first item based on the user's historical item purchase information.

[0052] Step S103: If it is determined that the current time is within the repurchase time interval, generate a repurchase reminder message for the first item, and perform prompt processing based on the repurchase reminder message.

[0053] The robot can use the service processing model to achieve various functions such as image recognition and shopping intent understanding. The service processing model can be a pre-trained multimodal large language model or other models, and can be deployed locally on the robot or in the cloud.

[0054] The robot retrieves the user's historical purchase records from the server and uses a service processing model to determine the repurchase time range for the first item. For example, the robot determines that the user's repurchase time range for the first item is from the 1st to the 10th of a certain month.

[0055] When the current time is the 5th of a certain month, the robot determines that the current time falls within the repurchase time range, generates a repurchase reminder message for the first item, and performs the prompt processing through voice broadcast or display screen.

[0056] Step S104: Based on the user shopping instruction information collected by the sensing device, execute the item order payment operation.

[0057] Robots can collect users' voice information, or voice and image information, through sensing devices, and use service processing models to analyze the voice information or voice and image information collected by the sensing devices to determine whether the user has the intention to make a purchase.

[0058] Once the robot determines that the user intends to purchase, it uses voice information, or a combination of voice and image information, as the user's shopping instructions. The robot can then execute the order placement and payment process based on these instructions.

[0059] The shopping processing method disclosed herein determines the inventory of goods based on images collected by sensing devices. When the inventory is less than a preset inventory threshold, it determines a repurchase time range by combining the user's historical purchase data. Within the repurchase time range, it triggers a repurchase reminder and executes the order and payment operation based on the user's shopping instructions collected by the sensing devices. It can automatically monitor the inventory of goods and trigger repurchase reminders, reducing the occurrence of shortages and untimely replenishment of goods. It can complete the entire shopping process through a robot, enabling convenient online shopping, reducing the learning cost and difficulty of use for users, and improving the efficiency and reliability of shopping execution.

[0060] In some embodiments, when the robot performs routine inspections in target areas of a home (such as the kitchen, living room, etc.), it collects first image information through sensing devices. The robot can use a service processing model to identify, based on the first image information, that there is one pack of brand A tissues (the first item) remaining in the kitchen.

[0061] The robot determines that the stock of the first item is less than the preset stock threshold of 2 packs; using the service processing model, the robot determines the user's usage habits and consumption rate of brand A tissues based on the user's historical purchase information, and determines the repurchase time range of tissues to be the 2nd to 4th of a certain month.

[0062] Given that the current time is the 3rd of a certain month, and falls within the repurchase period (between the 2nd and 4th of the month), the robot generates a repurchase reminder message for brand A tissues. For example, the repurchase reminder message could be something like, "Your brand A tissues are almost empty. Should you place an order for brand A tissues?" The robot then processes the message, either by broadcasting the repurchase reminder to the user via voice or by displaying it on the screen, to prompt the user to initiate a repurchase.

[0063] After the robot processes the repurchase reminder message, the user can choose to repurchase the item upon hearing or seeing the reminder, or choose not to. The robot then executes the order and payment process based on the user's shopping instructions collected by the sensing devices.

[0064] Figure 2 This is a flowchart illustrating the process of placing an order and making payment for an item according to some embodiments of the shopping processing method of this disclosure, such as... Figure 2 As shown:

[0065] Step S201: Based on the user's shopping instruction information, determine the target items that the user needs to purchase.

[0066] For example, after hearing or seeing a repurchase reminder for the first item, a user decides to purchase the first item, or the user decides to purchase a second item that is different from the first item. The second item may be of the same category as the first item, but different in specifications, etc., or the second item may be completely different from the first item and completely unrelated.

[0067] The robot can use a service processing model to determine the target item that the user needs to buy based on the user's shopping instructions. The target item can be a first item or a second item that is different from the first item.

[0068] Step S202: Generate the item order information corresponding to the target item.

[0069] The order information can include the item's identifier, payment amount, user's address, and other information.

[0070] Step S203: Based on the item order information, perform the item order payment operation.

[0071] The robot can call the settlement interface to submit the order information to the payment system; the payment system will synchronize the payment result to the robot, and the robot will announce the payment result information through voice and / or display it on the screen.

[0072] The payment system can be various, including e-commerce payment systems. The robot can submit order information to the payment system through multiple methods and receive the order payment result information returned by the system. This result information includes whether the payment was successful or failed. If the payment fails, the failure information includes the reason for the failure, such as insufficient balance or network error. The robot can announce the order payment result information via voice and / or display it on a screen.

[0073] In some embodiments, the target item is a first item. Based on historical item purchase information, the robot determines the first item's identifier, the user's address information, and the payment amount for the first item. The robot then generates item order information, which includes the first item's identifier, payment amount, and user's address information. The first item's identifier is a code or identifier that can identify the first item, such as the first item's SKU (Stock Keeping Unit) or item ID.

[0074] For example, the robot might play a voice message reminding the user to repurchase brand A tissues (the first item): "Your brand A tissues are almost empty. Would you like to order more brand A tissues?" After hearing this reminder, the user decides to repurchase brand A tissues and gives the voice order: "Buy brand A tissues for me." Based on this order, the robot determines that the user's actual target item is brand A tissues.

[0075] Based on the user's historical purchase information, the robot retrieves the first item identifier information (SKU 1) for brand A tissues, along with the user's address information. Using a service processing model, the robot can then determine the quantity of tissues A the user will purchase and the payment amount based on their historical purchase information.

[0076] The robot generates order information for the item, including SKU 1, payment amount, and user address. The robot can then call the settlement interface to submit the order information to the payment system for payment, automatically completing a repeat purchase of brand A tissues.

[0077] In some embodiments, the target item is a second item different from the first item. The robot determines the second item identification information corresponding to the second item based on the user's shopping instruction information. The second item identification information can be a code or identifier that can identify the second item. For example, the second item identification information can be the SKU, item ID, etc. of the second item. The robot generates item order information based on the second item identification information.

[0078] For example, the robot can play a repurchase reminder message for brand A tissues (the first item) via voice: "Your brand A tissues are almost empty. Would you like to place an order for brand A tissues?"

[0079] After hearing the repurchase reminder, the user doesn't want to repurchase brand A tissues but instead wants to buy brand B tissues. The user gives the shopping command via voice: "Buy brand B tissues for me." Based on this command, the robot determines that the user's actual target item is brand B tissues (the second item), which is different from brand A tissues (the first item).

[0080] Based on the user's shopping instructions, the robot can use various methods to determine the SKU 2 (second item identification information) corresponding to brand B's tissue paper (second item); based on SKU 2, the robot can use various methods to generate item order information.

[0081] Figure 3 This is a flowchart illustrating the determination of second item identification information in some embodiments of the shopping processing method according to this disclosure. The user shopping instruction information includes voice commands, such as... Figure 3As shown:

[0082] Step S301: Convert the voice command to obtain the corresponding text command.

[0083] There are various methods for converting voice commands. For example, robots can use technologies such as ASR (Automatic Speech Recognition) to convert voice commands into text commands.

[0084] Step S302: Obtain the first keyword information according to the text instruction.

[0085] There are several methods to obtain primary keyword information. For example, a robot can use a service processing model to process text commands and obtain primary keyword information, which may include information such as item name, brand, specifications, and category.

[0086] Step S303: Match the user's historical purchase information with the first keyword information to obtain the first purchased item that matches the second item.

[0087] There are several methods to obtain the first purchased item. For example, the robot can use a cloud-based order search interface to obtain the user's historical purchase information, such as historical orders. Historical orders can include information such as item SKU, name, specifications, purchase time, purchase quantity, order history, and shipping address.

[0088] The robot can use a service processing model to match historical item purchase information with the first keyword information to obtain the first purchased item that matches the second item.

[0089] The robot can also send the first keyword information to the cloud device, which will then match the user's historical purchase information with the first keyword information to obtain the first purchased item that matches the second item; the robot then receives the first purchased item information returned by the cloud device.

[0090] Step S304: Determine the identification information of the second item based on the first purchased item.

[0091] For example, the second item identification information of the first purchased item can be used as the second item identification information of the second item.

[0092] In some embodiments, user A inputs shopping instructions to the robot, including the voice command "Buy me tissues from brand B".

[0093] The robot uses ASR technology to convert the voice command "Buy me brand B tissues" into the text command "Buy me brand B tissues". The robot then uses a service processing model to determine that the first key information in the text command includes "brand B" and "tissues".

[0094] The robot can use the cloud-based order search interface to retrieve the user's historical item orders in reverse chronological order. It can then use a service processing model to match the historical item orders with the primary keyword information "Brand B" and "tissue paper" to obtain the matching degree between the primary keyword information and the historical item orders.

[0095] If the match between the first keyword information and one or more historical item orders is greater than a preset match threshold, these one or more historical item orders are identified as target orders. The items corresponding to these one or more target orders are used as the first purchased items to match the second item. The quantity of the first purchased items can be one or more. For example, if the first purchased item is identified as "S-code brand B tissue paper", the SKU of "S-code brand B tissue paper" is used as the second item identification information for brand B tissue paper.

[0096] Figure 4 This is a schematic diagram of a process for determining second item identification information by guidance in some embodiments of the shopping processing method according to this disclosure, such as... Figure 4 As shown:

[0097] Step S401: If the first purchased item cannot be obtained, at least one guiding content is determined according to the text instruction.

[0098] For example, if the match between the first keyword information and all of the user's historical item orders is less than a preset match threshold, the robot determines that it has failed to obtain the first purchased item and determines at least one guiding content based on the text instruction.

[0099] Step S402: Interact with the user based on at least one guiding message.

[0100] Step S403: Obtain the second keyword information based on the supplementary information and text instructions entered by the user during the information interaction.

[0101] Step S404: Match the user's historical purchase information and second keyword information to determine the second purchased item that matches the second item.

[0102] Step S405: Determine the identification information of the second item based on the second purchased item.

[0103] In some embodiments, when user A issues the voice command "Buy me tissues," the robot determines, based on this shopping command, that the target item the user actually wants to purchase is tissues (second item) that are different from brand A's tissues (first item). The robot converts the voice command "Buy me tissues" into the corresponding text command "Buy me tissues"; the robot determines the first keyword information, which includes "tissues."

[0104] The robot uses a service processing model to match the first keyword information with the user's historical purchase information. Since the historical purchase information contains records related to multiple brands and specifications of tissues, the matching degree between the first keyword information "tissues" and all of the user's historical order information is less than the matching degree threshold, so it is impossible to determine the first purchased item corresponding to the second item needed by the user.

[0105] The robot uses a service processing model to generate guiding content based on text commands: "Which brand of tissues would you like to buy?", and interacts with the user via voice. The user then inputs supplementary information, such as "Brand A," via voice, following the guiding content.

[0106] The robot uses a service processing model, combined with text instructions and user-supplemented information "brand A", to retrieve the second keyword information, including "brand A" and "tissue", and then matches the second keyword information with the user's historical purchase information again.

[0107] If the match between the second keyword information and one or more historical item orders is greater than a match threshold, these one or more historical item orders are identified as target orders, and the items corresponding to these one or more target orders are designated as the second purchased items that match the second item. For example, if the second purchased item is "M-size brand A tissues", the SKU of "M-size brand A tissues" is used as the item identifier information of the second item.

[0108] By interacting with users, we can clarify their needs and reconstruct keywords, thereby improving the success rate of item matching under vague voice commands, increasing the error tolerance of voice command recognition, solving the problem of item matching failure caused by vague user voice descriptions, and avoiding ordering errors caused by matching mistakes.

[0109] Figure 5 This is a flowchart illustrating the determination of second item identification information in some other embodiments of the shopping processing method according to this disclosure. The user shopping instruction information includes voice commands and second image information of the second item, such as... Figure 5 As shown:

[0110] Step S501: Obtain the first item feature information based on the second image information.

[0111] The robot can use a service processing model to analyze the second image information and obtain the first item feature information; the first item feature information may include information such as appearance shape, color, size, text markings, and packaging patterns.

[0112] Step S502: Match the first item feature information with the preset item feature information in the item feature database to determine the matching second item feature information.

[0113] Step S503: Determine the second item identification information based on the item corresponding to the matched second item feature information.

[0114] In some embodiments, after hearing or seeing the shopping reminder message "Your brand A tissues are almost empty, do you need to order more brand A tissues?", user A does not want to repurchase brand A tissues, but instead wants to buy another brand B tissues. The user gives the shopping instruction "Buy brand B tissues for me" via voice, and at the same time shows a physical tissue to the robot.

[0115] Based on this shopping instruction, the robot determines that the user actually wants to purchase brand B tissues (second item), which are different from brand A tissues (first item). If the user's shopping instruction information includes second image information of brand B tissues (second item), the robot determines the second item identification information of brand B tissues (second item) based on the second image information.

[0116] The robot can use a service processing model to perform target detection and visual feature extraction on the second image information to obtain the feature information of the first item. The feature information of the first item includes the appearance, shape, color, size, text markings, packaging patterns and other features of the tissue paper of brand B (the second item).

[0117] The robot can use various methods to perform similarity matching calculations between the feature information of the first item and the feature information of the second item of the third item pre-stored in the cloud item feature database, and calculate the similarity between the feature information of the first item and each feature information of the second item.

[0118] For example, the robot obtains the second item feature information pre-stored in the item feature database in the cloud, and uses a service processing model to determine the similarity between the first item feature information and each of the second item feature information; or, the robot sends the first item feature information to the cloud server, and the cloud server calculates the similarity between the first item feature information and each of the second item feature information.

[0119] The second item feature information that matches the first item feature information is the second item feature information that has the highest similarity to the first item feature information and has a similarity greater than a similarity threshold. The number of third items that match the second item can be one or more.

[0120] For example, if the third item corresponding to the second item feature information that matches the second keyword information is "M code brand B tissue paper", then the SKU code corresponding to the third item "M code brand B tissue paper" is used as the second item identification information of the second item.

[0121] The robot extracts product feature information from image information and matches it with the product feature information pre-stored in the product feature database. It can accurately determine the second product that the user needs to buy and the identification information of the second product. It can effectively solve the problem that the user cannot accurately describe the product name and specifications, and improve the accuracy of product recognition and matching efficiency.

[0122] Figure 6 This is a schematic diagram illustrating the process of generating order information corresponding to a second item in some embodiments of the shopping processing method according to this disclosure, such as... Figure 6 As shown:

[0123] Step S601: Based on the second item identification information, obtain the basic information of the second item and display it to the user.

[0124] Basic item information includes item name, specifications, inventory status, etc. For example, a robot can obtain basic item information for a second item through a remote interface of the item platform, based on the second item's identifier information. There can be one or more second item identifiers. The robot can display the basic item information on a screen and / or read it aloud via voice.

[0125] Step S602: Upon receiving the user's purchase confirmation instruction and the quantity confirmed by the user, determine the payment amount information based on the second item identification information and the quantity purchased.

[0126] The user's purchase confirmation instruction can be a voice confirmation instruction or a text instruction entered by the user through the display screen. The user's purchase confirmation instruction is used to confirm the purchase of an item corresponding to one or more second item identification information.

[0127] Users can specify the purchase quantity via voice or by inputting it on the display screen. The confirmed purchase quantity can be the quantity of items corresponding to one or more second item identifiers. The robot can use various price calculation rules to calculate the payment amount based on the second item identifier and the purchase quantity.

[0128] Step S603: Generate item order information, which includes second item identification information, payment amount information, and user address information.

[0129] In some embodiments, the robot can obtain the basic information of the "S-code brand A tissue" (second item) by calling the remote interface of the item platform based on the SKU (second item identification information) of the "S-code brand A tissue" (second item). The basic information of the item includes the item name, specifications, inventory status and other information of the "S-code brand A tissue" (second item). The specifications may include information such as capacity, size and weight.

[0130] The robot can display basic information about items to users through screen display and voice broadcast; users can issue purchase confirmation commands via voice and indicate that the purchase quantity is 2 items.

[0131] After receiving the purchase confirmation instruction and the purchase quantity, the robot, based on the SKU and the purchase quantity of 2 items, calls the remote interface of the price platform and combines rules such as promotional discounts and membership benefits to determine the payment amount for 2 items of "S-code brand A tissues". The robot calls the remote interface of the address platform to obtain the user's preset default delivery address, or the robot obtains the user's address information based on the user's historical purchase information.

[0132] The robot generates item order information that includes "SKU (second item identifier) ​​of brand A tissues with S code", payment amount information, user address information, etc., for payment and order processing.

[0133] By acquiring and displaying basic item information through the second item identification information, the system calculates the transaction price after receiving the user's purchase confirmation instruction and purchase quantity, and generates item order information by combining user address information and other information. This can automatically complete the integration of item order information, improve the convenience and accuracy of item ordering, and enhance the stability and reliability of payment and settlement.

[0134] In some embodiments, before executing the item order payment operation, the robot collects the user's biometric information and performs verification processing based on the biometric information; if the verification is successful, the robot executes the item order payment operation based on the item order information.

[0135] Biometric information can include at least one of voiceprint, facial, and iris features. This biometric information can be configured according to the robot's specifications and user needs, and is compatible with various biometric recognition technologies. The robot can collect user voice data and extract voiceprint features; it can collect user facial images and extract facial features; and it can collect user iris images and extract iris features.

[0136] The robot can perform identity verification and other verification processes based on biometric information. After successful verification, the robot can call the settlement interface to submit the order information to the payment system. The payment system will then synchronize the payment result to the robot, which will announce the payment result information via voice and / or display it on the screen.

[0137] In some embodiments, the robot obtains a biometric sample corresponding to the user from the server device based on the user's user identifier and the robot's device identifier; the robot compares the biometric information with the biometric sample for verification.

[0138] For example, a user's identifier can be a user ID, and a robot's device identifier can be a robot ID. A robot can obtain both the user's identifier and the robot's device identifier through various methods. The server-side device can be a server-side authentication device, etc.

[0139] The robot can collect users' voice data, images, and other information in real time to extract biometric information. The robot sends the user ID and robot ID to the server-side authentication device. Based on the user ID and robot ID, the server-side authentication device obtains biometric samples that are associated with the robot and the user. These biometric samples can include voiceprint features and other information.

[0140] The robot receives biometric samples from the server-side authentication device and compares the collected biometric information with the obtained biometric samples for verification. Verification is successful when the similarity between the biometric information and the biometric sample reaches a preset similarity threshold; otherwise, verification fails, and the robot displays the message "Authentication failed, please confirm payment again."

[0141] The robot can also send its user ID, robot ID, and biometric information to the server-side authentication device. The server-side authentication device retrieves the biometric information associated with the robot based on the user ID and robot ID. It then compares this biometric information with a biometric sample for verification. Verification is successful when the similarity between the biometric information and the biometric sample reaches a preset similarity threshold; otherwise, verification fails. The server-side authentication device then sends the verification result to the robot.

[0142] By comparing the user's biometric information with preset biometric samples for verification, it is possible to effectively prevent others from impersonating the user to make purchases through the robot, thereby improving the security and reliability of shopping.

[0143] In some embodiments, the robot can perform biometric verification before executing the order and payment operation. If the verification is successful, the robot will execute the order and payment operation. If the verification fails, the robot will prompt the user with the reason for the failure, such as failure to match biometric information or failure to enter biometric information.

[0144] If the robot determines during the shopping process that the user has not established a correlation between the user identifier, device identifier, and biometric sample in the server-side device, it will guide the user to complete the biometric information recording to establish a correlation between the user identifier, device identifier, and biometric sample.

[0145] Figure 7 This is a schematic diagram illustrating the process of establishing the association between user identifiers, device identifiers, and biometric samples in some embodiments of the shopping processing method according to this disclosure, such as... Figure 7 As shown:

[0146] Step S701: Establish the association between the user's user identifier and the robot's device identifier.

[0147] Step S702: Upon receiving a biometric recording instruction, control the robot to guide the user to record biometric samples.

[0148] Step S703: Send the user identifier, device identifier, and biometric sample to the preset server device so that the server device can establish an association between the user identifier, device identifier, and biometric sample.

[0149] In some embodiments, users can establish a link between their user identifier, device identifier, and biometric sample before placing an order and making payment. For example, this link can be established at any stage, such as before the first use of the robot for shopping, before determining item identifier information, before generating item order information, or before placing an order and making payment.

[0150] For example, users can complete voiceprint authentication before their first shopping experience with the robot. Users access the settings page through the robot's app installed on their phone or tablet and submit their user ID (user identifier). Upon receiving the user ID, the robot generates an encrypted token and establishes an association between the user ID and its own device identifier, encrypting the association using the encrypted token.

[0151] After the user confirms the binding, the server-side device (server-side authentication device, which can be deployed in the cloud) sends a biometric recording instruction to the target robot through an encrypted channel. Upon receiving the biometric recording instruction, the robot performs visual positioning, identifies the user's location, and autonomously moves in front of the user, guiding the user to record biometric samples.

[0152] Various methods can be used to guide users to input biometric samples. For example, the robot displays information on the screen and prompts the user to read it aloud; the robot then collects the audio information of the user's reading and generates the user's voiceprint sample.

[0153] The robot sends the user identifier, device identifier, and biometric sample to the preset server authentication device, which then establishes the association between the user identifier, device identifier, and biometric sample.

[0154] By establishing a binding relationship between user identifiers, robot device identifiers, and biometric samples, the system can effectively improve the security and reliability of shopping by binding users, robots, and biometric samples together.

[0155] In some embodiments, before user A makes a purchase using the robot for the first time, the server-side authentication device establishes an association between user A's user ID, robot ID, and voiceprint feature sample.

[0156] The robot plays a repurchase reminder message for brand A tissues (the first item) via voice: "Your brand A tissues are almost empty. Would you like to place an order for brand A tissues?" After hearing this repurchase reminder message, the user who wants to repurchase brand A tissues gives the shopping command via voice: "Buy brand A tissues for me."

[0157] Based on this voice command, the robot determines that user A is repurchasing brand A tissues. Using historical purchase information, the robot identifies the first item identifier for brand A tissues, the user's address, and the payment amount. The robot then generates order information, which includes the first item identifier, payment amount, and user's address.

[0158] The robot sends a request to the server-side authentication device based on user A's user ID and robot ID to obtain pre-stored voiceprint feature samples. The robot collects user A's current voiceprint feature information and uses the service processing model to compare the voiceprint feature information with the voiceprint feature samples for verification processing.

[0159] After verification, the robot submits the order information to the payment system, receives the order payment result information returned by the payment system, and displays it to user A.

[0160] During the shopping process, if the robot determines that the server-side authentication device has not established a relationship between user A's user ID, robot ID, and voiceprint feature sample, it will prompt user A to establish this relationship.

[0161] Once the user's biometric information is successfully verified, the order and payment process is executed based on the order information. The entire shopping process can be completed by the robot, making online shopping convenient, reducing the learning cost and difficulty of use for users, improving the efficiency and reliability of shopping, and enhancing the security of shopping through biometric verification. It can also monitor the inventory of items, predict the repurchase time based on historical purchase data, and proactively remind users to repurchase, thus improving the intelligence of item management.

[0162] In some embodiments, such as Figure 8A As shown, this disclosure provides a shopping processing device for use with a robot. The shopping processing device includes: an inventory determination module 801, a repurchase time determination module 802, a prompt processing module 803, and an order execution module 804.

[0163] The inventory determination module 801 determines the inventory information of the first item in the target area based on the first image information collected by the sensing device. The repurchase time determination module 802, if the inventory information is determined to be below the inventory threshold, determines the repurchase time range of the first item based on the user's historical item purchase information.

[0164] When the prompt processing module 803 determines that the current time falls within the repurchase time range, it generates a repurchase reminder message for the first item and performs prompt processing based on the message. The order execution module 804 executes the order and payment operation based on the user's shopping instruction information collected by the sensing device.

[0165] In some embodiments, such as Figure 8B As shown, the order execution module 804 includes an information determination module 8041, an order determination module 8042, an order execution module 8043, a verification processing module 8044, and an association construction module 8045.

[0166] The information confirmation module 8041 determines the target item the user needs to purchase based on the user's shopping instruction information. The order confirmation module 8042 generates the item order information corresponding to the target item. The order execution module 8043 executes the item order payment operation based on the item order information.

[0167] In some embodiments, the target item is a first item. The order confirmation module 8042 determines the first item identification information, the user's address information, and the payment amount information of the first item based on historical item purchase information. The order confirmation module 8042 generates item order information, wherein the item order information includes the first item identification information, the payment amount information, and the user's address information.

[0168] In some embodiments, the target item is a second item different from the first item. The order confirmation module 8042 determines the second item identification information corresponding to the second item based on the user's shopping instruction information; the order confirmation module 8042 generates item order information based on the second item identification information.

[0169] User shopping instructions include voice instructions. The order confirmation module 8042 converts the voice instructions into text instructions. Based on the text instructions, the order confirmation module 8042 obtains the first keyword information.

[0170] The order confirmation module 8042 matches the user's historical purchase information with the first keyword information to obtain the first purchased item that matches the second item. Based on the first purchased item, the order confirmation module 8042 determines the identification information of the second item.

[0171] If the first purchased item is not obtained, the order confirmation module 8042 determines at least one guiding content based on the text instruction; the order confirmation module 8042 interacts with the user based on the at least one guiding content; the order confirmation module 8042 obtains the second keyword information based on the supplementary information and text instruction entered by the user during the information interaction.

[0172] The order confirmation module 8042 matches the user's historical purchase information and second keyword information to determine the second purchased item that matches the second item; the order confirmation module 8042 determines the second item identification information based on the second purchased item.

[0173] In some embodiments, the user shopping instruction information includes voice instructions and second image information of the second item. The order confirmation module 8042 obtains item feature information based on the second image information. The order confirmation module 8042 matches the item feature information with third item feature information in the item feature library to determine the matched third item feature information. The order confirmation module 8042 determines the second item identification information based on the item corresponding to the matched second item feature information.

[0174] The order confirmation module 8042 retrieves the basic information of the second item based on the second item identification information and displays it to the user. Upon receiving the user's purchase confirmation instruction and the confirmed purchase quantity, the order confirmation module 8042 determines the payment amount based on the second item identification information and the purchase quantity. The order confirmation module 8042 generates item order information, which includes the second item identification information, payment amount information, and the user's address information.

[0175] The order execution module 8043 submits the item order information to the payment system; the order execution module 8043 receives the order payment operation result information returned by the payment system.

[0176] In some embodiments, the verification processing module 8044 collects the user's biometric information and performs verification processing based on the biometric information before executing the item order payment operation. If the verification is successful, the order execution module 8043 executes the item order payment operation according to the item order information.

[0177] The verification processing module 8044 obtains the biometric sample corresponding to the user from the server device based on the user's user identifier and the robot's device identifier; the verification processing module 8044 compares the biometric information with the biometric sample to perform verification processing.

[0178] Before determining the second item identification information, the verification processing module 8044 performs verification processing based on biometric information; if the verification is successful, the information determination module 8041 determines the second item identification information based on the user's shopping instruction information.

[0179] In some embodiments, the association construction module 8045 constructs an association between the user's user identifier and the robot's device identifier; upon receiving a biometric recording instruction, the association construction module 8045 controls the robot to guide the user to record a biometric sample; the association construction module 8045 sends the user identifier, device identifier, and biometric sample to a preset server device so that the server device establishes an association between the user identifier, device identifier, and biometric sample.

[0180] like Figure 9 As shown, the shopping processing apparatus may include a memory 901, a processor 902, a communication interface 903, and a bus 904. The memory 901 is used to store instructions, and the processor 902 is coupled to the memory 901. The processor 902 is configured to execute the shopping processing method described above based on the instructions stored in the memory 901.

[0181] The memory 901 can be a high-speed RAM, non-volatile memory, or a memory array. The memory 901 may also be divided into blocks, and these blocks can be combined into virtual volumes according to certain rules. The processor 902 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the shopping processing method of this disclosure.

[0182] In some embodiments, this disclosure provides a robot including the shopping processing apparatus as described in any of the above embodiments. The robot can be a service robot or other types of robots.

[0183] In some embodiments, this disclosure provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the shopping processing method as described in any of the foregoing embodiments.

[0184] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples (not an exhaustive list) of readable storage media may include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0185] Embodiments of this disclosure may also be computer program products, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0186] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0187] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0188] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0189] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0190] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0191] The above description has been given for illustrative and descriptive purposes. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although several exemplary aspects and embodiments have been discussed above, those skilled in the art will understand that the above embodiments are illustrative only and do not limit the scope of this disclosure. Those skilled in the art will understand that the above embodiments can be combined, modified, or replaced without departing from the scope and spirit of this disclosure.

Claims

1. A shopping processing method applied to a robot, the method comprising: Based on the first image information collected by the sensing device, determine the quantity information of the first item in the target area; If the stock information is determined to be below the stock threshold, the repurchase time range of the first item is determined based on the user's historical item purchase information. If it is determined that the current time is within the repurchase time interval, a repurchase reminder message for the first item is generated, and a prompt processing is performed based on the repurchase reminder message; Based on the user's shopping instruction information collected by the sensing device, the item order and payment operation is executed.

2. The method according to claim 1, wherein, The step of executing the item order and payment operation based on the user shopping instruction information collected by the sensing device includes: Based on the user's shopping instruction information, determine the target items the user needs to purchase; Generate the item order information corresponding to the target item; Based on the order information, execute the order payment operation for the item.

3. The method according to claim 2, wherein, The target item is the first item, and generating the item order information corresponding to the target item includes: Based on the historical purchase information, determine the first item identifier information of the first item, the user's address information, and the payment amount information of the first item; Generate the item order information, wherein the item order information includes the first item identification information, the payment amount information, and the user's address information.

4. The method according to claim 2, wherein, The target item is a second item different from the first item, and the generation of item order information corresponding to the target item includes: Based on the user's shopping instruction information, determine the second item identification information corresponding to the second item; Based on the second item identification information, the item order information is generated.

5. The method according to claim 4, wherein, The user shopping instruction information includes voice commands; determining the second item identifier information corresponding to the second item based on the user shopping instruction information includes: The voice command is converted to obtain the corresponding text command; Based on the text instructions, obtain the first keyword information; The user's historical purchase information and the first keyword information are matched to obtain the first purchased item that matches the second item; Based on the first purchased item, determine the identification information of the second item.

6. The method according to claim 5, further comprising: If the first purchased item cannot be obtained, at least one guiding content is determined according to the text instructions; Based on the at least one guiding content, information interaction is conducted with the user; Based on the supplementary information input by the user during the information interaction and the text instructions, obtain the second keyword information; The user's historical purchase information and the second keyword information are matched to determine the second purchased item that matches the second item; Based on the second purchased item, determine the identification information of the second item.

7. The method according to claim 4, wherein, The user shopping instruction information includes voice commands and second image information of the second item, and the determination of the second item identification information corresponding to the second item includes: Based on the second image information, obtain the first item feature information; The first item feature information is matched with the preset item feature information in the item feature database to determine the matching second item feature information; The second item identification information is determined based on the item corresponding to the matched second item feature information.

8. The method according to claim 4, wherein, The step of generating the item order information based on the second item identification information includes: Based on the second item identification information, the basic information of the second item is obtained and displayed to the user, wherein the basic information of the item includes the item name, specifications, and inventory status; Upon receiving the user's purchase confirmation instruction and the quantity confirmed by the user, the payment amount information is determined based on the second item identification information and the purchase quantity; Generate the item order information, wherein the item order information includes the second item identification information, the payment amount information, and the user's address information.

9. The method according to claim 1, further comprising: Before executing the order payment operation for the item, the user's biometric information is collected, and verification is performed based on the biometric information; If the verification is successful, proceed with the order and payment for the item.

10. The method according to claim 9, wherein, The verification process based on the biometric information includes: Based on the user's user identifier and the robot's device identifier, obtain the biometric sample corresponding to the user from the server device; The biometric information is compared with the biometric sample for verification.

11. The method of claim 9, further comprising: Establish the association between the user's user identifier and the robot's device identifier; Upon receiving a biometric recording instruction, the robot is controlled to guide the user in recording biometric samples; The user identifier, the device identifier, and the biometric sample are sent to a preset server device so that the server device can establish an association between the user identifier, the device identifier, and the biometric sample.

12. The method according to claim 9, wherein, The biometric information includes at least one of voiceprint information, facial information, and iris information.

13. The method according to claim 2, wherein, The step of performing the item order payment operation based on the item order information includes: Submit the order information for the items to the payment system; Receive the order payment operation result information returned by the payment system.

14. The method according to any one of claims 1 to 13, wherein, The robots include robots with mobility capabilities; The first image information was collected by the sensing device during the robot's movement within the target area.

15. A shopping processing device for use with a robot, the device comprising: The inventory determination module is used to determine the inventory information of the first item in the target area based on the first image information collected by the sensing device. The repurchase time determination module is used to determine the repurchase time range of the first item based on the user's historical item purchase information when the stock information is determined to be lower than the stock threshold. The prompt processing module is used to generate a repurchase reminder message for the first item when it is determined that the current time is within the repurchase time interval, and to perform prompt processing according to the repurchase reminder message; The order execution module is used to execute the order and payment operation based on the user shopping instruction information collected by the sensing device.

16. A shopping processing device, applied to a robot, comprising: Memory; and a processor coupled to the memory, the processor being configured to perform the method according to any one of claims 1 to 14 based on instructions stored in the memory.

17. A robot comprising: The shopping processing apparatus according to claim 15 or 16.

18. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 14.

19. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 14.

Images