Commodity settlement processing method and system, and computer program product

Abstract

The embodiment of the invention discloses a commodity settlement processing method and system and a computer program product. The commodity settlement processing method is applied to intelligent glasses provided by a physical store, after a consumer user completes commodity purchasing, the consumer user wears the intelligent glasses to perform commodity settlement processing on a purchased commodity, and the commodity settlement processing method comprises the steps of collecting a target image in a view range of the intelligent glasses in a process that the consumer user takes the purchased commodity to perform self-service settlement; if the target image comprises the first image content of holding the at least one commodity, commodity feature information of the at least one commodity is obtained from the target image through image analysis; and carrying out commodity identification according to the commodity feature information, adding at least one commodity which is successfully identified as a to-be-settled commodity into a to-be-settled commodity set until a settlement request submitted by the consumer user through the intelligent glasses is obtained, and carrying out settlement processing on the to-be-settled commodity. According to the scheme, the overall settlement efficiency of the offline physical store can be further optimized.

Description

Technical Field

[0001] The present application relates to the technical field of commodity information services, and in particular to a commodity settlement processing method and system, a computer-readable storage medium, an electronic device, and a computer program product. Background Art

[0002] In order to improve the shopping checkout efficiency of offline stores, specific self-service checkout areas are usually divided in the stores, and multiple POS machines are installed for consumers to scan and check out the products themselves.

[0003] Specifically, after purchasing items, users can go to the self-service checkout area and scan the product barcode with a POS machine to identify the item and complete the purchase. Compared to manual checkout methods that require cashier assistance to scan barcodes, this self-service checkout method can effectively improve store checkout efficiency. However, in actual application, it also has some drawbacks that affect the further optimization and improvement of store checkout efficiency.

[0004] For example, a single scan operation can only identify one product and complete the scan-to-purchase of a single product. The user is required to turn the product over before scanning, find the barcode printing area on the product packaging, and expose the product barcode printed in this area as fully as possible within the scanning range of the POS terminal. If the barcode printing area is damaged or the product barcode is not fully exposed due to wrinkles on the outer packaging, it may also cause a single scan to fail, increase the complexity of user operations, and restrict further optimization and improvement of store settlement efficiency.

[0005] How to further optimize store checkout efficiency has become a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0006] This application provides a commodity settlement processing method and system, a computer-readable storage medium, an electronic device, and a computer program product, which can further optimize the overall settlement efficiency of offline stores.

[0007] This application provides the following solutions:

[0008] A method for processing commodity settlement is applied to smart glasses provided by physical stores. After a consumer user completes the purchase of commodities, the consumer wears the smart glasses and performs commodity settlement processing on the purchased commodities. The method includes:

[0009] When the consumer user takes the selected product for self-service checkout, an image of a target within the field of view of the smart glasses is captured;

[0010] If the target image includes a first image content of holding at least one commodity, obtaining commodity feature information of the at least one commodity from the target image through image analysis;

[0011] Commodities are identified based on the commodity feature information, and at least one successfully identified commodity is added to a set of commodities to be settled as a commodity to be settled, until a settlement request is submitted by the consumer user through the smart glasses, and settlement processing is performed on the commodities to be settled.

[0012] When it is detected that the consumer user is wearing the smart glasses, it is determined that the consumer user has started a self-service checkout process, and the smart glasses are triggered to collect target images.

[0013] The field of view of the smart glasses matches the visual range of the consumer user's eyes.

[0014] The method further comprises:

[0015] If there is a target product for which product identification fails, a prompt message for the target product is generated based on the product feature information, and is pushed to the consumer user via the smart glasses to remind the consumer user to adjust the hand position, and the target product is displayed within the field of view of the smart glasses;

[0016] The image after the hand position adjustment is collected, and the target product is identified based on the new product feature information obtained by image analysis.

[0017] The process of collecting the target image within the field of view of the smart glasses during the process of the consumer user taking the selected goods for self-service checkout includes:

[0018] The target image is captured during the process of the consumer user taking the selected merchandise from the shopping cart provided by the physical store and packaging and settling the taken merchandise.

[0019] After obtaining the settlement request, the method further includes:

[0020] If the target image includes a second image of selected commodities placed in a shopping cart, obtaining quantity information of the selected commodities from the target image through image analysis;

[0021] According to the matching between the quantity information of the purchased goods and the quantity information of the goods to be settled, the abnormal operation of the consumer user is judged;

[0022] If the consumer user does not perform any abnormal operation, the commodity to be settled is settled.

[0023] The method further comprises:

[0024] If the consumer user performs an abnormal operation, the smart glasses will be used to remind the consumer user of the abnormality.

[0025] If the selected items placed in the shopping cart are blocked by items, the method further includes:

[0026] For at least one other target image collected during the self-service checkout process, the quantity of purchased goods is counted respectively, and the statistical results of the target image and the other target images are combined to obtain the quantity information of the purchased goods.

[0027] A commodity settlement processing system, comprising:

[0028] Edge servers and multiple smart glasses deployed in physical stores;

[0029] The smart glasses are configured to be worn by a consumer user after the consumer user completes the purchase of goods, and to capture a target image within the field of view of the smart glasses while the consumer user is picking up the purchased goods for self-service checkout; if the target image includes a first image content of the consumer holding at least one item, the target image is sent to the edge server;

[0030] The edge server is configured to obtain product feature information of the at least one product from the target image through image analysis, identify the product based on the product feature information, and add the at least one successfully identified product to a set of products to be settled as a product to be settled;

[0031] The smart glasses are further configured to provide the commodities to be settled to the consumer user for confirmation, and to perform settlement processing on the commodities to be settled upon receiving a settlement request submitted by the consumer user through the smart glasses.

[0032] A commodity settlement processing device is applied to smart glasses provided by physical stores. After a consumer user completes the purchase of goods, the consumer wears the smart glasses to perform commodity settlement processing on the purchased goods. The device includes:

[0033] An image acquisition unit is used to capture target images within the field of view of the smart glasses when the consumer user takes the selected goods for self-service checkout;

[0034] An image analysis unit, configured to obtain commodity feature information of the at least one commodity from the target image through image analysis when the target image includes a first image content of a person holding at least one commodity;

[0035] a commodity identification unit, configured to identify commodities according to the commodity feature information, and add at least one successfully identified commodity as a commodity to be settled to a set of commodities to be settled;

[0036] The settlement processing unit is used to obtain the settlement request submitted by the consumer user through the smart glasses and perform settlement processing on the goods to be settled.

[0037] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the aforementioned methods.

[0038] An electronic device, comprising:

[0039] one or more processors; and

[0040] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of any of the aforementioned methods.

[0041] A computer program product comprises a computer program / computer executable instructions, wherein the computer program / computer executable instructions are capable of implementing the steps of any of the aforementioned methods when executed by a processor in an electronic device.

[0042] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0043] In an embodiment of the present application, a plurality of smart glasses can be provided in offline physical stores for consumer users to choose from. After the consumer users complete the purchase of goods in the store, they can wear the smart glasses to start the self-service checkout process of the goods. During the entire self-service checkout process, target images within the field of view of the smart glasses are collected, and image analysis is performed on the first image content of the handheld goods included in the target image. Then, the goods are identified and added to the purchase according to the characteristic information of the goods obtained by the analysis, until a settlement request submitted by the user through the smart glasses is received, and settlement processing is performed on the goods to be settled that have been successfully identified and added to the purchase.

[0044] The self-service checkout solution implemented through smart glasses is more convenient for consumers and helps reduce the space requirements of the store's self-service checkout area for the self-service checkout function. More smart glasses can be deployed in the same size self-service checkout area for users to choose from, thereby reducing the waiting time for users to check out when the store has a large flow of customers, and improving the store's overall checkout efficiency.

[0045] In addition, in the embodiments of the present application, the user's action of picking up at least one item from the selected items indicates their intention to self-checkout the picked item. Through image acquisition and image analysis, the smart glasses can simply and conveniently identify and add the picked item to the cart. This does not require the user to flip the item over to find the barcode printed area, nor does it require the user to deliberately scan the product barcode in a designated area. This helps to simplify user operations and thus improve the store's overall checkout efficiency. In addition, if a user picks up multiple items at the same time, they can also identify and add multiple items to the cart through a single image acquisition and image analysis, which also helps improve the store's overall checkout efficiency.

[0046] Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 is a schematic diagram of a commodity settlement processing system provided in an embodiment of the present application;

[0049] Figure 2 This is a flowchart of the commodity settlement processing method provided in an embodiment of the present application;

[0050] Figure 3 is a schematic diagram of a commodity settlement processing device provided in an embodiment of the present application;

[0051] Figure 4 Schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0053] In order to further optimize the store checkout efficiency of offline stores, the embodiment of the present application provides a commodity checkout processing solution based on smart glasses. In terms of self-service checkout hardware equipment, the present application adopts smart glasses that are more convenient to use to replace POS machines that need to be fixed and have high installation space requirements. This can not only reduce equipment costs, but also reduce the space requirements for setting up self-service checkout areas in stores, which is conducive to promoting the promotion and application of self-service checkout functions in offline stores with different space areas. In addition, compared with POS machines, more smart glasses can be deployed in the self-service checkout area of ​​the same size for consumers to choose during the self-service checkout process. It also helps to reduce the waiting time for users to check out when the store has a large customer flow, thereby achieving the purpose of improving the overall checkout efficiency of the store.

[0054] From the perspective of the self-service checkout process, after the consumer wears the smart glasses, the field of view of the smart glasses matches the visual range of the user's eyes, and usually the field of view of the smart glasses can completely cover the visual range of the user's eyes; in addition, when the user wants to settle the purchased goods, he or she usually takes the goods to be settled from the shopping cart (a physical shopping cart provided by the store, which is used to place the goods purchased by the user during the store), and this action will be accompanied by the user's line of sight, that is, the user's eyes will look at the shopping cart to determine which (which) goods to be settled to take, and / or to ensure that the hands can stably take the goods to be settled. During this process, the user's eyes will intentionally or unintentionally check the goods in his or her hand. The embodiment of the present application makes full use of the inevitable picking-up action that users must make when settling goods, as well as the characteristics of the line of sight when making the picking-up action, and provides a method of collecting target images within the field of view of the smart glasses by matching the user's visual range. Through image analysis, the characteristic information of the goods currently picked up by the user is obtained, and then the identity of the goods is identified. The goods picked up by the user are added to the set of goods to be settled (also known as a "shopping cart", which is a virtual "shopping cart" provided by the smart glasses, as opposed to the physical shopping cart provided by the store). In this way, the user does not need to flip over the goods to find the barcode printing area, nor does he need to deliberately scan the product barcode at the POS machine scanning terminal, which helps to simplify user operations. When the user picks up multiple items at the same time, the identification and purchase of multiple items can be realized in a single user operation, which also helps to improve the overall settlement efficiency of the store.

[0055] In the embodiments of this application, the smart glasses can simultaneously perform image acquisition and image analysis functions, and the corresponding product settlement processing solution can be applied to the smart glasses. Alternatively, the smart glasses can cooperate with the edge server deployed in the store to implement the product settlement processing process of this application.

[0056] Specifically, the smart glasses can be worn by a consumer user after the consumer user completes the purchase of goods, and when the consumer user takes the purchased goods for self-service checkout, the target image within the field of view of the smart glasses is collected; if the target image includes a first image content of holding at least one item, the target image is sent to the edge server.

[0057] Correspondingly, the edge server can obtain the product feature information of at least one product from the target image through image analysis, and then identify the product based on the product feature information, add the at least one successfully identified product as a product to be settled to the set of products to be settled, and provide the product to be settled to the consumer user through the smart glasses for confirmation, so that when the smart glasses receive the settlement request submitted by the consumer user through the smart glasses, the settlement process of the product to be settled is performed.

[0058] In other words, after the smart glasses capture a target image, they can first perform a simple recognition of the image content included in the target image. If they determine that the user's hand is present and holding at least one item, they can then send the target image to the edge server for further image analysis. This targeted triggering of the edge server can avoid frequent communication between the smart glasses and the edge server, and alleviate the concurrency issues that the edge server faces when multiple users are performing self-service checkout simultaneously, helping to improve the efficiency of product checkout processing and, in turn, the overall checkout efficiency of the store.

[0059] In addition, during the settlement process of this application, the settlement amount of the goods to be settled can also be calculated by combining the discount information such as the coupons held by the user and the marketing activities participated by the store. Considering that the changes of this part of information are relatively frequent, the embodiment of this application can save the discount information in the cloud server, and the cloud server cooperates with the edge server to realize the settlement amount calculation function, forming Figure 1 The following is a schematic diagram of the cloud-edge-end system architecture. This approach can fully leverage the advantages of efficient communication between smart glasses and edge servers, as well as the flexibility of cloud server information updates, to improve the efficiency of commodity checkout processing and, in turn, the overall store checkout efficiency.

[0060] The following is an explanation of the implementation process of the commodity settlement processing method of the embodiment of the present application with reference to specific examples. Figure 2 The flowchart shown may include:

[0061] S201: When the consumer user takes the selected merchandise for self-service checkout, a target image within the field of view of the smart glasses is captured.

[0062] In an embodiment of the present application, when a consumer user goes to an offline store to purchase goods, he or she can wear the smart glasses provided by the store after completing the purchase of goods, and use the smart glasses to perform the goods settlement processing for the purchased goods, replacing the traditional POS machine to realize self-service settlement of goods.

[0063] Specifically, when a user performs self-service checkout, the camera in the smart glasses can capture target images within the user's field of view as the consumer picks up items from the shopping cart provided by the physical store and packages and settles the items. The camera then uses image analysis to identify the items being picked up, allowing for add-on purchases and settlement, thus enabling self-service checkout. As an example of a usage scenario, packaging and settling items can involve the consumer placing the items in the shopping cart into their own shopping bags in batches.

[0064] To accurately and comprehensively identify products and ensure the correct number of identified products, the smart glasses can be triggered to capture target images when the consumer is wearing smart glasses, i.e., when the consumer initiates the self-service checkout process. For example, the smart glasses can be configured to sense whether the glasses are being worn by the user through a gravity sensor. This embodiment of the application does not limit the specific detection method.

[0065] As an example, after being triggered, the smart glasses can continuously and uninterruptedly capture target images and perform image analysis on the captured target images including the first image content. Alternatively, target image capture can be performed according to an image capture instruction. For example, when the smart glasses are equipped with a voice interaction module, the user can send an image capture instruction to the smart glasses through voice control. For example, each time the user takes an item, the user can voice-control the smart glasses' camera to capture an image. For another example, when the smart glasses sense that the user has taken an item from a shopping cart, an image capture instruction can be generated to control the camera to capture an image.

[0066] S202: If the target image includes a first image content of a person holding at least one commodity, obtain commodity feature information of the at least one commodity from the target image through image analysis.

[0067] The embodiment of the present application uses the target image including the first image content of the user's hand holding the product as the analysis basis, extracts the product feature information from it, and identifies the product identity based on it to perform add-to-cart processing.

[0068] As an example, product feature information can be reflected as the product's own pattern, outline, color, and other characteristic information. If the user holds the product and exposes the product's barcode to the camera, the product feature information extracted from the target image can also be reflected as the product's barcode information. In other words, product identity can be recognized by combining the product's own features and / or barcode information.

[0069] S203: Identify the product according to the product feature information, and add at least one successfully identified product as a to-be-settled product to a set of to-be-settled products, until a settlement request is submitted by the consumer user through the smart glasses, and then settle the to-be-settled products.

[0070] The product is identified based on its characteristic information. If the identification is successful, the identified product can be added to the virtual shopping cart as a pending item, thus completing the identification process for the user's currently picked-up product. Furthermore, the consumer user can continue to pick up other selected items, and after completing image acquisition and image recognition according to the present application solution, continue to add other selected items to the cart until a settlement request is received from the consumer user for the pending items in the virtual shopping cart. It is determined that the self-service identification of all pending items has been completed, and settlement processing can be performed for the pending items, and the user can be guided to complete the payment of the items.

[0071] It is understandable that when a user picks up multiple items at a time, the identity recognition of some of the items may fail because the items block each other. Correspondingly, the embodiment of the present application can also instantly push a prompt message of recognition failure to the user, so that the user can adjust the angle of the hand holding the item within the field of view of the smart glasses, display the target item as fully as possible within the field of view of the smart glasses, perform image capture and image analysis again, and re-identify the target item and add it to the purchase.

[0072] Specifically, it can be reflected as follows: if there is a target product that fails to be identified, prompt information for the target product is generated based on the product feature information, and pushed to the consumer user through the smart glasses to remind the consumer user to adjust the hand position, and the target product is displayed within the field of view of the smart glasses; an image after the hand position is adjusted is collected, and the target product is identified based on the new product feature information obtained by image analysis.

[0073] As an example, the voice interaction module can be used to broadcast the target product's prompt information to the consumer user in the form of a voice prompt, or the target product's prompt information can be projected onto the lenses of the smart glasses for the consumer user to view. The embodiment of the present application does not limit the push method of the prompt information.

[0074] Understandably, product identification failures often occur because only partial product feature information is collected. However, this information is not unique and cannot pinpoint a specific product. Therefore, it is important to provide users with as comprehensive product feature information as possible to help them identify the specific product(s) for which identification failed. For example, a prompt might read, "The yellow product in your hand is obstructed. Please adjust the angle."

[0075] In addition, in actual applications, intentional or unintentional abnormal operations such as missed recognition and repeated recognition may occur. In order to ensure the accuracy of the self-service checkout process, the embodiment of the present application can also provide an abnormal operation detection solution.

[0076] Specifically, if the target image includes a second image content of the selected goods placed in the shopping cart, the quantity information of the selected goods is obtained from the target image through image analysis; and based on the matching between the quantity information of the selected goods and the quantity information of the goods to be settled, the consumer user is judged to have performed an abnormal operation.

[0077] That is to say, after the target image is captured, image content recognition can be performed on the target image to determine that there is a shopping cart and that the user's purchased goods are placed in the shopping cart. Then, image analysis can be performed on the target image to obtain the quantity information of the purchased goods. In one example, the edge contour of the goods can be identified, and different purchased goods can be selected according to the contour box, and then the number of purchased goods can be counted. In this example, there is no need to perform product identity recognition, which helps to simplify the calculation process of determining the number of goods and improve the efficiency of counting the number of purchased goods. In another example, the product feature information of the purchased goods can be obtained from the target image, and after the product identity is identified according to the product feature information, the number of purchased goods can be obtained. The embodiment of the present application does not limit the implementation process of determining the quantity information of the purchased goods.

[0078] In actual applications, the first image content and the second image content may be simultaneously contained in the same target image, that is, the target image captured by the smart glasses includes both the image content of the user holding the product and the image content of the purchased products placed in the shopping cart. Based on this target image, the identity of the product taken this time can be analyzed and the quantity information of the purchased products can be fully or partially counted. Among them, complete statistics means that the number of all purchased products placed in the shopping cart can be counted by using a single target image. This is mostly applicable to situations where the number of purchased products is small and there are no completely obscured products. Partial statistics means that only the number of part of the purchased products placed in the shopping cart can be counted by using a single target image. The statistical results of multiple target images collected during the self-service checkout process can be merged to calculate the number of products. That is, the number of purchased products can be counted for at least one other target image collected during the self-service checkout process, and the statistical results of the target image and other target images can be merged to obtain the number of all purchased products. This is mostly applicable to situations where the purchased products placed in the shopping cart are obscured. It is understandable that the product obstruction may be caused by the overlapping of multiple products for purchase, resulting in partial obstruction of some products, and / or the user's body may be causing partial obstruction of some products for purchase.

[0079] Of course, the first image content and the second image content may also be included in different target images, and the identity of the product picked up by the user is identified through different target images, and the quantity information of the purchased products in the shopping cart is counted.

[0080] That is to say, this application can identify and add the goods in hand, as well as count and update the quantity of purchased goods during the entire process of the user's self-service checkout, until the consumer user submits a checkout request.

[0081] In addition, it should be noted that users may abandon the purchase of certain items during the checkout process. These abandoned items will remain in the shopping cart until the user submits the checkout request. Correspondingly, the quantity information of the purchased items obtained by the statistics of this application can be reflected as the number of all purchased items minus the number of abandoned items.

[0082] After obtaining the quantity information of the purchased goods and the quantity information of the goods to be settled, the consistency of the two can be determined. If the quantities of the two are consistent, it can be determined that the consumer user has not performed any abnormal operation during the self-service checkout process, and the settlement of the goods to be settled can be processed according to the settlement request submitted by the user. Conversely, if the quantities of the two are inconsistent, whether it is because the number of purchased goods is greater than the number of goods to be settled, indicating that there may be missed identification of goods, or the number of purchased goods is less than the number of goods to be settled, indicating that there may be repeated identification of goods, it can be determined that the consumer user has performed abnormal operation during the self-service checkout process. Correspondingly, the smart glasses can be used to remind the consumer user of the abnormality so that the user can check the goods and avoid the impact of missed identification or repeated identification on the user's experience of self-service checkout of goods through smart glasses.

[0083] Furthermore, while the abnormality alert is being sent to the consumer user, an early warning can also be sent to the store staff so that the staff can assist the user in completing the product verification. Alternatively, in one example, an early warning can be sent to the staff after the abnormality alert is sent to the user and the user has not completed the product verification.

[0084] As an example, the image analysis of the embodiment of the present application can be implemented by an AI model. The AI ​​model can be deployed on smart glasses and / or edge servers, and the target image is used as the model input. The AI ​​model identifies the first image content representing the user holding at least one commodity, and completes the processing processes such as commodity feature information extraction and commodity identification; in addition, according to usage needs, the AI ​​model can also identify the second image content representing the purchased commodities placed in the shopping cart from the target image, and complete the processing processes such as the number of purchased commodities. Specifically, the AI ​​model can be embodied as an image detection model obtained through massive sample training, or it can be embodied as an image detection model obtained by fine-tuning the model through a small number of samples. The embodiment of the present application does not limit this.

[0085] To sum up, the embodiment of the present application can provide multiple smart glasses for consumer users to choose from in offline physical stores. After the consumer users complete the purchase of goods in the store, they can wear the smart glasses to start the self-service checkout process of the goods. During the entire self-service checkout process, the target image within the field of view of the smart glasses is collected, and the first image content of the handheld goods included in the target image is analyzed. Then, the goods are identified and added to the purchase according to the product feature information obtained by the analysis, until the settlement request submitted by the user through the smart glasses is received, and the settlement processing is performed on the goods to be settled that have been successfully identified and added to the purchase.

[0086] The self-service checkout solution implemented through smart glasses is more convenient for consumers and helps reduce the space requirements of the store's self-service checkout area for the self-service checkout function. More smart glasses can be deployed in the same size self-service checkout area for users to choose from, thereby reducing the waiting time for users to check out when the store has a large flow of customers, and improving the store's overall checkout efficiency.

[0087] In addition, in the embodiments of the present application, the user's action of picking up at least one item from the selected items indicates their intention to self-checkout the picked item. Through image acquisition and image analysis, the smart glasses can simply and conveniently identify and add the picked item to the cart. This does not require the user to flip the item over to find the barcode printed area, nor does it require the user to deliberately scan the product barcode in a designated area. This helps to simplify user operations and thus improve the store's overall checkout efficiency. In addition, if a user picks up multiple items at the same time, they can also identify and add multiple items to the cart through a single image acquisition and image analysis, which also helps improve the store's overall checkout efficiency.

[0088] As described above, this application considers the user's act of picking up at least one item from the selected items as an "add-to-carry" action during the self-service checkout process. While the entire checkout process relies on the user's "add-to-carry" action, this action is a natural one during the checkout process and completely different from the conventional, rigid action of bringing an item to a POS terminal and scanning it. This significantly simplifies and naturalizes the "add-to-carry" action without departing from the "add-to-carry" action, enabling the user to add items to their cart while they're bagging their selected items.

[0089] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0090] Corresponding to the aforementioned method embodiment, the present application embodiment also provides a commodity settlement processing device, which is applied to smart glasses provided by physical stores. After the consumer user completes the purchase of goods, he wears the smart glasses to perform commodity settlement processing on the purchased goods. Figure 3 , the apparatus may include:

[0091] The image acquisition unit 301 is used to capture target images within the field of view of the smart glasses when the consumer user takes the purchased goods for self-service checkout;

[0092] An image analysis unit 302 is configured to obtain commodity feature information of the at least one commodity from the target image through image analysis when the target image includes a first image content of a person holding at least one commodity;

[0093] The commodity identification unit 303 is configured to identify commodities according to the commodity feature information, and add at least one successfully identified commodity as a commodity to be settled to a set of commodities to be settled;

[0094] The settlement processing unit 304 is configured to obtain the settlement request submitted by the consumer user through the smart glasses and perform settlement processing on the goods to be settled.

[0095] Wherein, the device further includes:

[0096] The trigger unit is used to determine that the consumer user starts a self-service checkout process when detecting that the consumer user is wearing the smart glasses, and trigger the smart glasses to capture the target image.

[0097] The field of view of the smart glasses matches the visual range of the consumer user's eyes.

[0098] Wherein, the device further includes:

[0099] a prompt information pushing unit, configured to, when there is a target product for which product identification fails, generate prompt information for the target product based on the product feature information, push the prompt information to the consumer user via the smart glasses to remind the consumer user to adjust the hand position, and display the target product within the field of view of the smart glasses;

[0100] The image acquisition unit can be specifically used to acquire the image after the hand position is adjusted, so that the commodity recognition unit can identify the target commodity based on the new commodity feature information obtained by image analysis.

[0101] The image acquisition unit may be specifically configured to acquire the target image when the consumer user takes the selected merchandise from the shopping cart provided by the physical store and packages and settles the purchase.

[0102] Wherein, the device further includes:

[0103] a quantity information obtaining unit configured to obtain quantity information of the selected commodities from the target image through image analysis after receiving the settlement request if the target image includes a second image of the selected commodities placed in the shopping cart;

[0104] An abnormal operation judgment unit, configured to judge an abnormal operation of the consumer user based on a match between the quantity information of the purchased goods and the quantity information of the goods to be settled;

[0105] The settlement processing unit may be specifically configured to perform settlement processing on the commodity to be settled when the consumer user does not perform any abnormal operation.

[0106] Wherein, the device further includes:

[0107] The abnormality reminder unit is used to remind the consumer user of the abnormality through the smart glasses when the consumer user has an abnormal operation.

[0108] If the selected commodities placed in the shopping cart are blocked by commodities, the device further comprises:

[0109] The quantity merging unit is used to count the quantity of purchased goods for at least one other target image collected during the self-service checkout process, and to merge the statistical results of the target image and the other target images to obtain the quantity information of the purchased goods.

[0110] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.

[0111] And an electronic device comprising:

[0112] one or more processors; and

[0113] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of any one of the method embodiments described above.

[0114] In addition, an embodiment of the present application also provides a computer program product, including a computer program / computer executable instructions, which, when executed by a processor in an electronic device, implements the steps of any of the methods described in the aforementioned method embodiments.

[0115] in, Figure 4The architecture of an electronic device is exemplarily shown. For example, the device 400 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, an aircraft, etc.

[0116] Reference Figure 4 , device 400 may include one or more of the following components: a processing component 402 , a memory 404 , a power component 406 , a multimedia component 408 , an audio component 410 , an input / output (I / O) interface 412 , a sensor component 414 , and a communication component 416 .

[0117] The processing component 402 generally controls the overall operation of the device 400, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the method provided by the technical solution of the present disclosure. In addition, the processing component 402 may include one or more modules to facilitate interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate interaction between the multimedia component 408 and the processing component 402.

[0118] The memory 404 is configured to store various types of data to support operations on the device 400. Examples of such data include instructions for any application or method operating on the device 400, contact data, phone book data, messages, pictures, videos, etc. The memory 404 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0119] The power supply component 406 provides power to the various components of the device 400. The power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 400.

[0120] The multimedia component 408 includes a screen that provides an output interface between the device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 408 includes a front camera and / or a rear camera. When the device 400 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0121] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC) that is configured to receive external audio signals when the device 400 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 404 or transmitted via the communication component 416. In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.

[0122] Input / output (I / O) interface 412 provides an interface between processing component 402 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0123] The sensor assembly 414 includes one or more sensors for providing various aspects of the status assessment of the device 400. For example, the sensor assembly 414 can detect the open / closed state of the device 400, the relative positioning of components, such as the display and keypad of the device 400. The sensor assembly 414 can also detect changes in the position of the device 400 or a component of the device 400, the presence or absence of user contact with the device 400, the orientation or acceleration / deceleration of the device 400, and temperature changes of the device 400. The sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0124] The communication component 416 is configured to facilitate wired or wireless communication between the device 400 and other devices. The device 400 can access a wireless network based on a communication standard, such as WiFi, or a mobile communication network such as 2G, 4G, 4G / LTE, 5G, etc. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0125] In an exemplary embodiment, the device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0126] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions. The instructions can be executed by the processor 420 of the device 400 to perform the method provided by the technical solution of the present disclosure. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0127] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application or certain parts of the embodiments.

[0128] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0129] The above is a detailed introduction to the solution provided by this application. Specific examples are used herein to illustrate the principles and implementation methods of this application. The description of the above embodiments is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the contents of this specification should not be understood as limiting this application.

Claims

1. A commodity settlement processing method, characterized in that: Applied to smart glasses provided by physical stores, after a consumer user completes the purchase of goods, the consumer wears the smart glasses to perform a settlement process for the purchased goods. The method includes: When the consumer user takes the selected product for self-service checkout, an image of a target within the field of view of the smart glasses is captured; If the target image includes a first image content of holding at least one commodity, obtaining commodity feature information of the at least one commodity from the target image through image analysis; Commodities are identified based on the commodity feature information, and at least one successfully identified commodity is added to a set of commodities to be settled as a commodity to be settled, until a settlement request is submitted by the consumer user through the smart glasses, and settlement processing is performed on the commodities to be settled.

2. The method according to claim 1, characterized in that When it is detected that the consumer user is wearing the smart glasses, it is determined that the consumer user starts a self-service checkout process, and the smart glasses are triggered to collect target images.

3. The method according to claim 1, characterized in that The field of view of the smart glasses matches the visual range of the consumer user's eyes.

4. The method according to claim 1, wherein The method further comprises: If there is a target product for which product identification fails, a prompt message for the target product is generated based on the product feature information, and is pushed to the consumer user via the smart glasses to remind the consumer user to adjust the hand position, and the target product is displayed within the field of view of the smart glasses; The image after the hand position adjustment is collected, and the target product is identified based on the new product feature information obtained by image analysis.

5. The method according to claim 1, characterized in that The collecting of target images within the field of view of the smart glasses during the process of the consumer user taking the selected goods for self-service checkout includes: The target image is captured during the process of the consumer user taking the selected merchandise from the shopping cart provided by the physical store and packaging and settling the taken merchandise.

6. The method according to claim 5, characterized in that After obtaining the settlement request, the method further includes: If the target image includes a second image of selected commodities placed in a shopping cart, obtaining quantity information of the selected commodities from the target image through image analysis; According to the matching between the quantity information of the purchased goods and the quantity information of the goods to be settled, the abnormal operation of the consumer user is judged; If the consumer user does not perform any abnormal operation, the commodity to be settled is settled.

7. The method according to claim 6, characterized in that The method further comprises: If the consumer user performs an abnormal operation, the smart glasses will be used to remind the consumer user of the abnormality.

8. The method according to claim 6 or 7, characterized in that If the selected items placed in the shopping cart are blocked by items, the method further includes: For at least one other target image collected during the self-service checkout process, the quantity of purchased goods is counted respectively, and the statistical results of the target image and the other target images are combined to obtain the quantity information of the purchased goods.

9. A commodity settlement processing system, characterized in that: The system comprises: Edge servers and multiple smart glasses deployed in physical stores; The smart glasses are configured to be worn by a consumer user after the consumer user completes the purchase of goods, and to capture a target image within the field of view of the smart glasses while the consumer user is picking up the purchased goods for self-service checkout; if the target image includes a first image content of the consumer holding at least one item, the target image is sent to the edge server; The edge server is configured to obtain product feature information of the at least one product from the target image through image analysis, identify the product based on the product feature information, and add the at least one successfully identified product to a set of products to be settled as a product to be settled; The smart glasses are further configured to provide the commodities to be settled to the consumer user for confirmation, and to perform settlement processing on the commodities to be settled upon receiving a settlement request submitted by the consumer user through the smart glasses.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

11. An electronic device, characterized in that: include: one or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions, wherein when the program instructions are read and executed by the one or more processors, the steps of the method according to any one of claims 1 to 8 are performed.

12. A computer program product comprising a computer program / computer executable instructions, characterized in that When the computer program / computer executable instructions are executed by a processor in an electronic device, the steps of the method according to any one of claims 1 to 8 are implemented.

Images