Store consumption management method and system based on face recognition and iris recognition
Through facial recognition and iris recognition technology, combined with big data analysis, the lack of targeted problems in store consumption management has been solved, personalized product recommendations have been achieved, and service efficiency and customer experience have been improved.
Patent Information
- Application Number
- CN202411006849.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-12
AI Technical Summary
In the prior art, store consumption management is not targeted and cannot understand consumers' needs, resulting in the inability to provide personalized recommendation services.
Using a method based on facial recognition and iris recognition, consumers' shopping data in the store are collected and processed, their staying actions and movement trajectories are judged, and product recommendations are combined with big data and user portraits.
It improves the service efficiency and customer experience of the store, provides more accurate data support for store operations, and realizes personalized product recommendations.
Smart Images

Figure CN118942008A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of consumer management technology, and more specifically to a store consumer management method and system based on facial recognition and iris recognition. Background Technology
[0002] Currently, brick-and-mortar stores are a promising new business model, and shopping in stores has become a way of life for modern people living in a fast-paced lifestyle. With continuous technological advancements, smart store systems can more efficiently manage store consumption, sales analysis, and other activities.
[0003] However, existing technologies for managing in-store consumption often lack specificity, fail to understand consumer needs, and lack a method for analyzing user-related data, resulting in an inability to provide targeted recommendations to consumers.
[0004] Therefore, how to provide a store consumption management method that can solve the above problems is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] In view of this, the present invention provides a store consumption management method and system based on facial recognition and iris recognition, which can provide personalized recommendations to consumers by recognizing and processing the data generated by consumers during the shopping process in the store.
[0006] To achieve the above objectives, the present invention adopts the following technical solution:
[0007] A store consumption management method based on facial recognition and iris recognition includes the following steps:
[0008] Collect initial facial and iris information of consumers entering the store, as well as videos of their activities within the store;
[0009] The activity video is processed, and the processing result is used to determine whether a consumer stops in front of a certain shelf.
[0010] If a consumer pauses for a period of time that is greater than or equal to a first preset time threshold, the consumer's hand movements are identified. When there is an action of picking up goods, the consumer's movement trajectory is tracked.
[0011] The consumer's need for payment is determined based on their movement trajectory.
[0012] Preferably, the specific process of determining whether a consumer needs to pay based on the movement trajectory includes:
[0013] Obtain the consumer's real-time location;
[0014] Trajectory prediction is performed based on the described movement trajectory and real-time location;
[0015] When the trajectory prediction result points to a checkout terminal, it helps consumers complete the checkout process.
[0016] Preferably, the specific process for completing the settlement of goods includes:
[0017] The consumer's facial and iris images are collected again and compared with the initial facial and iris information.
[0018] When the comparison results match, the payment is completed through a third-party payment platform based on either the facial image or the iris image.
[0019] Preferably, the specific process of determining whether a consumer lingered in front of a particular shelf based on the processing results of the activity video further includes:
[0020] When a consumer's dwell time on a certain shelf is less than a first preset time threshold but greater than or equal to a second preset time threshold, big data is used to associate the consumer's initial facial information and initial iris information, and to determine the associated consumer's public social media platform information and online platform consumption review information.
[0021] The publicly available social media platform information and the online platform consumer review information are fused and identified to obtain the corresponding fusion identification result;
[0022] Based on the fusion recognition results, a consumer profile is obtained, and product recommendations are made based on the user profile.
[0023] Preferably, the specific process of determining whether a consumer lingered in front of a particular shelf based on the processing results of the activity video further includes:
[0024] When a consumer stays on a shelf for less than a second preset time threshold, the activity video is extracted to obtain the consumer's region of interest, and product information is extracted from the region of interest.
[0025] Obtain product promotion information from stores via the internet, and match the product promotion information with the product information;
[0026] Based on the matched product promotion information, product recommendations are made to consumers.
[0027] Preferably, the specific process of recommending products to consumers based on the matching results also includes:
[0028] When no product promotion information is found, a voice conversation is conducted with the consumer via a voice device.
[0029] The voice dialogue is recognized, and product recommendations are made based on the voice recognition results.
[0030] Preferably, the specific process for fusing and identifying the publicly available social media platform information and the online platform consumer review information includes:
[0031] The publicly available social media platform information and the online platform consumer review information are preprocessed, and the corresponding first feature vector set and second feature vector set are extracted.
[0032] The first feature vector set and the second feature vector set are weighted and fused to obtain the corresponding fused feature set;
[0033] A recognition network is constructed, and the fused feature set is input into the recognition network for recognition to obtain the corresponding fused recognition result.
[0034] This invention also provides a store consumption management system based on facial recognition and iris recognition, comprising the following steps:
[0035] The data collection module is used to collect initial facial information, initial iris information, and video of activities within the store from consumers entering the store.
[0036] The processing module is used to process the activity video and determine whether a consumer stops in front of a certain shelf based on the processing result of the activity video;
[0037] The judgment module is used to identify the consumer's hand movements when the consumer pauses for a period of time greater than or equal to a first preset time threshold, and to track the consumer's movement trajectory when the consumer takes a product.
[0038] The settlement module is used to determine whether the consumer needs to settle the bill based on the movement trajectory.
[0039] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a store consumption management method and system based on face recognition and iris recognition. By collecting and processing user activity videos, movement trajectories, facial information and iris information of consumers entering the store, and executing relevant voice product recommendations and Q&A services based on the processing and judgment results of the above data, the service efficiency and customer experience of the store can be improved, and more accurate data support can be provided for store operation. Attached Figure Description
[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0041] Figure 1 The present invention provides an overall flowchart of a store consumption management method based on face recognition and iris recognition;
[0042] Figure 2 This invention provides a structural principle block diagram of a store consumption management system based on face recognition and iris recognition. Detailed Implementation
[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0044] See Figure 1 As shown in the figure, this invention discloses a store consumption management method based on face recognition and iris recognition, including the following steps:
[0045] Collect initial facial and iris information of consumers entering the store, as well as videos of their activities within the store;
[0046] Process the event video and determine whether consumers lingered in front of a particular shelf based on the processing results;
[0047] If a consumer pauses for a period of time that is greater than or equal to a first preset time threshold, the consumer's hand movements are identified. When there is an action of picking up goods, the consumer's movement trajectory is tracked.
[0048] Determine whether a consumer needs to pay based on their movement trajectory.
[0049] In one specific embodiment, the process of determining whether a consumer needs to pay based on their movement trajectory includes:
[0050] Obtain the consumer's real-time location;
[0051] Trajectory prediction is performed based on movement patterns and real-time location.
[0052] When the trajectory prediction result points to a checkout terminal, it helps consumers complete the checkout process.
[0053] In one specific embodiment, the specific process of completing the settlement of goods includes:
[0054] The consumer's facial and iris images are collected again and compared with the initial facial and iris information.
[0055] When the comparison results match, the payment is completed through a third-party payment platform based on either the facial image or the iris image.
[0056] In one specific embodiment, the process of determining whether a consumer lingered in front of a particular shelf based on the processing results of the activity video further includes:
[0057] When a consumer's dwell time on a certain shelf is less than a first preset time threshold but greater than or equal to a second preset time threshold, big data is used to associate the consumer's initial facial information and initial iris information, and to determine the associated consumer's public social media platform information and online platform consumption review information.
[0058] The information from publicly available social media platforms and consumer reviews from online platforms are integrated and identified to obtain the corresponding integrated identification results.
[0059] Based on the fusion recognition results, a user profile of the consumer is obtained, and products are recommended based on the user profile.
[0060] Specifically, in the process of building consumer profiles, data mining algorithms and association rule mining are used to process the fusion recognition results to generate corresponding consumer profiles. These profiles can include various types of information such as price, product type, product brand, and promotional information, which helps to better meet consumer needs.
[0061] In one specific embodiment, the process of determining whether a consumer lingered in front of a particular shelf based on the processing results of the activity video further includes:
[0062] When a consumer stays on a shelf for less than a second preset time threshold, the activity video is extracted to obtain the consumer's region of interest, and the product information included in the region of interest is extracted from the region of interest.
[0063] Obtain product promotion information from stores online and match the promotion information with product information;
[0064] Based on the matched product promotion information, product recommendations are made to consumers.
[0065] In one specific embodiment, the process of recommending products to consumers based on the matching results further includes:
[0066] When no product promotion information is found, a voice conversation is conducted with the consumer via a voice device.
[0067] The system recognizes voice conversations and recommends products based on the recognition results.
[0068] Specifically, when no product promotion information is matched, the system can also obtain the consumer's historical consumption information at the store through initial facial information and initial iris information, including the products purchased and the amount spent, in order to understand the consumer's preferences and consumption habits and make recommendations.
[0069] In a specific embodiment, the process of fusing and identifying publicly available social media platform information and online platform consumer review information includes:
[0070] The information from publicly available social media platforms and consumer reviews from online platforms are preprocessed, and the corresponding first feature vector set and second feature vector set are extracted.
[0071] The first feature vector set and the second feature vector set are weighted and fused to obtain the corresponding fused feature set;
[0072] A recognition network is constructed, and the fused feature set is input into the recognition network for recognition to obtain the corresponding fused recognition result.
[0073] See Figure 2 As shown, this embodiment of the invention also provides a management system based on face recognition and iris recognition for store consumption management, utilizing any one of the embodiments, comprising the following steps:
[0074] The data collection module is used to collect initial facial information, initial iris information, and video of activities within the store from consumers entering the store.
[0075] The processing module is used to process the event video and determine whether a consumer stops in front of a certain shelf based on the processing results.
[0076] The judgment module is used to identify the consumer's hand movements when the consumer pauses for a period of time that is greater than or equal to a first preset time threshold, and to track the consumer's movement trajectory when the consumer takes a product.
[0077] The checkout module is used to determine whether a consumer needs to pay based on their movement trajectory.
[0078] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0079] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A store consumption management method based on face recognition and iris recognition, characterized in that: The following steps are involved: Collect the initial facial information and iris information of consumers entering the store, as well as videos of their activities in the store; Processing the activity video, and judging whether the consumer stays in front of a certain shelf according to the processing result of the activity video; If the consumer stops for a period of time greater than or equal to a first preset time threshold, the consumer's hand movements are identified; if the consumer takes a product, the consumer's movement trajectory is tracked; Determine whether the consumer needs to settle the bill based on the action trajectory.
2. A store consumption management method based on face recognition and iris recognition according to claim 1, characterized in that: The specific process of determining whether the consumer needs to settle the bill based on the action trajectory includes: Get the real-time location of consumers; Perform trajectory prediction based on the action trajectory and real-time position; When the target of the trajectory prediction result is the settlement terminal, it helps consumers complete the settlement of goods.
3. The store consumption management method based on face recognition and iris recognition according to claim 1 is characterized in that: The specific process of completing the settlement of goods includes: The facial image and iris image of the consumer are collected again, and compared with the initial facial information and initial iris information; When the comparison results are consistent, payment is completed based on either the face image or the iris image in combination with a third-party payment platform.
4. The method for store consumption management based on face recognition and iris recognition according to claim 1, characterized in that: The specific process of judging whether the consumer stays in front of a shelf according to the processing result of the activity video also includes: When the consumer's stay time at a certain shelf is less than a first preset time threshold and greater than or equal to a second preset time threshold, the consumer's initial face information and initial iris information are used for association through big data, and the associated public social platform information and online platform consumption evaluation information of the consumer are determined; Performing fusion recognition on the public social platform information and the network platform consumption evaluation information to obtain a corresponding fusion recognition result; A user portrait of the consumer is obtained based on the fusion recognition result, and product recommendations are made based on the user portrait.
5. The store consumption management method based on face recognition and iris recognition according to claim 4 is characterized in that: The specific process of judging whether the consumer stays in front of a shelf according to the processing result of the activity video also includes: When a consumer stays at a shelf for less than a second preset time threshold, extracting the activity video to obtain the consumer's area of interest, and extracting the commodity information included in the area of interest; Obtaining the store's product promotion information through the Internet, and matching the product promotion information with the product information; Product recommendations are made to consumers based on the matched product promotion information.
6. A store consumption management method based on face recognition and iris recognition according to claim 5, characterized in that: The specific process of completing product recommendations for consumers based on matching results also includes: When no product promotion information is matched, a voice conversation is conducted with the consumer through a voice device; The voice conversation is recognized, and product recommendations are made based on the voice recognition results.
7. The store consumption management method based on face recognition and iris recognition according to claim 4 is characterized in that: The specific process of integrating and identifying the public social platform information and the network platform consumption evaluation information includes: Preprocessing the public social platform information and the network platform consumption evaluation information, and extracting the corresponding first feature vector set and second feature vector set; Performing weighted fusion on the first feature vector set and the second feature vector set to obtain a corresponding fused feature set; A recognition network is constructed, and the fused feature set is input into the recognition network for recognition to obtain a corresponding fused recognition result.
8. A management system using the store consumption management method based on face recognition and iris recognition as described in any one of claims 1 to 7, characterized in that: The following steps are involved: The collection module is used to collect the initial face information and initial iris information of consumers entering the store, as well as the activity videos in the store; A processing module, used to process the activity video and determine whether the consumer stays in front of a certain shelf according to the processing result of the activity video; A judgment module, used to identify the hand movement of the consumer if the consumer stops for a period of time greater than or equal to a first preset time threshold, and to track the movement trajectory of the consumer when the consumer takes the product; The settlement module is used to determine whether the consumer needs to settle the bill based on the action trajectory.
Citation Information
Patent Citations
Video-analysis-based antitheft monitoring method for physical store
CN102881100A
A payment device and a payment method integrating iris recognition and face recognition
CN109409895A
Potential customer mining and recommending method
CN110222272A
Multi-source heterogeneous data hybrid recommendation model based on deep learning
CN110263257A
Unmanned convenience store settlement method and device, computer and storage medium
CN110490697A
