Store customer emotion visualization guidance and ar decision assistance method and related device
By constructing an emotion fluctuation function and a stability determination mechanism, the problems of instantaneous jitter and multimodal misalignment in emotion recognition in stores were solved, achieving continuous and controllable emotion guidance and privacy protection, and improving the intelligent interaction capabilities of stores.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- LIGHT OF THE EARTH MUSEUM OPERATIONS & MANAGEMENT (WUXI) CO LTD
- Filing Date
- 2026-02-06
- Publication Date
- 2026-06-23
Smart Images
Figure CN122264795A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the cross-technical fields of multimodal emotion perception, temporal signal modeling, human-computer interaction and smart retail systems, and in particular to a method and related equipment for visually guiding customer emotions and AR decision-making assistance in stores. Background Technology
[0002] Customer emotion perception and interaction guidance in offline stores is a crucial topic in the fields of new retail and smart commerce. Without altering the product structure and pricing system, it can help stores reduce decision-making hesitation and customer churn by identifying and responding to customers' psychological states during the shopping process, thereby improving sales efficiency and conversion rates, and enhancing the overall service experience and operational efficiency in high-traffic, complex environments.
[0003] Currently, various technical approaches have emerged in the industry and academia regarding customer state recognition and experience optimization in stores. These primarily include behavior analysis methods based on statistical rules, state classification methods based on traditional machine learning, visual / voice emotion recognition methods based on deep learning, and attention mechanism models based on temporal modeling. However, these methods still have significant limitations in real-world store environments: 1) Transient and noise sensitive: Factors such as changes in store lighting, people blocking the view, crowding, and short-term changes in customer facial expressions can easily cause severe jitter in the emotion recognition output based on a single frame or short window, resulting in poor reliability.
[0004] 2) Asynchronous multimodal fusion: There is a natural time difference and signal-to-noise ratio difference in the response of different modalities such as vision (micro-expressions), speech (acoustic features), and behavior (movement lines) to the same emotional change. Simple feature splicing or decision-level fusion can hardly distinguish between "real emotional trend changes" and "noise caused by short-term misalignment of multimodal signals".
[0005] 3) Difficult to translate into controllable physical interaction: The results of existing emotion recognition solutions usually remain at the level of background data analysis or simple screen text / icon prompts, lacking a continuous and smooth control mechanism that maps continuously changing emotional states to guiding parameters (such as brightness, range, and frequency) of the physical space of the store (such as the floor and shelves).
[0006] 4) Significant privacy compliance pressure: Collecting biometric data such as facial and voice data in store settings is sensitive and poses a risk of data leakage. The solution must meet compliance requirements such as localized processing and data minimization.
[0007] Therefore, it is necessary to propose a technical solution for the store purchase decision-making process, which improves multimodal emotion recognition from outputting "instantaneous, discrete emotion labels" to generating "continuous, smooth, and interpretable emotion time-series signals", and through stability judgment and hysteresis triggering mechanism, stably drives ground / shelf projection thermal guidance and AR decision assistance, thereby forming a practical "perception-response-optimization" closed loop. Summary of the Invention
[0008] The main objective of this application is to propose a method, electronic device, storage medium, and program product for visual guidance and AR decision assistance of store customers' emotions based on emotion fluctuation function modeling. This method elevates emotions from "instantaneous labels" to "continuous and controllable temporal waveforms," and drives visual guidance and AR assistance that are deeply integrated with the store scenario through stability judgment and hysteresis triggering mechanism.
[0009] To achieve the above objectives, one aspect of this application proposes a method for visually guiding customer emotions and providing AR decision support in stores, the method comprising: S1: Collect multimodal data of customers in the store, including at least visual data, speech acoustic data, and behavioral data; S2: Preprocess and time-align the multimodal data to obtain an aligned time-series data stream; S3: Based on the time-series data stream, calculate the estimated value of the emotion intensity for each modality; S4: Construct the corresponding emotion fluctuation function based on the estimated emotion intensity values of each modality; S5: Integrate the multimodal emotion fluctuation function to obtain a comprehensive emotion fluctuation signal, and calculate the stability index and trend index of the comprehensive emotion fluctuation signal within a preset time window; S6: Based on the stability and trend indicators, combined with the preset hysteresis triggering conditions and the store decision-making stage state machine, generate interactive control instructions; S7: In response to the interactive control command, control the projection device to generate visual emotion guidance information in the associated shelf area, and / or control the AR device to generate emotion statistical tags associated with the current product.
[0010] In some embodiments, step S4, constructing the emotion fluctuation function specifically includes: The first Modality at time Estimated value of emotional intensity Mapped to the amplitude of emotional fluctuations ; The emotional fluctuation phase is updated based on historical changes in the emotional intensity estimate. ; According to the amplitude Fundamental angular frequency and phase Generate the initial emotion fluctuation function ; The initial emotion fluctuation function is damped and smoothed to obtain the smoothed emotion fluctuation function. .
[0011] In some embodiments, in step S5, the stability index is the variance or standard deviation of the comprehensive emotional fluctuation signal within the time window, and the trend index is the first-order difference mean of the comprehensive emotional fluctuation signal within the time window.
[0012] In some embodiments, in step S6, the hysteresis triggering condition includes: When the mean of the comprehensive emotional fluctuation signal within the time window is greater than or equal to the first threshold, and the stability index is less than or equal to the stability threshold, and this continues for a first predetermined duration, the guidance is triggered to start. When the average value is less than or equal to the second threshold and continues for a second predetermined duration, the guidance shutdown is triggered. Wherein, the first threshold is greater than the second threshold.
[0013] In some embodiments, step S7, controlling the projection device to generate visual emotion guidance information includes: The intensity or range of the projection guidance is dynamically adjusted based on the mean of the comprehensive emotional fluctuation signal. The update frequency of projection guidance is dynamically adjusted based on the stability index. Bind the projection area to the customer's currently associated shelf area ID.
[0014] In some embodiments, in step S7, the emotion statistics label is generated based on historical anonymous group data, including the historical statistical proportion of positive emotions evoked by the current product under similar decision-making stages.
[0015] In some embodiments, the method further includes edge computing and privacy protection steps: The processing of steps S1 to S6 is completed on the local edge device; Only the interactive control commands, projection control parameters, and anonymized tag data are transmitted externally; the original biometric data is not transmitted.
[0016] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.
[0017] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.
[0018] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the method described above.
[0019] Compared with the prior art, this application has the following beneficial effects: 1) Strong anti-interference capability: By modeling emotions as a continuous fluctuation function and performing damping smoothing and stability determination, the noise of the store environment and instantaneous signal jitter are effectively filtered out, which greatly reduces the false triggering rate of the guidance system.
[0020] 2) Smarter decision-making: By combining three-dimensional indicators of emotional intensity, stability, and trend with the business stage state machine, the guidance strategy achieves smooth and continuous control of "enhancement-maintenance-convergence", avoiding guidance flickering and better conforming to the customer's decision-making psychology.
[0021] 3) More precise interaction: Visual guidance (projection) is strongly linked to the shelf area where the customer is in real time, and AR information is strongly linked to product and group statistics, making the interactive prompts highly targeted and contextually relevant, thus improving guidance efficiency.
[0022] 4) Easy to implement and compliant: Adopting an edge computing architecture, raw data is processed locally, and only anonymized control signals and statistical labels are output, which perfectly complies with data privacy protection regulations and lowers the deployment threshold.
[0023] 5) Good system scalability: This application can be modularly integrated into existing smart shelves, shopping guide robots, and store projection systems, empowering traditional retail spaces to upgrade into smart interactive spaces. Attached Figure Description
[0024] Figure 1 This is a flowchart of the store customer emotion visualization guidance and AR decision-making assistance method provided in the embodiments of this application.
[0025] Figure 2 This is a flowchart of a method for visualizing and guiding the emotional space of store customers and providing AR decision support based on emotional fluctuation function modeling, provided in an embodiment of this application.
[0026] Figure 3 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.
[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0029] This application aims to address the following issues in existing in-store emotional interaction systems: 1) Solve the problem of frequent flickering and false triggering of spatial guidance caused by instantaneous jitter in emotion recognition output.
[0030] 2) Solve the problem of inconsistent judgment of emotional state and unreliable triggering caused by misalignment of multimodal emotional signals.
[0031] 3) Establish a smooth and continuous mapping and control mechanism from continuous emotional states to the visual output of store space (such as projection heatmaps, shopping guide paths, AR tags).
[0032] 4) Meets the requirements for low latency (e.g., <200ms) real-time response and data privacy compliance in store edge deployment.
[0033] To address the aforementioned issues, this application provides a method, electronic device, storage medium, and program product for visually guiding and assisting store customers' emotions using AR. The method constructs an emotion fluctuation function based on customers' multimodal emotion signals and uses this function to drive an interactive control scheme for ground / shelf projection thermal guidance and AR product emotion tag display.
[0034] The store customer emotion visualization guidance and AR decision-making assistance method provided in this application relates to the fields of smart retail and human-computer interaction technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application that implements the store customer emotion visualization guidance and AR decision-making assistance method, but is not limited to the above forms.
[0035] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0036] like Figure 1 As shown, this embodiment provides a method for visually guiding customer emotions and using AR to assist decision-making in stores, including the following steps: Step S101: Collect multimodal time-series data of customers in the store, including at least visual data, voice acoustic data, and behavioral movement data.
[0037] Step S102: Preprocess the modal data (such as cleaning and normalization) and align the time series with a unified time base.
[0038] Step S103: Based on the aligned time series data, estimate the emotional intensity value of each modality at consecutive time points, wherein the emotional intensity value is a continuous value.
[0039] Step S104: Based on the emotion intensity sequence of each modality, construct the corresponding emotion fluctuation function. This function transforms discrete emotion intensity points into continuous waveform signals. Its construction process includes amplitude mapping, phase update (reflecting the trend of emotion changes), and damping smoothing (anti-jitter).
[0040] Step S105: Weight and fuse the multimodal emotion fluctuation functions to obtain a comprehensive emotion fluctuation signal. Within a sliding time window, calculate the statistical characteristics of this signal, including the mean value representing the average intensity of emotion (…). ), variance representing emotional stability ( ) or standard deviation ( ), and the first-order difference mean representing the trend of emotion changes ( ).
[0041] Step S106: Introduce a hysteresis triggering mechanism and a state machine for store business scenarios. Based on the stability calculated in step S105... ) and mean intensity ( Combined with a preset hysteresis threshold (activation threshold) >Close threshold ) and minimum stable duration ( This determines whether to trigger guidance. Simultaneously, it considers trends ( Based on customer behavior stages (such as browsing, hesitation, confirmation), determine specific interaction strategies (such as enhanced guidance, providing AR explanations, and convergent guidance).
[0042] Step S107: Map the decision results to spatial control instructions: a) Control the projection device to generate a dynamic heat map in the customer's currently associated shelf area (ROI), the intensity, range, and update frequency of which are determined by... and Dynamic adjustment; b) When customers use AR devices, display sentiment labels based on historical anonymous group statistics (such as "85% of similar customers are satisfied") overlaid on relevant products, rather than personal real-time sentiment.
[0043] Step S108: The entire processing flow is first completed on the store's edge computing device. The original biometric data is discarded or desensitized after local processing, and only control commands and anonymous statistical information are output to ensure privacy and security.
[0044] The solutions of the embodiments of this application will be described in detail and explained below with reference to specific application examples.
[0045] (1) Definition of the store's "decision-making stage" The shopping process for in-store customers is divided into at least three stages. : Browsing phase (low interaction, low dwell time); : Comparison / hesitation stage (increased lingering time, more revisiting, richer micro-expressions); : Confirmation / Decision-Making Stage (Emotions stabilize, behavior tends towards purchasing or leaving).
[0046] This embodiment not only outputs the emotion category, but also outputs the interactive control quantity associated with the store decision-making stage, which is used to determine the projection guidance strategy and AR tag content.
[0047] (2) Data source and fields 2.1) Visual data fields (3D vision / depth camera): The slight tremor of the corners of the mouth , Pupil diameter change , eyelid opening and closing degree , Head posture , upper limb movement frequency .
[0048] 2.2) Speech acoustic fields (microphone array / voice terminal): baseband Sound intensity Speech rate .
[0049] Semantic content is not collected; only acoustic sentiment features are considered.
[0050] 2.3) Behavioral Fields (Visual Movement / Shelf Sensing): Duration of stay ; Replay frequency ; Number of times to wander ; Changes in distance from the shelf .
[0051] like Figure 2 As shown, this embodiment provides a method for visual guidance and AR decision support of store customers' emotional space based on emotional fluctuation function modeling, including the following steps: Step S1: Multimodal data acquisition.
[0052] Data collection devices are deployed on store ceilings, shelf edges, or shopping guide robots to record three types of data streams using a unified timestamp:
[0053] in, , , These represent visual, speech, and behavioral modalities, respectively.
[0054] Step S2: Preprocessing and timing alignment (required for stores).
[0055] 1) Cleaning: Remove severely occluded / outliers (such as those with abnormal depth values or excessively low speech energy); 2) Normalization: Perform Z-score or Min-Max normalization on fields with different dimensions; 3) Alignment: To ensure uniform sampling period Resampling and aligning modes:
[0056] 4) Visual association: Bind customer ID to shelf area ID (ROI / area number) to form a store "people-shelf" association sequence for subsequent thermal projection area mapping.
[0057] Step S3: Multimodal sentiment intensity estimation (continuous values rather than labels).
[0058] Output the emotional intensity for each modality.
[0059] It can also output an emotion category probability vector. .
[0060] To enhance in-store applications, at least two key emotional dimensions should be defined: Hesitation / Anxiety Intensity With interest / pleasure intensity .
[0061] This can be combined into a single example of "decision-making pressure intensity":
[0062] Step S4: Construction of the Emotional Fluctuation Function (Core Barrier).
[0063] Will Mapped to a mood fluctuation function : 1) Amplitude mapping:
[0064] 2) Phase update (reflecting trends):
[0065] 3) Emotional fluctuation function:
[0066] 4) Damping smoothing (store anti-vibration):
[0067] Key technical points: Transforming "instantaneous emotions" into "continuous waveforms," with all subsequent triggering and control based on... It is based on, rather than on instantaneous classification results, thus forming a technological barrier.
[0068] Step S5: Multimodal emotional wave fusion and stability determination.
[0069] 1) Integration:
[0070] 2) Store Decision Window (linked to Browse / Hesitation / Confirmation): For window Calculate stability and trend:
[0071]
[0072]
[0073] in: Large indicates high emotional intensity (e.g., significant hesitation / anxiety); The small indicator shows a stable emotional state, and guidance can be safely triggered (without flashing). This indicates heightened emotions (increased hesitation), and guidance or supplementary information should be provided. A negative result indicates a decline in sentiment (decision-making tends to be confirmed), which can be used to guide the market and turn into a transaction alert.
[0074] Step S6: Delayed triggering + store scenario state machine (strongly bound to the application).
[0075] Hysteresis thresholding is used to prevent projection flicker. A trigger threshold is set. , closing threshold ,and And set a stability threshold. and sustained frame rate :
[0076] And by introducing a store state machine, triggers will be linked to business actions: 1) If And the customer is in the "hesitation stage". The strategy of "enhanced guidance + AR explanation" is implemented. 2) If the trend If the values are continuously negative and the dwell time increases, it is determined to be in the "confirmation phase". The strategy of "guided convergence + transaction auxiliary information" is implemented.
[0077] Step S7: Spatial visualization guidance output (projection heat map bound to shelf area).
[0078] Will Mapped to projection control parameters: Guide intensity (brightness / saturation):
[0079] Guiding range (thermal diffusion radius):
[0080] Update frequency (to avoid flickering):
[0081] Region binding (key application point): The projection area is selected as the shelf area currently associated with the customer. :
[0082] This creates an emotionally charged atmosphere "around the customer and around the shelf," rather than a general projection.
[0083] Step S8: AR emotion tagging (binding group statistics and product hierarchy).
[0084] When a customer uses AR glasses / terminals, the corresponding shelf / product will be displayed. Generate tags:
[0085] in Based on historical statistics: the percentage of times a product triggers feelings of "surprise" or "satisfaction" among similar groups of people / in similar emotional states. This method avoids the privacy risks of "directly displaying personal emotions" and emphasizes "group statistical tags".
[0086] Step S9: Edge computing and privacy protection (a strength in practical application).
[0087] 1) The original image and audio are only cached locally for a short time and then discarded after processing; 2) Output only , , And projection control parameters, without transmitting biological characteristics; 3) Optional: Hash the customer ID or map it to a temporary anonymous ID.
[0088] Example 1: Proactive guidance during the hesitation phase A customer paused in front of the brand's coffee machine display area. The system detected an increased frequency of subtle tremors at the corners of their mouth and a slight dilation of their pupils (micro-expressions of hesitation / anxiety) using a 3D vision sensor. The microphone array captured their increased speaking speed and rising tone during conversations with the sales assistant. Behavioral analysis also showed the duration of their stay in front of the shelf. It exceeded the average level and showed a replay action.
[0089] The data processing module calculates the increase in "hesitation intensity" in the visual and speech modalities in real time and constructs an emotion fluctuation function. After fusion, within a set decision window, the average value of the combined emotion fluctuation signals is used. Continues to rise and exceeds the threshold Meanwhile, its standard deviation The level remains low (indicating stable emotional fluctuations, not noise interference). The hysteresis trigger condition is met.
[0090] The system determines that the customer is in the "hesitation stage" ( The decision module issued an "enhanced guidance" command. The projection control module then projected a gradient warm-colored halo path onto the floor in front of the shelf where the coffee machine was located. The brightness of the halo ( ) and scope ( )and The values are positively correlated, and at the same time because The value is small, and the halo appears at a lower frequency. The transitions are smooth to avoid flickering and disturbing customers. Meanwhile, if a customer wears AR glasses or uses a mobile AR app, a label will appear above the coffee machine in the lens: "Data shows that in the past week, 78% of customers with similar browsing history to you were ultimately satisfied with this model." This label is based on desensitized group behavior statistics, not the customer's personal emotions.
[0091] This guidance aims to reduce customers' decision-making anxiety and provide cues for social validation, thereby facilitating decision-making.
[0092] Example 2: Guided convergence during the confirmation phase Following the scenario in Example 1, the customer's emotional state changed. The system detected that their micro-expressions became more relaxed, their tone of voice slowed down, and they stopped repeatedly comparing different products, instead focusing intently on the target product.
[0093] Trend indicators calculated by the emotion fusion module Multiple consecutive windows showed negative values (a decline in sentiment intensity). Despite the mean... Still relatively high, but combined Based on the trend and extended dwell time, the scenario state machine updates the customer state to the "confirmation stage". )".
[0094] The decision-making module then issues a "guided convergence" command. The projection control module begins to gradually reduce the radius R of the ground halo and decrease its brightness I, eventually converging the halo into a gentle, constantly lit circle at the bottom of the product or an arrow pointing towards the checkout counter. The update frequency F is also further reduced until it remains stationary. The AR tag content may also switch to "Please signal to the sales assistant for purchase assistance" or display promotional information. This process is smooth and uninterrupted, designed to provide final confirmation assistance for the upcoming purchase, rather than continuing to exert decision-making pressure.
[0095] In summary, compared with the prior art, the method of this embodiment has at least the following advantages and beneficial effects: 1) The emotion is upgraded from "instantaneous label" to "continuous and controllable waveform", which significantly reduces false triggering caused by store environmental noise; 2) By using stability and trend indicators, continuous control of the "enhancement / convergence" process is achieved to avoid flickering; 3) Projection and AR output are strongly linked to "shelf ROI product level / decision stage", forming a practical closed loop; 4) Edge computing and anonymized output meet privacy compliance deployment requirements; 5) It can be integrated with smart shelves, shopping guide robots, and in-store projection systems, and has strong scalability.
[0096] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.
[0097] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0098] Please see Figure 3 , Figure 3 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 301 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 302 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 302 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 302 and is called and executed by the processor 301 using the methods described above in the embodiments of this application. Input / output interface 303 is used to implement information input and output; The communication interface 304 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 305 transmits information between various components of the device (e.g., processor 301, memory 302, input / output interface 303, and communication interface 304); The processor 301, memory 302, input / output interface 303, and communication interface 304 are connected to each other within the device via bus 305.
[0099] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.
[0100] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.
[0101] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0102] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.
[0103] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented in the embodiments of this program product are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.
[0104] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.
[0105] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.
[0106] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0107] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.
[0108] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0109] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.
[0110] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0111] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0113] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0114] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.
Claims
1. A method for visually guiding customer emotions and using AR to assist decision-making in stores, characterized in that, The method includes the following steps: S1: Collect multimodal data of customers in the store, including at least visual data, speech acoustic data, and behavioral data; S2: Preprocess and time-align the multimodal data to obtain an aligned time-series data stream; S3: Based on the time-series data stream, calculate the estimated value of the emotion intensity for each modality; S4: Construct the corresponding emotion fluctuation function based on the estimated emotion intensity values of each modality; S5: Integrate the multimodal emotion fluctuation function to obtain a comprehensive emotion fluctuation signal, and calculate the stability index and trend index of the comprehensive emotion fluctuation signal within a preset time window; S6: Based on the stability and trend indicators, combined with the preset hysteresis triggering conditions and the store decision-making stage state machine, generate interactive control instructions; S7: In response to the interactive control command, control the projection device to generate visual emotion guidance information in the associated shelf area, and / or control the AR device to generate emotion statistical tags associated with the current product.
2. The method according to claim 1, characterized in that, In step S4, constructing the emotion fluctuation function specifically includes: The first Modality at time Estimated value of emotional intensity Mapped to the amplitude of emotional fluctuations ; The emotional fluctuation phase is updated based on historical changes in the emotional intensity estimate. ; According to the amplitude Fundamental angular frequency and phase Generate the initial emotion fluctuation function ; The initial emotion fluctuation function is damped and smoothed to obtain the smoothed emotion fluctuation function. .
3. The method according to claim 1 or 2, characterized in that, In step S5, the stability index is the variance or standard deviation of the comprehensive emotional fluctuation signal within the time window, and the trend index is the first-order difference mean of the comprehensive emotional fluctuation signal within the time window.
4. The method according to claim 1, characterized in that, In step S6, the hysteresis triggering condition includes: When the mean of the comprehensive emotional fluctuation signal within the time window is greater than or equal to the first threshold, and the stability index is less than or equal to the stability threshold, and this continues for a first predetermined duration, the guidance is triggered to start. When the average value is less than or equal to the second threshold and continues for a second predetermined duration, the guidance shutdown is triggered. Wherein, the first threshold is greater than the second threshold.
5. The method according to claim 1, characterized in that, In step S7, controlling the projection device to generate visual emotion guidance information includes: The intensity or range of the projection guidance is dynamically adjusted based on the mean of the comprehensive emotional fluctuation signal. The update frequency of projection guidance is dynamically adjusted based on the stability index. Bind the projection area to the customer's currently associated shelf area ID.
6. The method according to claim 1, characterized in that, In step S7, the sentiment statistics label is generated based on historical anonymous group data, including the historical statistical proportion of positive emotions evoked by the current product under similar decision-making stages.
7. The method according to claim 1, characterized in that, The method also includes edge computing and privacy protection steps: The processing of steps S1 to S6 is completed on the local edge device; Only the interactive control commands, projection control parameters, and anonymized tag data are transmitted externally; the original biometric data is not transmitted.
8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.