Using voice prints and / or large language models to characterize conversations and make recommendations in countertop retail

Abstract

Methods and systems for monitoring employee-customer conversations in commercial settings to identify participants, detect key events, and generate actionable insights into employee performance, customer sentiment, and operational efficiency are described. In some instances, for example the disclosed methods comprise capturing audio data for a transaction conversation at a transaction location within a commercial setting using at least one microphone positioned at the transaction location; generating a transaction conversation record based on the captured audio data; and processing the transaction conversation record using at least one machine learning model, where the at least one machine learning model is configured to process the transaction conversation record and detect a transaction event.

Description

USING VOICE PRINTS AND / OR LARGE LANGUAGE MODELS TO CHARACTERIZE CONVERSATIONS AND MAKE RECOMMENDATIONS IN COUNTERTOP RETAILCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the priority benefit of United States Provisional Patent Application Serial No. 63 / 545,437, filed October 24, 2023, and of United States Provisional Patent Application Serial No. 63 / 545,448, filed October 24, 2023, the contents of each of which are incorporated herein by reference in their entirety.FIELD

[0002] The present disclosure relates generally to methods and systems for identifying individuals and analyzing customer and employee interactions in commercial settings (e.g., retail settings) by capturing, analyzing, and characterizing transaction conversations (e.g., in-store conversations) through the use of voice biometrics and / or large language models (LLMs).BACKGROUND

[0003] In the digital world, it is standard practice to track user behavior through the use of cookies, enabling businesses to understand consumer preferences, engagement patterns, and website behavior. This information helps build detailed profiles of online users, allowing for personalized experiences, targeted marketing, and performance optimization. However, in physical environments (e.g., retail locations), capturing the same level of detail about interactions (e.g., in-store interactions between both customers and employees) has proven to be far more challenging.

[0004] Current methods for gathering in-store insights in the retail sector, such as transaction logs (TLOGs), customer surveys, and manual observation, are limited in their ability to provide realtime, detailed data. TLOGs may show what was purchased, but they don’t reveal the customer’s decision-making process or how employees influenced the sale. Similarly, customer complaints or feedback are often shared verbally with store employees but rarely reach management or decisionmakers in a way that can drive timely action. As a result, retailers often lack critical insights into who is engaging with their store, how employees are influencing customer decisions, and howcustomers are reacting to promotions, service quality, or operational issues. Improved methods and systems are required to address these deficiencies.BRIEF SUMMARY OF THE INVENTION

[0005] The methods and systems disclosed herein provide managers of commercial work environments (e.g., retail locations) with access to real-time data on customer base, employee base, customer and / or employee interactions, service quality, and / or operational issues, etc., by introducing systems that capture, analyze, and summarize conversations between customers and / or employees in real time using voice biometrics and / or advanced machine learning models (e.g., large language models (LLMs)). The disclosed methods and systems can be used to create unique profiles for both customers and employees based on in-store interactions. By capturing and analyzing conversations at, for example, the point of sale (POS) or other countertop service areas, the disclosed systems can generate voiceprints for each individual, thereby allowing retailers (or other managers) to track and understand the behavior of both customers and employees in real time. This approach is akin to how online businesses use cookies to build user profiles, but applied to, e.g., the physical retail space. The disclosed methods and systems enable retailers to go beyond transaction data to capture insights into the interactions that shape customer decisions and employee performance.

[0006] The methods and systems described herein can be used to identify individual speakers (e.g., cashiers, managers, customers, etc.), transcribe in-store conversations, detect key events, and enable retailers to gather detailed insights about how interactions take place, who is influencing purchase decisions, and how customers are responding to in-store experiences. Over time, voiceprints can be used to build comprehensive profiles of individuals, allowing for better personalization, performance evaluation, and decision-making. This technology is designed to provide the same level of insight into customer-employee interactions that voice analytics provides for contact centers, allowing retailers to track not just transactions, but the conversations that influence them. Implemented through a combination of machine learning (ML) models, noise-canceling microphonearrays, and real-time voice diarization, the disclosed systems ensure accurate identification of individuals, even in noisy or busy environments.

[0007] In some instances, the disclosed systems utilize domain-specific large language models, optimized for the unique vocabulary, challenges, and interaction types commonly found in retail locations (e.g., fuel stations and convenience stores). Unlike generic voice analytics systems, the disclosed systems are capable of detecting nuanced events, such as employee engagement with loyalty programs, product quality issues, and customer complaints. The ability to categorize and characterize these conversations enables retailers to respond to operational challenges in real time while recognizing long-term patterns across locations.

[0008] In addition, this disclosed systems directly impact customer experience. By analyzing feedback in real time, store operators can address customer complaints immediately — whether related to product quality, service speed, or facility issues — ultimately improving customer satisfaction and loyalty. The ability to act on these insights allows retailers to enhance service quality while identifying and resolving issues that might otherwise go unnoticed.

[0009] The benefits of deploying the disclosed systems extend beyond operational insights for store operators. For example, by analyzing employee behavior during customer interactions, the systems can provide feedback for training and performance improvements, helping to mitigate the high turnover rates that are typical in some retail locations (e.g., fuel stations and convenience stores). At the chain level, the disclosed systems can aggregate data across multiple locations, allowing corporate decision-makers to detect trends, address service gaps, and make data-driven decisions on staffing, promotions, and training.

[0010] The disclosed systems also address privacy concerns, even when operating in public spaces with no expectation of privacy, by ensuring that voice data is stored securely, with retailers owning and controlling the data. By applying voice biometrics and / or machine learning to the analysis of transaction conversations in physical retail setting (or other commercial environments), the disclosed systems provide retailers with a powerful tool to understand their workforce and customers, creatingdetailed, actionable profiles that enable improvements in employee training, customer experience, and operational efficiency. These insights, which were previously unavailable in offline environments, give retailers a new level of visibility into in-store interactions, helping them drive better business outcomes.

[0011] Disclosed herein are systems comprising: at least one microphone positioned at a transaction location within a commercial setting, and configured to capture audio data from participants in a transaction conversation at the transaction location; an edge device configured to transcribe the captured audio data and generate a record of the transaction conversation; and at least one processor configured to run a machine learning model, wherein the machine learning model is configured to process the transaction conversation record and detect a transaction event.

[0012] In some embodiments, the machine learning model is further configured to identify a participant in the transaction conversation. In some embodiments, the machine learning model is further configured to identify a category of participant for at least one participant in the transaction conversation. In some embodiments, the machine learning model is further configured to identify a type of transaction based on the transaction event detected in the transaction conversation record. In some embodiments, the machine learning model is further configured to generate a transaction event-specific summary of the transaction conversation. In some embodiments, the machine learning model is further configured to generate a sentiment evaluation score for the transaction conversation.

[0013] In some embodiments, the system further comprise a cloud-based server configured to store uploaded transaction conversation records and / or associated metadata. In some embodiments, the associated metadata comprises transaction type data, transaction event-specific summaries, transaction log (TLOG) data, loyalty program data, sentiment evaluation scores, or any combination thereof.

[0014] In some embodiments, the transaction conversation record comprises a transcript of the transaction conversation and / or a voiceprint for at least one participant in the transaction conversation.

[0015] In some embodiments, the machine learning model runs on a cloud-based computing platform.

[0016] In some embodiments, the at least one microphone is equipped with noise cancellation technology to filter out ambient noise from the commercial setting. In some embodiments, the at least one microphone comprises a microphone array. In some embodiments, the system comprises at least two microphones positioned in at least two transaction locations. In some embodiments, the at least two transaction locations are located at a same geographical location. In some embodiments, the at least two transaction locations are located in at least two different geographical locations.

[0017] In some embodiments, the edge device is configured to perform real-time transcription of the captured audio data using a speech recognition model. In some embodiments, the edge device is configured to segment the captured audio data into predefined intervals and perform real-time transcription using a Whisper model.

[0018] In some embodiments, the machine learning model is trained on a training dataset comprising human-tagged audio data captured for a plurality of transaction conversations involving a plurality of different participants. In some embodiments, the machine learning model is trained on a training dataset comprising domain-specific data for the commercial setting.

[0019] In some embodiments, identification of a participant in the transaction conversation by the machine learning model comprises a comparison of a voiceprint for a participant in the transaction conversation with a database of voiceprints to identify the participant. In some embodiments, the participant is identified if a match between a voiceprint for the participant to a voiceprint in the database of voiceprints exceeds a predetermined confidence threshold.

[0020] In some embodiments, the machine learning model comprises a large language model (LLM) configured to process transaction conversation records and detect transaction events, generate transaction event-specific summaries, and / or generate sentiment evaluation scores.

[0021] In some embodiments, the commercial setting is a retail setting. In some embodiments, the transaction location is a point-of-sale (POS) location.

[0022] In some embodiments, the commercial setting is a retail setting, and the category of participant identified for the at least one participant is an employee, a cashier, a manager, a customer, a daily customer, a weekly customer, a monthly customer, or an irregular customer.

[0023] In some embodiments, the commercial setting is a retail setting, and the associated metadata further comprises transaction data, timestamp data, point-of-sale (POS) register identifiers, or any combination thereof.

[0024] In some embodiments, the retail setting is a fuel station or convenience store setting.

[0025] Also disclosed herein are methods comprising: capturing audio data for a transaction conversation at a transaction location within a commercial setting using at least one microphone positioned at the transaction location; generating a transaction conversation record based on the captured audio data; and processing the transaction conversation record using at least one machine learning model, wherein the at least one machine learning model is configured to process the transaction conversation record and detect a transaction event.

[0026] In some embodiments, the machine learning model is further configured to identify a participant in the transaction conversation. In some embodiments, the machine learning model is further configured to identify a category of participant for at least one participant in the transaction conversation. In some embodiments, the machine learning model is further configured to identify a type of transaction based on the transaction event detected in the transaction conversation record. In some embodiments, the machine learning model is further configured to generate a transaction event-specific summary of the transaction conversation. In some embodiments, the machine learning model is further configured to generate a sentiment evaluation score for the transaction conversation.

[0027] In some embodiments, the at least one microphone is equipped with noise cancellation technology to filter out ambient noise from the commercial setting. In some embodiments, the atleast one microphone comprises a microphone array. In some embodiments, the at least one microphone comprises at least two microphones positioned in at least two transaction locations. In some embodiments, the at least two transaction locations are located at a same geographical location. In some embodiments, the at least two transaction locations are located at least two different geographical locations.

[0028] In some embodiments, the transaction conversation record comprises a transcript of the transaction conversation and / or a voiceprint for at least one participant in the transaction conversation.

[0029] In some embodiments, the transaction conversation record is generated by an edge device configured to perform real-time transcription of the captured audio data using a speech recognition model. In some embodiments, the transaction conversation record is generated by an edge device configured to segment the captured audio data into predefined intervals and perform real-time transcription using a Whisper model. In some embodiments, the transaction conversation record is generated by an edge device configured to upload transaction conversation records and / or associated metadata to a cloud-based server. In some embodiments, the associated metadata comprises transaction type data, transaction event-specific summaries, transaction log (TLOG) data, loyalty program data, sentiment evaluation scores, or any combination thereof.

[0030] In some embodiments, the machine learning model runs on a cloud-based computing platform.

[0031] In some embodiments, the machine learning model is trained on a training dataset comprising human-tagged audio data captured for a plurality of transaction conversations involving a plurality of different participants. In some embodiments, the machine learning model is trained on a training dataset comprising domain-specific data for the commercial setting.

[0032] In some embodiments, identification of a participant in the transaction conversation by the machine learning model comprises a comparison of a voiceprint for a participant in the transaction conversation with a database of voiceprints to identify the participant. In some embodiments, theparticipant is identified if a match between a voiceprint for the participant to a voiceprint in the database of voiceprints exceeds a predetermined confidence threshold.

[0033] In some embodiments, the machine learning model comprises a large language model (LLM) configured to process transaction conversation records and detect transaction events, generate transaction event-specific summaries, and / or generate sentiment evaluation scores.

[0034] In some embodiments, the commercial setting is a retail setting. In some embodiments, the transaction location is a point-of-sale (POS) location.

[0035] In some embodiments, the commercial setting is a retail setting, and the category of participant identified for the at least one participant is an employee, a cashier, a manager, a customer, a daily customer, a weekly customer, a monthly customer, or an irregular customer.

[0036] In some embodiments, the commercial setting is a retail setting, and the associated metadata further comprises transaction data, timestamp data, point-of-sale (POS) register identifiers, or any combination thereof.

[0037] In some embodiments, the retail setting is a fuel station or convenience store setting.

[0038] It should be appreciated that all combinations of the foregoing concepts and additional concepts discussed in greater detail below (provided such concepts are not mutually inconsistent) are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein.INCORPORATION BY REFERENCE

[0039] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference in their entirety to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated byreference in its entirety. In the event of a conflict between a term herein and a term in an incorporated reference, the term herein controls.BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Various aspects of the disclosed methods, devices, and systems are set forth with particularity in the appended claims. A better understanding of the features and advantages of the disclosed methods, devices, and systems will be obtained by reference to the following detailed description of illustrative embodiments and the accompanying drawings, of which:

[0041] FIG. 1 provides a non-limiting example of a process flowchart for identifying a participant and / or detecting a transaction event in a transaction conversation in a commercial setting, in accordance with one implementation of the methods and systems disclosed herein.

[0042] FIG. 2 provides a non-limiting schematic illustration of a system for identifying a participant and / or detecting a transaction event in a transaction conversation in a commercial setting, in accordance with one implementation of the methods and systems disclosed herein.

[0043] FIG. 3 provides a non-limiting example of a process flowchart for analyzing audio recordings to identify individuals (e.g., cashiers and / or customers in retail locations) using voice biometrics, in accordance with one implementation of the methods and systems disclosed herein.

[0044] FIG. 4A provides a schematic illustration of a mobile phone (e.g., a cashier’s mobile phone).

[0045] FIG. 4B provides a non-limiting example of a process flowchart for creating a voiceprint for an individual (e.g., a cashier), in accordance with one implementation of the methods and system disclosed herein.

[0046] FIG. 5 provides a non-limiting example of a process flowchart for routing voiceprints to specified locations for use in identifying individuals (e.g., cashiers) engaged in transactions at location(s) (e.g., retail location(s)) where the cashier is employed, in accordance with one implementation of the methods and system disclosed herein.

[0047] FIG. 6 provides a non-limiting example of a process flowchart for identifying individuals (e.g., cashiers) through chunk audio stream processing and biometric voiceprint comparison, in accordance with one implementation of the methods and systems disclosed herein.

[0048] FIG. 7 provides a non-limiting example of a process flowchart for comparing a real-time chunk voiceprint for an individual (e.g., a cashier) with the voiceprints for individuals (e.g., cashiers) working in a given retail location, in accordance with one implementation of the methods and systems disclosed herein.

[0049] FIG. 8 provides a non-limiting example of a process flowchart for capturing voiceprints for individuals of interest (e.g., cashiers and / or customers) during an incident at a specified location (e.g., a retail location), in accordance with one implementation of the methods and systems disclosed herein.

[0050] FIG. 9 provides a non-limiting example of a process flowchart for managing voiceprints for individuals of interest (e.g., cashiers and / or customers) and routing them to a recorder, in accordance with one implementation of the methods and systems disclosed herein.

[0051] FIG. 10 provides a non-limiting example of a process flowchart for identifying individuals (e.g., cashiers and / or customers) through chunk audio stream processing and biometric voiceprint comparison, in accordance with one implementation of the methods and systems disclosed herein.

[0052] FIG. 11 provides a non-limiting example of a process flowchart for comparing chunk voiceprints with individuals’ (e.g., cashiers’ and / or customers’) voiceprints, in accordance with one implementation of the methods and systems disclosed herein.

[0053] FIG. 12A provides a non-limiting schematic illustration of a microphone array and edge computing device where edge recording and transcription is managed by a set of processes running in a Docker container, in accordance with one implementation of the methods and systems disclosed herein.

[0054] FIG. 12B provides a non-limiting example of a process flowchart for performing edge recording and transcription using a set of processes running in a Docker container, in accordance with one implementation of the methods and systems disclosed herein.

[0055] FIG. 13 provides a non-limiting schematic illustration of the orchestration and management of an edge recording and transcription system comprising a plurality of edge computing devices, in accordance with one implementation of the methods and systems disclosed herein. Inset: a nonlimiting schematic illustration of a microphone array and edge computing device where edge recording and transcription is managed by a set of processes running in a Docker container.

[0056] FIG. 14 provides a non-limiting example of a process flowchart for using a machine learning model (e.g., a large language model (LLM) to analyze transaction conversations in a commercial setting, in accordance with one implementation of the methods and systems disclosed herein.

[0057] FIG. 15 provides a non-limiting example of a process flowchart for using large language models (LLMs) to characterize cashier engagement in countertop retail and make recommendations, in accordance with one implementation of the methods and systems disclosed herein.

[0058] FIG. 16 provides a non-limiting schematic illustration of a computing device or system in accordance with one or more examples of the methods and systems disclosed herein.DETAILED DESCRIPTION

[0059] The following description is presented to enable a person of ordinary skill in the art to make and use the various embodiments. Descriptions of specific devices, techniques, and applications are provided only as examples. Various modifications to the examples described herein will be readily apparent to those of ordinary skill in the art, and the general principles defined herein may be applied to other examples and applications without departing from the spirit and scope of the various embodiments. Thus, the various embodiments are not intended to be limited to the examples described herein and shown but are to be accorded the scope consistent with the claims.

[0060] The methods and systems described in the present disclosure address the limitations of existing methods to monitor employee-customer interactions, such as after-purchase surveys, cashier feedback, and spot-checks by managers, in commercial settings by utilizing voice biometrics and / or advanced machine learning models (e.g., large language models (LLMs)) to capture, analyze, and summarize conversations between customers and employees in real time. Through the use of LLMs, alone or in combination with voiceprint technology, the disclosed systems transcribe in-store conversations, detect key events, and generate actionable insights into employee performance, customer sentiment, and operational efficiency based on the nature of these interactions. The disclosed technology is designed to provide the same level of insight into customer-employee interactions that voice analytics provides for contact centers, allowing retailers to track not just transactions but also the conversations that influence them.

[0061] By providing a comprehensive, real-time understanding of in-store conversations, this invention enables retailers to improve both employee performance and customer experience, delivering insights that were previously unattainable in the offline retail space.Definitions

[0062] Unless otherwise defined, all of the technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art in the field to which this disclosure belongs.

[0063] As used in this specification and the appended claims, the singular forms “a”, “an”, and “the” include plural references unless the context clearly indicates otherwise. Any reference to “or” herein is intended to encompass “and / or” unless otherwise stated, and encompasses any and all possible combinations of one or more of the associated listed items.

[0064] As used herein, the terms “includes, “including,” “comprises,” and / or “comprising” specify the presence of stated features, integers, steps, operations, elements, components, and / or units but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, units, and / or groups thereof.

[0065] Throughout this application, various parameter values may be presented in a range format. It should be understood that the description in range format is merely for convenience and brevity, and should not be construed as an inflexible limitation on the scope of the disclosure. Accordingly, the description of a range should be considered to have specifically disclosed all possible subranges as well as individual numerical values within that range, irrespective of whether a specific numerical value or specific sub-range is expressly stated. For example, description of a range such as from 1 to 6 should be considered to have specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6, etc., as well as individual numbers within that range, for example, 1, 1.4, 2, 3, 3.6, 4, 5, 5.8, and 6. This applies regardless of the breadth of the range.

[0066] Numbers may be expressed herein as being “about” a particular value. Similarly, ranges may be expressed herein as from “about” one particular value and / or to “about” another particular value. The terms “about” and “approximately” shall generally mean an acceptable degree of error or variation for a given value or range of values, such as, for example, a degree of error or variation that is within 20 percent (%), within 15%, within 10%, or within 5% of a given value or range of values.

[0067] It should be recognized that use of ordinal terms such as “first” and “second” in the description of methods and systems disclosed herein does not by itself connote any priority, order of importance of one system component over another, or the temporal order in which acts of a method are performed, but are used merely as labels to distinguish, for example, one system component having a certain name from another system component having the same name but for the use of the ordinal term to distinguish the two system components.

[0068] Additionally, various implementations of the methods and systems set forth herein may be described in terms of exemplary block diagrams, process flow charts, and other illustrations. As will become apparent to one of ordinary skill in the art after reading this document, the various implementations set forth herein can be implemented without confinement to the illustrated examples. For example, block diagrams and their accompanying description should not be construedas mandating a particular architecture or configuration. Similarly, in exemplary process flow charts, some blocks are optionally combined, the order of some blocks is optionally changed, and some blocks are optionally omitted. In some implementations, additional steps may be performed in combination with the exemplary processes. Accordingly, the methods and systems as described and illustrated in greater detail below are exemplary by nature and, as such, should not be viewed as limiting.

[0069] As used herein, the term “real-time” may refer to the rate at which data is acquired, processed, and / or used, e.g., in a feedback loop to update, e.g., a prediction, a decision, a control signal, a set of instructions, or other form of output, in response to a change in one or more input data streams such that there is a minimal delay between the change in one or more input data streams and the update of the prediction, decision, control signal, set of instructions, or other form of output. In some instances, the term “real-time” may refer to the rate at which data is acquired, processed, and / or used in a feedback loop with a machine learning or artificial intelligence algorithm to update, e.g., a prediction, a decision, a control signal, a set of instructions, or other form of output, in response to a change in one or more input data streams such that there is a minimal delay between the change in one or more input data streams and the update of the prediction, decision, control signal, set of instructions, or other form of output.

[0070] As used herein, the term “machine learning” may refer to the use of any of a variety of algorithms known to those of skill in the art that may be trained to process input data and map it to a learned output, e.g., a prediction, decision, control signal, or set of instructions. In some instances, the term “artificial intelligence” may be used interchangeably with the term “machine learning”.

[0071] As used herein, the term “cloud” refers to shared or sharable storage of software and / or electronic data using, e.g., a distributed network of computer servers. In some instances, the cloud may be used, e.g., for archiving electronic data, sharing electronic data, and analyzing electronic data using one or more software packages residing locally or in the cloud.

[0072] The section headings used herein are for organizational purposes only and are not to be construed as limiting the subject matter described. The description is presented to enable one of ordinary skill in the art to make and use the invention, and is provided in the context of a patent application and its requirements.Methods and systems for identifying participants in a transaction and analyzing transaction conversations in a commercial setting

[0073] FIG. 1 provides a non-limiting example of a flowchart for a process 100 for identifying a participant and / or detecting a transaction event in a transaction conversation in a commercial setting. Process 100 can be performed, for example, as a computer-implemented method using software running on one or more processors of one or more electronic devices, computers, or computing platforms. In some examples, process 100 is performed using a client-server system, and the blocks of process 100 are divided up in any manner between the server and a client device. In other examples, the blocks of process 100 are divided up between the server and multiple client devices. Thus, while portions of process 100 are described herein as being performed by particular devices of a client-server system, it will be appreciated that process 100 is not so limited. In other examples, process 100 is performed using only a client device or only multiple client devices. In process 100, some blocks are, optionally, combined, the order of some blocks is, optionally, changed, and some blocks are, optionally, omitted. In some examples, additional steps may be performed in combination with the process 100. Accordingly, the operations as illustrated (and described in greater detail below) are exemplary by nature and, as such, should not be viewed as limiting.

[0074] At step 102 in FIG. 1, audio data is captured for a transaction conversation at a transaction location within a commercial setting.

[0075] In some instances, the commercial setting may be, for example, a retail setting (e.g., a department store, a grocery store, a drug store, an electronics store (e.g., a cellular phone, cable, television, or internet service provider store), a specialty store (e.g., a hardware store, a paint store, etc.), a fuel station, a convenience store, etc.), an automotive repair or sales setting, a bankingsetting, a food service setting (e.g., a quick service restaurant (QSR)), a government agency setting (e.g., a Department of Motor Vehicles setting), an airport check-in counter or gate, etc. In some instances, the retail setting may be a fuel station or convenience store setting.

[0076] In some instances, the transaction location may be, for example, a Point of Sale (POS) register in a retail setting, a bank teller window in a banking setting, a service window in a government agency setting, or the like.

[0077] In some instances, the audio data is captured by at least one microphone (e.g., 1, 2, 3, 4, 5, 6, or more than 6 microphones) positioned at the transaction location. In some instances, the at least one microphone may comprise a microphone array. In some instances, the at least one microphone or microphone array may be equipped with noise cancellation technology to filter out ambient noise from the commercial setting.

[0078] In some instances, the at least one microphone may comprise at least two microphones positioned in at least two transaction locations. In some instances, the at least two transaction locations may be located at a same geographical location. In some instances, the at least two transaction locations may be located at least two different geographical locations.

[0079] At step 104 in FIG. 1, a transaction conversation record is generated based on the captured audio data. In some instances, the transaction conversation record comprises, for example, a transcript of the transaction conversation and / or a voiceprint for at least one participant in the transaction conversation.

[0080] In some instances, the transaction conversation record may be generated by an edge device (e.g., an edge device interfaced with the at least one microphone) configured to perform real-time transcription of the captured audio data using a speech recognition model.

[0081] In some instances, the transaction conversation record may be generated by an edge device configured to segment the captured audio data into predefined intervals and perform real-time transcription using a Whisper model, e.g., the Whisper base. en model (OpenAI, San Francisco, CA)for English language translation, or an applicable model for other languages supported by Whisper such as Spanish, Portuguese, and French, etc.

[0082] In some instances, the transaction conversation record may be generated by an edge device configured to upload transaction conversation records and / or associated metadata to a cloud-based server. In some instances, the associated metadata may comprise, for example, transaction type data, transaction event-specific summaries, transaction log (TLOG) data, loyalty program data, sentiment evaluation scores, or any combination thereof. In some instances, for example if the commercial setting is a retail setting, the associated metadata may further comprise transaction data, timestamp data, point-of-sale (POS) register identifiers, or any combination thereof.

[0083] At step 106 in FIG. 1, the transaction conversation record is processed using at least one machine learning model, where the at least one machine learning model is configured to process the transaction conversation record and detect a transaction event.

[0084] In some instances, the machine learning model may be further configured to identify a participant in the transaction conversation. In some instances, the identification of a participant in the transaction conversation by the machine learning model may comprise a comparison of a voiceprint for a participant in the transaction conversation with a database of voiceprints to identify the participant. In some instances, the participant is identified if a match between a voiceprint for the participant to a voiceprint in the database of voiceprints exceeds a predetermined confidence threshold (e.g., a confidence threshold of 90%, 92%, 94%, 96%, 98%, or 99% confidence).

[0085] In some instances, the machine learning model may be further configured to identify a category of participant for at least one participant in the transaction conversation. For example, if the commercial setting is a retail setting, the at least one participant may be identified as an employee, a cashier, a manager, a customer, a daily customer, a weekly customer, a monthly customer, or an irregular customer.

[0086] In some instances, the machine learning model may be further configured to identify a type of transaction based on the transaction event detected in the transaction conversation record.

[0087] In some instances, the machine learning model may be further configured to generate a transaction event-specific summary of the transaction conversation.

[0088] In some instances, the machine learning model may be further configured to generate a sentiment evaluation score for the transaction conversation. In some instances, for example, the sentiment evaluation score may comprise a positive, neutral, or negative assessment of the transaction conversation.

[0089] In some instances, the machine learning model may run on a cloud-based computing platform (e.g., the Google Cloud Platform).

[0090] In some instances, the machine learning model may be trained on, for example, a training dataset comprising human-tagged audio data captured for a plurality of transaction conversations involving a plurality of different participants. In some instances, the machine learning model may be trained on a training dataset comprising domain-specific data for the commercial setting.

[0091] In some instances, the machine learning model may comprise a large language model (LLM) configured to process transaction conversation records and detect transaction events, generate transaction event-specific summaries, and / or generate sentiment evaluation scores.

[0092] FIG. 2 provides a non-limiting schematic illustration of a system for identifying a participant and / or detecting a transaction event in a transaction conversation in a commercial setting. In some implementations, the system may comprise: (i) at least one microphone (or microphone array), 202, positioned at a transaction location within a commercial setting, where the at least one microphone is configured to capture audio data from participants in a transaction conversation at the transaction location; (ii) an edge device, 204, configured to transcribe the captured audio data and generate a record of the transaction conversation; and (iii) at least one processor, 206, configured to run a machine learning model, 208, where the machine learning model 208 is configured to process the transaction conversation record and detect a transaction event. In some instances, at least a portion of the processing of the audio data and / or the processing of the transcription record by the machine learning model 208 is performed by a cloud-based computing platform, 212.

[0093] In some instances, the commercial setting may be, for example, a retail setting (e.g., a department store, a grocery store, a fuel station, a convenience store, etc.), a banking setting, a government agency setting (e.g., a Department of Motor Vehicles setting), etc. In some instances, the retail setting may be a fuel station or convenience store setting.

[0094] In some instances, the transaction location may be, for example, a Point of Sale (POS) register in a retail setting, a bank teller window in a banking setting, a service window in a government agency setting, or the like.

[0095] In some instances, the at least one microphone (e.g., at least 1, 2, 3, 4, 5, 6, or more than 6 microphones) is equipped with noise cancellation technology to fdter out ambient noise from the commercial setting. In some instances, the at least one microphone may comprise a microphone array, where the microphone array may also be equipped with noise cancellation technology.

[0096] In some instance, the system may comprise at least two microphones positioned in at least two transaction locations. In some instances, the at least two transaction locations may be located at a same geographical location. In some instances, the at least two transaction locations may be located in at least two different geographical locations.

[0097] In some instances, the edge device (also referred to herein as an edge compute device) may be configured to perform real-time transcription of the captured audio data using a speech recognition model. In some instances, the edge device may be configured to segment the captured audio data into predefined intervals and perform real-time transcription using a Whisper model (OpenAI, San Francisco, CA), e.g., the Whisper model base.en for English language translation, or an applicable model for other languages supported by Whisper such as Spanish, Portuguese, and French, etc.

[0098] In some instances, the transaction conversation record may comprise, for example, a transcript of the transaction conversation and / or a voiceprint for at least one participant in the transaction conversation.

[0099] In some instances, the machine learning model may comprise at least one machine learning model. In some instances, the machine learning model my comprise an ensemble of 1, 2, 3, 4, or more than 4 machine learning models (e.g., 2, 3, 4, or more than 4 different machine learning models) that collectively perform the processing steps required to process audio data, transcribe audio data, generate voiceprints based on the audio data and / or transcriptions, identify participants in the transaction conversation, and / or detect transaction events based on the transaction conversation.

[0100] In some instances, the machine learning model may be configured to identify a participant in the transaction conversation. In some instances, the identification of a participant in the transaction conversation by the machine learning model may comprise, for example, a comparison of a voiceprint for a participant in the transaction conversation with a database of voiceprints to identify the participant. In some instances, the participant may be identified if a match between a voiceprint for the participant to a voiceprint in the database of voiceprints exceeds a predetermined confidence threshold (e.g., a confidence threshold of 90%, 92%, 94%, 96%, 98%, or 99% confidence).

[0101] In some instances, the machine learning model may be configured to identify a category of participant for at least one participant in the transaction conversation. For example, where the commercial setting is a retail setting, the category of participant identified for the at least one participant may be an employee, a cashier, a manager, a customer, a daily customer, a weekly customer, a monthly customer, or an irregular customer.

[0102] In some instances, the machine learning model may be configured to identify a type of transaction based on the transaction event detected in the transaction conversation record. In some instances, the machine learning model may be configured to generate a transaction event-specific summary of the transaction conversation.

[0103] In some instances, the machine learning model may be configured to generate a sentiment evaluation score for the transaction conversation.

[0104] In some instances, the machine learning model may comprise a large language model (LLM) configured to process transaction conversation records and detect transaction events, generate transaction event-specific summaries, and / or generate sentiment evaluation scores.

[0105] In some instances, the machine learning model may be trained on, for example, a training dataset comprising human-tagged audio data captured for a plurality of transaction conversations involving a plurality of different participants. In some instances, the machine learning model may be trained on a training dataset comprising domain-specific data for the commercial setting.

[0106] In some instances, the system may further comprise a cloud-based server configured to store uploaded transaction conversation records and / or associated metadata. In some instances, the associated metadata may comprise transaction type data, transaction event-specific summaries, transaction log (TLOG) data, loyalty program data, sentiment evaluation scores, or any combination thereof. In some instances, for example when commercial setting is a retail setting, the associated metadata further comprise transaction data, timestamp data, point-of-sale (POS) register identifiers, or any combination thereof.Voiceprint-based identification of participants in commercial transactions

[0107] In some aspects, the present disclosure provides methods and systems for identifying individuals participating in transaction conversations in commercial settings (e.g., conversations between cashiers and customers in retail settings) through the use of voice biometrics. More specifically, the disclosed methods and systems enable the identification of employees and customers at transaction locations, such as point of sale (POS) locations or other countertop service areas, by generating unique voiceprints based on in-person transaction conversations (also referred to herein as interactions).

[0108] The disclosed methods and systems are, in some instances, based on utilizing the identifying characteristics or traits of a speaker (referred to as voice biometrics), rather than on the words spoken, to identify the speaker and / or initiate an action. Each of our voices has distinguishing features that are determined by our anatomy and behavioral speech patterns.

[0109] The disclosed methods and systems, in some instances, use voice biometrics to generate an interaction (e.g., transaction conversation) voiceprint of a participant (e.g., a cashier and / or customer) involved in a transaction at, e.g., a POS register, and then use voice biometrics to compare the interaction voiceprint to a database of previously acquired voiceprints to identify the participant.

[0110] The disclosed methods are, in some instances, enabled by systems that leverage advanced machine learning (ML) models, noise-canceling microphone arrays, and real-time voice diarization to accurately capture and process voice data in complex commercial settings. The disclosed methods and systems utilize edge computing devices (i.e., physical edge computer hardware running application software that collect audio recordings, transcribe the audio data to text, and optionally transmits the data to the cloud) to perform on-site voice analysis and diarization, while in some instances cloud infrastructure can be employed for the storage and / or further processing of transaction event data. The systems provide managers in commercial settings (e.g., retailers) with actionable insights into employee performance and consumer behavior, similar to the way online tracking tools profde users, but applied to interactions in physical commercial locations.

[0111] Process flow for voiceprint-based identification of conversation participants: A nonlimiting example of a process flow for analyzing audio recordings to identify individuals (e.g., cashiers and / or customers in retail locations) using voice biometrics is shown in FIG. 3. In this example, the process comprises three stages: (i) voice diarization using edge devices, (ii) classification of voice user type, and (iii) voice recognizable identification.

[0112] Voice diarization using edge devices: In the first step of the process, raw audio data is recorded using an edge device and, optionally, compressed (e.g., using the FLAC (http: / / xiph.org / flac) audio coding format for lossless compression of digital audio and / or the FFmpeg tool (hhp / / fynpeg org) for recording, converting, and streaming video, audio, and other multimedia data). The audio data is then segmented using voice spectrogram biometric techniques (i.e., based on differences in amplitude, volume, and / or frequency) to distinguish between different speakers’ voices to distinguish between different speakers’ voices, and the different voices identifiedcan be assigned a unique voiceprint ID for use in downstream commercial (e.g., retail) analytics. In some instances, the audio recording configuration utilizes a microphone array which performs background noise cancellation at the microphone during recording. In some instances, overlapping conversations can be separated by processing using machine learning models classify overlapping speech segments of dyadic conversations into competitive vs. non-competitive acts using lexical and acoustic cues, as well as their surrounding context. The machine learning models, trained on tagged conversation data, can intelligently separate transcripts into segments, e.g., “conversation 1” vs. “conversation n”. In some instances, all or a portion of this step of the process may be handled by local hardware. In some instances, all of a portion of the processing of audio data captured locally may be performed by a cloud-based computing platform.

[0113] Classification of voice user type: In the second step of the process, the audio data for each unique voiceprint ID is classified (e.g., through the use of a trained classifier model based on an open source voiceprint model which is then tuned to address regional or vertical contextual specifics) to assign, e.g., a User Type, Employee Type, or Customer Type tag to Voice IDs for use in downstream commercial (e.g., retail) analytics. In some instances, training a voiceprint classifier may comprise the use of a training data set of labeled voiceprints in which the label assigned to a given voiceprint is based on, e.g., the frequency with which the voiceprint is detected. For example, voiceprints detected multiple times per day in a retail setting may be labeled with a User Type label of Employee, whereas voiceprints detected no more than once per day may be labeled with a User Type of Customer. Furthermore, in some instances voiceprints labeled with a User Type of Employee may be further subdivided by a label specifying an Employee Type (such as Cashier, Manager, etc.), while voiceprints labeled with a User Type of Customer may be further subdivided based on frequency of detection. For example, customer voiceprints that are detected more than three times per week on average may be assigned a Customer Type of Loyal. In some instances, the labeling of training data is performed automatically by the system over time.

[0114] Coice recognizable identification: In the third step of the process, voiceprints for employees (e.g., cashiers) and / or customers may be either manually or automatically assigned a cashier or customer recognizable name and / or ID number. In some instances, the metadata associated with thetimestamped customer voiceprints may be integrated with corresponding TLOG data to identify the customer. In some instances, a frequency with which a given customer voiceprint is detected may qualify the customer for membership in a loyalty program (and the customer voiceprint may be further associated with a loyalty program member ID and a record of purchases).

[0115] In some instances, the disclosed methods and systems record, e.g., a cashier’s conversations or utterances with customers using edge compute devices (also referred to herein as edge devices) equipped with one or more microphones located at or near, e.g., a POS register in countertop retail locations. In some instances, the disclosed methods and systems utilize edge compute devices equipped with microphone arrays (located at or near, e.g., a POS register) which can identify the direction of the speaker in order to distinguish between the captured utterances of cashiers from those of customers. The captured audio data can be processed in real-time (or near real-time) to generate voiceprints for each participant in a given conversation. Unique voiceprints for cashiers and / or other individuals (e.g., customers) can then be compared with a database of previously generated voiceprints to identify the speaker.

[0116] In some instances, the process of recording voice samples to create a voiceprint of, e.g., a cashier, can comprise having the cashier use a mobile web app to record specific phrases as audio recording samples during their onboarding. In some instances, the mobile web app uses a biometric engine to create voiceprint templates of the cashier’s voice recording samples which are then merged to create the voiceprint of the cashier.

[0117] FIG. 4B provides a non-limiting example of a flowchart (or block diagram) for a process 400 for creating a voiceprint for an individual (e.g., a cashier). The flowchart depicted in FIG. 4B details how a mobile web app running on a mobile phone (e.g., a cashier’s mobile phone; FIG. 4A) can be used to record audio samples of the cashier for processing by a voice biometrics engine to generate a unique voiceprint for the cashier.

[0118] At step 402 in FIG. 4B, the cashier receives an onboarding email or text (Short MessageService (SMS)) from Human Resources (HR) prompting the cashier to open the mobile web app. Atstep 404, the cashier opens the mobile web app to view the mobile web app’s onboarding landing page. At step 406, the cashier registers their name and related personal information. At step 408, the cashier is prompted to repeat phrases (e.g., 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, or more than 10 different phrases suggested by the mobile app) verbally to capture voice recording samples. In some instances, the cashier may be prompted to repeat a given phrase two or more times (e.g., 2, 3, 4, 5, 6, 7, 8, 9, 10, or more than 10 times) to record two or more voice recording samples for the phrase. At step 408, the voice recording samples (audio data) are merged to generate a unique voiceprint for the cashier, as described in more detail below.

[0119] The disclosed methods and systems further comprise a set of post-recording audio processing steps that can include, but are not limited to, extracting Direction of Arrival (DoA) data, voice activity detection (VAD), and / or transcription of the audio recording data, which is then passed to a biometrics engine to create a voiceprint in real-time (or near real-time). The generated voiceprint can then be compared with a database of voiceprints for cashiers and / or customers to identify the speaker.

[0120] In some instances, edge compute devices are used for recording and / or transcription of audio data for transaction conversations recorded at transaction locations (e.g., POS registers) within a commercial setting (e.g., a retail location).

[0121] In some instances, the voiceprints for a cashier (or other employee) are routed only to the edge compute device(s) used for recording and transcription at the retail location(s) where the cashier is employed - rather than to all affiliated retail locations - to, for example, optimize processing. The processing optimization is achieved by reducing the number of voiceprints which must be compared to a voiceprint for an incoming audio stream by limiting the list of voiceprints for comparison to only the cashiers working in that location. FIG. 5 provides a flowchart for a process 500 for routing voiceprints to the appropriate edge compute device(s) at specified location(s) for use in identifying the cashier (or other employee) engaged in transactions at a location where the cashier is employed.

[0122] At step 502 in FIG. 5, a cashier’s voiceprint (e.g., for a newly employed cashier) is made available for use in matching and identification. At step 504, store ID(s) for the cashier are identified based on an HR roster. At step 506, the active cashier’s voiceprint is routed to all stored located in a district or geographical region based on the store ID(s) identified in the HR roster. At step 508, a recorder in the edge device located at each identified store receives the voiceprint.

[0123] In some instances, the voiceprints for the cashier are routed to all affiliated retail locations in a district or geographical region where the cashier is employed, and loaded into the edge compute devices used for audio recording and transcription at each location.

[0124] In some instances, a set of voiceprints used for comparison to identify a cashier may be stored in an edge compute device (or a peripheral memory / storage device) at a given retail location, and may include all relevant cashier voiceprints for the cashiers located across a district or geographical region.

[0125] In some instances, the disclosed methods and systems comprise the use of chunking the audio data stream (i.e., dividing a long audio data stream into shorter duration segments in order to reduce the probability that the shorter segment includes multiple speakers). In some instances, the disclosed methods and systems identify the speaker (e.g., a cashier or customer) speaking in chunked utterances by comparing the calculated (or generated) biometric voiceprint of the input chunked audio data with the roster of cashier voiceprints associated with the retail location.

[0126] In some instances, the process of identifying which cashier is speaking with customers is performed by the edge compute in the retail location as presented in FIG. 6. The block diagram depicted in FIG. 6 illustrates the pipeline of tasks used to process audio data, including chunking the audio and then passing the chunked audio to a voice biometrics engine to generate a chunk voiceprint for comparison to a database of voiceprints to identify the speaker. The block diagram depicted in FIG 6 also illustrates the differences between using hard chunking (e.g., equal time duration segments, such as 2 or 5 minutes), versus soft chunking (e.g., variable length time duration segments which may have a maximum length of, e.g., 5 or 10 minutes, but are then truncated whensilence is detected for over a threshold duration, such as 5 seconds). As illustrated in FIG. 6, in some instances, audio data, voiceprints, and / or part of the processing (e.g., part of the processing performed by the voice biometrics engine) may be performed in the cloud.

[0127] In some instances, the process for comparison of a real-time cashier’s voiceprint with the previously-stored voiceprints for all cashiers who work in a given retail location may be performed as depicted in FIG. 7. The process of comparing sample audio from one or more speakers to identify the specific speaker for a voice sample involves first capturing an identified voice sample from a specific speaker, and then creating a voice biometrics map of the speaker's unique characteristics. A speaker provides one or more audio samples, which the system analyzes to create a unique voiceprint for the speaker. Whenever the speaker is detected, the software compares the speaker's voiceprint to the voiceprint on file.

[0128] In more general terms, in some instances the capture and calculation (or generation) of a voiceprint for a person of interest (e.g., a cashier and / or a customer in a retail setting) during a transaction (or other incident) can be performed using a process 800 as depicted in the block diagram in FIG. 8, which details how audio data captured at the time of an incident can be used to create a real-time voiceprint of a customer (or other person of interest involved in the incident) and identify the customer (or other person of interest) using voice biometrics.

[0129] At step 802 in FIG. 8, an incident is detected in, e.g., a physical store location (e.g., as reported by a cashier and / or detected in an audio recording of utterances that occur between cashiers and shoppers at or near a POS register in the physical store location). At step 804, a determination is made as to whether the incident occurred during a purchase transaction. If the answer is “yes”, a determination is made at step 806 as to whether or not TLOG data including loyalty program and / or credit card information is available to identify the customer. If the answer at step 804 is “no”, the process proceeds to step 812 with the capture of a voiceprint for the customer during the incident. The captured voiceprint is then entered (along with appropriate metadata) to a pool (or database) of voiceprints for persons of interest. If the answer at step 1006 is “no”, then the process proceed to steps 812 and 814 as indicated in the figure. If the answer at step 806 is “yes”, then the processproceeds to a capture of a voiceprint for the customer (along with other available metadata) at step 808, and the data may be used to prosecute the customer at step 810.

[0130] In some instances, the voiceprint for the person of interest may be routed to edge compute hardware located in retail stores in a district or geographical region where an incident occurred. The block diagram depicted in FIG. 9 details a process 900 for how a voiceprint for a person of interest can be routed to the appropriate edge compute device in the retail locations for the district or geographical region.

[0131] At step 902 in FIG. 9, a voiceprint for a person of interest (along with associated metadata) is added to a pool (or database) of voiceprints. At step 904, a determination is made as to whether or not the voiceprint for the person of interest is an active voiceprint or whether it corresponds to a voiceprint that has been inactivated (e.g., removed from the pool). If the answer at step 904 is “no” (i.e., the voiceprint is inactive), the voiceprint is labeled as not authorized for retrieval at step 912, and the inactive voiceprint is removed from the edge compute device hardware located at all stores in the district or geographical region at step 914. If the answer at step 904 is “yes”, an authorized inquiry request for access to the voiceprint of the person of interest (e.g., by a manager and / or law enforcement official) may be granted at step 908, the voiceprint of the person of interest may be routed to all of the stores located in the district or geographical region where an incident occurred at step 908, where it is received by the edge compute device hardware located in the stores at step 910.

[0132] In some instances, the process of identifying a person of interest is performed by the edge compute device in the retail location as presented in the block diagram depicted in FIG. 10. The block diagram depicted in FIG. 10 illustrates the pipeline of tasks used to process audio data, including chunking the audio and then passing the chunked audio to a voice biometrics engine to generate a chunk voiceprint for comparison to a database of voiceprints of persons of interest to identify the speaker. The block diagram depicted in FIG 10 also illustrates the differences between using hard chunking (e.g., equal time duration segments, such as 2 or 5 minutes), versus soft chunking (e.g., variable length time duration segments which may have a maximum length of, e.g., 5 or 10 minutes, but are then truncated when silence is detected for over a threshold duration, such as5 seconds). As illustrated in FIG. 10, in some instances, audio data, voiceprints, and / or part of the processing (e.g., part of the processing performed by the voice biometrics engine) may be performed in the cloud.

[0133] In some instances, the process for comparison of a real-time voiceprint for a person of interest involved in the incident at a given store location with previously-stored voiceprints for persons of interest involved in incidents at any store location in the surrounding district or geographical region may be performed as depicted in FIG. 11. The process of comparing sample audio from one or more speakers to identify the specific speaker for a voice sample involves first capturing an identified voice sample from a specific speaker, and then creating a voice biometrics map of the speaker's unique characteristics. A speaker provides one or more audio samples, which the system analyzes to create a unique voiceprint for the speaker. Whenever the speaker is detected, the software compares the speaker's voiceprint to the voiceprint on file.

[0134] In some instances, the disclosed methods and apparatus may further comprise software for effectively deploying and managing the distributed recording, audio transcription and / or analytical software used to generate or process voiceprints across cloud computing and / or edge compute devices located in a plurality of commercial settings, e.g., retail locations.

[0135] In some instances, the software for managing the distributed recording, audio transcription and / or analytical software used to generate or process voiceprints is deployed on edge compute devices as one or more processes running in a Docker container. For example, in some instances, the processes included in the Docker container can comprise audio recording, transcription, network management, Wi-Fi connect, metadata associated with the audio recording, and / or the file transfer of audio data files to a cloud repository for analysis, visualization, and / or deriving recommendations based thereon. In some instances, the suite of processes included in the workload of the container may include utilities for displaying device metadata, e.g., CPU, device memory, and storage utilization, as well as device temperature among other useful data.

[0136] In some instances, deploying the software in Docker containers consumes fewer resources compared to traditional virtual machines, where the edge compute devices have limited CPU and memory.

[0137] In some instances, deploying the software in Docker containers encapsulates the software and its dependencies, thereby ensuring that the application runs consistently across different edge devices (also referred to herein as edge compute devices) and environments. This portability reduces the risk of compatibility issues and enables easy scaling of edge applications. Adding computing capacity is as easy as deploying an additional edge compute device when traffic or workloads increase, and then scale down when they decrease.

[0138] In some instances, different versions of the one or more processes activated in the Docker container for the edge compute device may be remotely managed using orchestration and management tools. This innovation distributes the heavy compute workload of recording and transcribing cashier and customer conversations which occur at, e.g., POS registers, to edge compute devices located at POS retail locations. This also reduces the telephony workloads by capturing the recording, transcribing the raw audio from the conversations, and then compressing the raw audio using edge compute devices before transmitting it from the remote location to cloud storage repositories for subsequent processing and analytics.

[0139] In some instances, deploying the software in Docker containers in conjunction with container orchestration platforms, e.g., like Kubemetes, allows one to automate the deployment, scaling, and management of containers across a fleet of edge devices, thereby ensuring high availability and load balancing.

[0140] Remote management of different versions of the recorder and transcription software in containers can be especially helpful for updating or rolling back applications on edge devices, ensuring consistency and reliability. The ability to enable roll back to previous versions or configurations quickly is important for mitigating the risk of failed updates or changes, which iscrucial in edge computing situations where reliability is paramount. This minimizes downtime and service interruptions, which is particularly important in critical edge applications.

[0141] In some instances, deploying the software in Docker containers can include providing built- in tools for monitoring and logging containerized applications. This helps in gaining insights into the performance and health of edge software, thereby aiding in troubleshooting and performance optimization.

[0142] In some instances, deploying the software in Docker containers can provide a level of security through isolation. Each container can have its own file system and processes, thereby reducing the attack surface (i.e., the total number of possible points of entry for an unauthorized user to access a system or network) compared to running multiple applications directly on a host system. This is particularly important for edge devices that may be exposed to physical threats like those potentially occurring in commercial settings such as retail locations.

[0143] FIG. 12A provides a non-limiting schematic illustration of a microphone array and edge computing device where edge recording and transcription is managed by a set of processes running in a Docker container. As depicted in FIG. 12A, the software used to capture and process audio data may reside in a Docker container comprising a recorder process, a transcription process, and / or other processes (e.g., WiFi connect).

[0144] FIG. 12B provides a non-limiting example of a flowchart for a process 1200 for performing edge recording and transcription using a set of processes running in a Docker container. At step 1202 in FIG. 12B, a conversation takes place between, e.g., a cahier an customer at a POS register. At step 1204, an edge compute device interfaced with a microphone or microphone array captures a recording of the conversation using a recorder process running in the Docker container. At step 1206, the edge compute device transcribes the captured audio data using a transcription process running in the Docker container. At step 1208, the edge compute device compresses the recording. At step 1210, the edge device sends the compressed recording and transcript to a cloud-based repository.

[0145] FIG. 13 provides a non-limiting schematic illustration of the orchestration and management of an edge recording and transcription system comprising a plurality of edge computing devices (and associated microphones or microphone arrays), where each of the individual edge computing devices may be configured as illustrated in FIG. 12A. The orchestration and management software may comprise, for example, Docker management software and / or process management software. FIG. 13, inset: a non-limiting schematic illustration of a microphone array and edge computing device where edge recording and transcription is managed by a set of processes running in a Docker container.

[0146] In some instances, the disclosed methods and systems can be used to address the observed operational issue of the “floater” or “backup” cashier who may not be logged in properly and is thus difficult to track. These are employees who would normally be assigned to another store or location in the district or geographical region. Through the use of the disclosed methods and systems, these employees are identifiable in a given location through comparison of their real-time voiceprint to previously-saved voiceprints, thereby allowing their actions to be tracked.

[0147] In some instances, the disclosed methods and systems can also be used to address another observed operational issue, i.e., that one or more cashiers may share a POS register without logging in / out between their use. In fact, in some cases the POS register may be logged into by a cashier at the beginning of the day or shift, and remain logged in to the same cashier throughout the day until the next shift change or close of business. For operational reasons, a shared POS register can make sense by saving the time. But with multiple cashiers sharing the same POS register during the day, any detail on individual cashier performance is lost. Through the use of the disclosed methods and systems, the identities of the cashiers using the same POS register, and the details of the transactions they perform, can be tracked.

[0148] The disclosed methods and systems can also be used to also address and correct the misidentification of cashiers when using TLOG data to identify the cashier, i.e., by using the voiceprint to accurately identify which cashier was involved with a specific sales transaction. Hence, the observed operational issue that the cashier involved in each transaction may be incorrectlyidentified by TLOG data as the cashier that logged in to the POS register at the beginning of the day or shift change can be corrected.

[0149] In some instances, the disclosed methods and systems may be used to enhance, e.g., operational efficiency and / or cashier engagement at the point-of-sale (POS) in a commercial setting, such as a retail location.

[0150] In some instances, the disclosed methods and systems may be used to identify a customer (or a person of interest) present at the time an incident in which a criminal act was identified occurred, e.g., in conjunction with law enforcement. For privacy purposes, the declaration of a customer as a person of interest may require jurisdictional approvals by the local authority.

[0151] In some instances, the disclosed methods and systems may be used to capture voiceprints for a plurality of persons of interest and manage a database of such voiceprints that includes relevant incident-specific metadata, such as a location of the incident, or a time at which the incident occurred.

[0152] In some instances, the disclosed methods and systems can be used to enable law enforcement or retailer-designated loss prevention professionals to initiate a search for a matching voiceprint of the person(s) of interest.

[0153] In some instances, the disclosed methods and systems may support removal of a voiceprint for a specific person of interest from a database of voiceprints for persons of interest, if deemed appropriate by law enforcement.

[0154] In summary, in some instances the disclosed methods and systems can be used to identify a person of interest involved in an incident, and enable law enforcement officers to search transaction conversation voiceprints to determine if there is a match with the voiceprint for the person of interest. If a match occurs, additional metadata associated with the transaction conversation voiceprint may be used to determine the identity of the person of interest. In some instances, the disclosed methods and systems can be used to provide law enforcement with TLOG data or Loyaltyprogram data (including, e.g., registered address and credit card information) associated with a person of interest identified by matching a transaction conversation voiceprint at the POS register to that of a customer.

[0155] Voiceprint-based system architecture & process flow: The disclosed systems enable one or more of: (i) capturing and processing speaker voiceprints (e.g., cashier and customer voiceprints) using edge devices; (ii) continuous generation of voiceprints during conversations (e.g., transaction conversations or interactions) at the point of sale (POS) or other commercial countertop service areas, (iii) using of a combination of one or more microphones (or microphone arrays) equipped with noise-cancellation and machine learning models for voice diarization and differentiation of speakers in noisy, multi-speaker environments; (iv) tagging and categorizing employees and / or customers by frequency of visits to a store and / or roles within the store; (v) integration with cloudbased platforms to handle further analysis and identification of speakers, while maintaining data privacy and security; and (vi) prioritizing high-confidence identifications, and providing flexibility for handling uncertain or ambiguous voice matches. The disclosed methods and systems integrate seamlessly with transaction logs (TLOGs) and loyalty programs, thereby providing actionable insights into cashier and / or customer behaviors that were previously unavailable in the offline commercial (e.g., retail) space.

[0156] System hardware: The system is built around edge devices, such as Raspberry Pis (Raspberry Pi Foundation, UK) or other small single-board computers, equipped with multi-array microphones (e.g., Seeed Studio’s ReSpeaker (Seeed Technology Co. Ltd., Shenzhen, China)) that are strategically positioned at POS locations or other countertop service areas. These microphones capture audio from both cashiers and customers in real time, and feature built-in noise cancellation to filter out ambient noise common in retail environments. Each microphone is linked to a specific POS register or other countertop service area, ensuring precise localization of conversations.

[0157] System software: The system uses adapted open-source models for voice diarization, which separates speaker roles (e.g., cashier vs. customer) and processes overlapping conversations. Proprietary machine learning models are employed to generate and compare voiceprints, using acombination of speech patterns, voice characteristics, and tagged training data. These models are robust enough to handle variations in background noise and overlapping dialogues.

[0158] Voiceprint generation: For each interaction (e.g., a transaction conversation), the system generates voiceprints continuously during the conversation. These voiceprints are then categorized into specific user roles, for example employee or customer:Employee Type: Identifies the speaker as a “cashier” (e.g., present at a single store) or a “manager” (e.g., present across multiple stores within the same chain).Customer Type: Classifies the customer based on the frequency of visits (e.g., “daily”, “weekly”, “monthly”, or “irregular” customers).Person-of-Interest Type: When a security event is triggered, a person of interest can be categorized into, e.g. “shoplifting”, “threatening”, “lawsuit”, or “other” categories.

[0159] Voiceprints can be stored with metadata that includes, but is not limited to, timestamps, location identifiers (e.g., register 1, food counter, etc.), interaction event type (e.g., facility issue, car wash, new loyalty program sign up, upsell, etc.), interaction summaries, and other relevant contextual information to build a robust customer profile.

[0160] Privacy and consent: Customer consent for voice recording may, for example, be implied upon entering the store (e.g., by placement of signage at the entrance informing them of ongoing audio capture). In some implementations, recording devices may only be placed in public areas of the commercial setting (e.g., store) where there is no expectation of privacy such as the register, food service area or self-checkout. Recorders are prohibited from areas with an expectation of privacy such as bathrooms, back offices, storage rooms, or other employee-only areas. The retailer owns the voice data and is responsible for managing its use, including any cooperation with law enforcement agencies. The system is designed to comply with privacy regulations, including SOC2 standards, ensuring the secure storage and processing of voice data.

[0161] Use of edge devices: In some instances, edge devices (or edge compute devices) may process the audio data locally at each store, performing voice diarization, transcription, and initial voiceprint generation in real time. Once processed, the audio data files, voiceprints, and transcriptions can, in some instances, be uploaded to a cloud-based platform (e.g., Google Cloud Platform, Microsoft Azure Platform, etc.) for long-term storage and / or further analysis. A distributed architecture enables seamless operation across multiple commercial (e.g., retail) locations, thereby allowing voiceprints to be shared within a given retailer’s account. This allows for efficient identification of employees and customers across a plurality of different stores in a chain.

[0162] Voiceprint matching algorithm: The system uses a confidence threshold for voiceprint matching. Speech embedding models are trained to turn a series of .wav samples into numeric vectors that contain a fingerprint of the speakers voice. The voiceprint matching algorithm compares the vectors of the two or more speakers to identify the match. Only voiceprints that meet or exceed the confidence threshold for voiceprint matching are processed for identification, ensuring high accuracy. Partial matches or ambiguous results are deprioritized and flagged for further review, allowing the system to handle edge cases or lower-quality audio without sacrificing overall accuracy. The system can store ambiguous matches and revisit them if future data raises confidence levels.

[0163] Voiceprint integration with TLOGs and loyalty programs: Voiceprints are linked with transaction log (TLOG) data, allowing the system to associate specific customer interactions with purchases, promotions, and loyalty membership data. For instance, if a customer is identified by their voiceprint as a loyalty member and raises a concern about a broken car wash, the system can trigger an automated SMS apology, offering a discount on their next fuel purchase. This linkage enables personalized interactions and improves customer experience based on real-time feedback.

[0164] Cashier identification: Each cashier’s voiceprint is captured using the same process as that used for other speakers, e.g., customers. Conversations are recorded during a cashier’s shift and, by virtue of the cashier being present throughout the day, will be automatically tagged as an employee. cashier. Each voiceprint is stored, and can be accessed by managers or administratorsthrough an online portal to listen to and / or tag the voiceprint with a recognizable attribute, such as cashier name or employee number. This system overcomes the limitations of conventional POS systems, where cashiers often share logins and thus make it difficult to track individual performance.

[0165] Customer identification: Customer voiceprints may be generated at every POS location or other commercial countertop service area. The system classifies these voiceprints based on the frequency of visits and links them to transaction history and / or loyalty programs. Frequent customers (e.g., daily, weekly, or monthly visitors) can be profiled, and their feedback or purchasing behavior can be used to influence marketing efforts. The system automatically links customer voiceprints with loyalty member IDs through timestamp and register matching, thereby ensuring seamless integration with loyalty and rewards programs.

[0166] Person of Interest (Pol) identification: Pol voiceprints are generated during a POS or other countertop interaction where a security or offensive event is triggered. The system classifies these voiceprints based on the frequency of visits, coincidence with events of interest, and links them to transaction history or loyalty programs. Frequent Pol visits (e.g. , daily, weekly, or monthly visitors) can be profiled, and their feedback or purchasing behavior can be used to identify the Pol for public safety applications.

[0167] Interaction Parsing to handle multiple speakers and conversations: In some instances, the system may employ a specialized phase in its Artificial Intelligence (Al) / Machine Learning (ML) pipeline called interaction parsing, which uses machine learning models to determine when unique conversations begin and end. In some instances, the interaction parsing model may use LLM embeddings, which are vector representations of words, phrases, or entire texts generated by language models; the embeddings capture the semantic meaning and represent the meaning in a high-dimensional space. This interaction parsing model can be trained on hundreds of tagged datasets, comprising labeled speaker segments when multiple conversations occur simultaneously. In addition to analyzing transcription segments, the system can factor in audio volume and frequency to gauge the relative distance of each speaker from the microphone. This allows thesystem to differentiate between speakers in simultaneous conversations and assign voiceprints accordingly.

[0168] Overlapping Conversations: The system is designed to disambiguate overlapping conversations in real-time, separating interactions into primary and secondary conversations. The machine learning models (comprising the use of embeddings to represent the meaning of a segment) can identify and segment conversations based on speaker roles, helping the system distinguish between multiple customers speaking at once or when a customer and employee engage in different conversations at the same countertop area.

[0169] Privacy and security: In some instances, the system can be designed to be compliant with, e.g., System and Organization Controls 2 (SOC2) standards, to ensure data privacy and security. Voice data can be encrypted and securely stored in the cloud, with access limited to authorized personnel. System users, e.g., retailers, can have control over how voice data is used, including the ability to delete or anonymize voiceprints. Data sharing with law enforcement can be managed by the user, e.g., retailer, to ensure that all regulatory and legal requirements are met.Use of Large Language Models (LLMs) to characterize transaction conversations in commercial settings

[0170] Customer-employee interactions are critical touchpoints that define the customer experience, particularly in retail environments. In industries such as contact centers, voice analytics are widely used to monitor and improve customer-agent interactions, providing real-time insights into customer sentiment, agent performance, and operational challenges. However, some commercial settings, such as retail environments (especially fuel and convenience stores) have lagged in their ability to capture, analyze and quantify in-store interactions (or transactions) between employees and customers in a measurable and useful way. While transaction logs (TLOGs) capture the details of purchases, they miss critical conversational data that reflects the true nature of these interactions.

[0171] Current methods to measure these interactions, such as after-purchase surveys, are often unreliable as they are rarely completed and tend to reflect only the extremes of customersatisfaction. Cashier-provided feedback is often incomplete, as cashiers may not be motivated or are unclear on where to direct feedback. Spot-checks by managers capture only a small subset of interactions since managers cannot be present at all times, and mystery shoppers provide limited insight as they represent a very small percentage of the total interactions. These methods fail to scale and do not provide real-time insights into the broader range of customer-employee engagements.

[0172] Retail employees, much like contact center agents, are the face of the business. They manage customer service, handle complaints, and influence purchasing decisions. Despite this, retail employees often face high turnover rates — exceeding 100% annually in fuel and convenience stores — similar to the 90% attrition in contact centers. These frontline workers are typically paid minimum wage, possess a high school education, and receive limited formal training, yet their interactions with customers can significantly impact the store’s overall performance and customer satisfaction. The absence of voice analytics in retail environments means retailers miss out on opportunities to track customer sentiment, employee engagement, and service quality in real time.

[0173] Current methods of data collection in retail, such as TLOGs, fall short because they only capture transactional data. While they show what was purchased, they fail to provide the context of the purchase. For example, was the customer satisfied with the product? Did the cashier influence the sale by promoting a discount or resolving a customer complaint? Without capturing the conversations themselves, valuable insights that could improve operational performance and customer satisfaction are lost. Retailers are left without the tools to understand how employees impact customer behavior or how customers feel about their shopping experience.

[0174] The methods and systems described in the present disclosure address these limitations by, in some instances, applying advanced machine learning models (e.g., large language models (LLMs)) to capture, analyze, and summarize conversations between customers and employees in real time and make personalized recommendations. Through the use of LLMs, alone or in combination with voiceprint technology as described elsewhere herein, the disclosed systems transcribe in-store conversations, detect key events, and generate actionable insights into employee performance, customer sentiment, and operational efficiency based on the nature of these interactions. Thedisclosed technology is designed to provide the same level of insight into customer-employee interactions that voice analytics provides for contact centers, allowing retailers to track not just transactions but also the conversations that influence them.

[0175] By providing a comprehensive, real-time understanding of in-store conversations, this invention enables retailers to improve both employee performance and customer experience, delivering insights that were previously unattainable in the offline retail space.

[0176] Specifically, the disclosed systems process audio recordings captured at point-of-sale (POS) locations and / or other key locations in a commercial setting (e.g., a retail setting), transcribe them into text, and apply LLMs to detect events, summarize conversations, and analyze trends. The disclosed systems further address privacy and compliance considerations by following established practices similar to those implemented for public video surveillance systems, e.g., by ensuring that recordings are made in public areas where there is no expectation of privacy.

[0177] In some instances, the machine learning-based (e.g., LLM-based) methods and systems described herein may or may not utilize all or portions of the voiceprint technology described elsewhere herein, which relies on a process of first transcribing spoken conversations to text, utilizing microphone arrays and direction of arrival (DoA) data from the audio to distinguish between speakers, e.g., a cashier or customer, generating a voice print, and using voice biometrics in an edge compute device to pattern match the voiceprint for the cashier or customer to a roster of voiceprints for cashiers and / or customers in a given district or geographical region, and then mining the conversation utterances for meaningful phrases using compound queries leveraging a SQL-like construct.

[0178] In some instances, the methods and systems described in this section of the present disclosure improve on voiceprint technology by replacement of the complex but static query methodology used in voiceprint analysis with a machine learning-based (e.g., LLM-based) approach. The former builds a complex query which attempts to identify a significant number of the optional ways of saying something specific, such as a friendly greeting by the cashier to a customer.The disclosed methods and systems can leverage LLMs to reduce development time since the approach doesn't require creating and maintaining complex database structures and SQL queries. The LLM-based approach can work with unstructured or semi-structured data (i.e., they can work with data in its natural form, thus eliminating the need for extensive data transformation and preparation steps), making the approach versatile and applicable to a wide range of recommendation scenarios such as the varied and complex conversations which occur at a retail POS register. Furthermore, LLMs can handle large volumes of data and scale easily, making them suitable for applications with high user engagement, such as conversations at a POS register in countertop retail.

[0179] FIG. 14 provides a non-limiting example of a flowchart for a process 1400 for using at least one machine learning model (e.g., a large language model (LLM) to analyze transaction conversations in a commercial setting. At step 1402, audio data is recorded (e.g., using an edge computing device as described elsewhere herein). At step 1404, the audio data us transcribed (e.g., using a Whisper model (OpenAI, San Francisco, CA), as described below). At step 1406, conversation parsing is performed (e.g., using a ML-based parsing model, as described elsewhere herein). At step 1408, an interaction (e.g., conversation) summary is generated (e.g., using an LLM, as described elsewhere herein). At step 1410, a summary of detected events is generated (e.g., using the same or a different LLM, as described elsewhere herein). At step 1412, trend and anomaly detection is performed (e.g., by tracking event frequency and patterns over time, as described elsewhere herein). At step 1414, a sum of sums (e.g., an aggregated summary of multiple conversations) is generated and output, allowing users to quickly view high-level insights across large datasets.

[0180] In some instances, the disclosed methods and systems can be adapted to the different regional conversation styles and use of colloquial expressions found in commercial settings such as countertop retail (e.g., by modifying a set of labeled conversation training data used to train the LLM), making them suitable for a wide range of recommendation tasks for improving cashier engagement. Examples of the recommendation tasks that may be performed by the LLM include, but are not limited to: (1) rather than asking if a customer is a loyalty member, instead recommending that the cashier ask if the customer wants to save money, (2) if a friendly greetingisn’t detected for a significant number of intersections, recommending that the cashier offer a friendly greeting to the next customer, (3) identifying a cashier that has gone above and beyond expectations, e.g., by mentioning a promotion in the context of a customer purchase (e.g., a customer puts two cases of beer on the counter, and the cashier mentions that it looks like the customer is having a party and points out that chips and salsa are on special).

[0181] In some instances, the disclosed methods and systems leverage LLMs to continually leam from new data, which means their recommendations can improve over time as they encounter more conversations and user interactions. Furthermore, LLMs can be trained to infer subtle features of context, allowing them to provide more contextually -relevant recommendations based on the nuances of a conversation.

[0182] In some instances, the disclosed methods and systems may be configured to analyze conversation date for conversations recorded in a commercial setting (including adaption for different regional conversation styles and colloquial expressions found in, e.g., countertop retail sales) and make recommendations for improving cashier engagement, improving customer experience, etc. FIG. 15 provides a non-limiting example of a flowchart for a process 1500 for using large language models (LLMs) to characterize cashier engagement in countertop retail and make recommendations. At step 1502 in FIG. 15, a conversation takes place between a cashier and customer at a POS register. At step 1504, a microphone array is used to record the conversation and transmit the audio data to an edge compute device at or near the POS register. At step 1506, a speech recognition model is used to transcribe the recorded audio data into text. At step 1508, a LLM parses the transcript to identify participants in the conversation and / or classify the participants (e.g., as cashiers or customers) and / or conversation (e.g., as a specific type of conversation). At step 1510, the LLM outputs one or more recommendations based on insights inferred from the conversation transcript (e.g., recommendations for improving cashier engagement, recommendations for improving customer experience, etc.).

[0183] ML-based analysis method features: As noted, the disclosed methods and systems enable the capturing, analyzing, and characterizing of in-store conversations between customers andemployees in commercial settings using machine learning models, such as large language models (LLMs). The systems are specifically designed for commercial settings, such as retail environments (e.g., fuel and convenience stores), and enable real-time analysis of customer-employee interactions.

[0184] In some implementations, the key features of the disclosed methods and systems include:• Continuous audio capture and transcription: o The system can use, e.g., edge devices, to continuously capture audio during store hours at point-of-sale (POS) locations and other relevant areas. o Audio recordings can be transcribed in real-time using, e.g., a speech recognition model, with the transcripts uploaded to a cloud-based server for further analysis. In some instances, a predetermined glossary of “threat words” can be detected in realtime to alert the authorities about a potential incident. In some instances, a command and control mode (e.g., an “Alexa”-like keyword command mode) can be used by a manager to identify technical issues (e.g., lights not on in a specified area) which need to be fixed.• LLM-based interaction analysis: o The system can apply domain-specific (e.g., commercial setting-specific, geographical setting-specific, etc.) large language models to transcribe and summarize conversations. o LLMs can be configured to detect key events during customer and employee conversations, and provide summaries of important operational details, such as product complaints, loyalty program interactions, and service gaps.Event detection and categorization:o The system can use a detailed event hierarchy to detect and categorize specific operational events, including:■ Product issues (e.g., out-of-stock issues, product quality issues, etc.)■ Facility issues (e.g. , car wash issues, bathroom complaint issues, etc.)■ Employee performance (e.g., upselling, promotions, loyalty engagement) o In some instances, the detected events may be tagged with resolution status and sentiment (e.g., was a customer’s request satisfied or not) for deeper insights.• Trend and anomaly detection: o In some instances, the system may track event frequency and event patterns over time to identify trends and anomalies. o Users (e.g., retailers) can be alerted to recurring problems or unexpected spikes in specific issues, thereby allowing them to take proactive measures to improve store performance.• “Sum of Summaries” feature: o In some instances, the system can provide an aggregated summary of multiple interactions, allowing users to quickly view high-level insights across large datasets. o This feature is particularly useful for corporate decision-makers managing multiple store locations.• Privacy and Compliance: o In some instances, the system may operate only in public areas of retail environments where there is no expectation of privacy, similar to standards applied to video surveillance systems.o In some instances, users (e.g., retailers) may be are advised to display signage notifying customers that conversations may be recorded, ensuring transparency and compliance with privacy laws.

[0185] The disclosed methods can provide users, such as retailers, with the ability to capture, analyze, and act on in-store interactions, helping to improve customer satisfaction, optimize employee performance, and enhance operational efficiency. The system’s real-time insights enable data-driven decisions that improve both service quality and business outcomes.

[0186] Al-based analysis pipeline architecture: The Al-based (or ML-based) analysis pipeline begins after an audio recording is captured on an edge device, and can proceed through several stages including, but not limited to, recording, transcription, interaction parsing, interaction summary, event detection, trend and anomaly identification, and summary aggregation.

[0187] Al-based analysis pipeline process flow: The Al-based (or ML-based) analysis process includes one or more of the following steps:

[0188] Recording: The system can be configured to record audio continuously during store hours, with an option to capture audio for specified time periods before and / or after hours when employees are still active in the store. To optimize file size and ease of segmentation, recordings are typically divided into 5 -minute segments, though the duration can be configured to range between 2 and 10 minutes. These recordings are captured by, e.g., edge devices and associated microphones or microphone arrays, installed throughout the store and processed locally for real-time insights.

[0189] Transcription: After recording, the system can transcribe the audio data in real time using, e.g., a Whisper model (OpenAI, San Francisco, CA), e.g., the Whisper base.en model for English language translation, or a model for other languages supported by Whisper such as Spanish, Portuguese, and French, etc. This speech-to-text process can occur directly on the edge device, ensuring low latency and immediate access to transcribed data. Once transcription is complete, both the audio file and the transcript can be uploaded to the cloud for further analysis and / or storage.

[0190] Interaction parsing: Upon receiving the transcript in the cloud, the system may apply machine learning models to parse the transcript and identify distinct interactions. An interaction is defined as a conversation between two or more individuals within the commercial setting (e.g., store). The machine learning (e.g., LLM) pipeline makes use of dedicated steps to ensure the improvement of results as audio samples move through the pipeline. The LLM pipeline steps start with basic transcription, but additional steps can be inserted to perform, e.g., a confidence test on the accuracy of the transcription. Inaccurate transcription could result from, e.g., bad audio data, or the presence of multi-lingual speakers. An additional technique to improve the contextual accuracy involves using Retrieval augmented generation (RAG), which enables the incorporation of verticalspecific, retail chain-specific, or other domain-specific training materials. The system tags each interaction with start and end timestamps, enabling the identification of multiple conversations within a single recording or across several recordings. This step ensures that individual interactions are separated and treated as unique data points for further analysis.

[0191] Interaction summary: Once the interactions are parsed, the system can summarize them using, e.g., a large language model (LLM). In some instances, this process may employ commercially-available and / or open source models, e.g., ChatGPT-3.5 Turbo (OpenAI, San Francisco, CA) or Claude (Anthropic, San Francisco, CA). In some instances, the process may be performed using e.g., customized open-source LLMs optimized for specific commercial setting transactions, e.g., fuel and convenience store conversations. For each transaction conversation, the LLM can, in some implementations, generate a summary of the conversation, assign a sentiment evaluation score (e.g., positive, neutral, or negative), and classify the conversation type as, e.g., either:• With Customer (i.e., customer-facing interactions), or• With Others (i.e., internal or non-customer interactions).

[0192] In some implementations, these summaries, along with the transcripts, can be stored in a vector database (e.g., Weaviate, Amsterdam, The Netherlands) to support semantic and keyword-based search functionality, thereby enabling users (e.g., retailers) to quickly find and analyze specific conversations based on content or context.

[0193] Event detection: After summarizing an interaction, the system can move to the event detection phase. Here, machine learning models (e.g., large language models (LLMs)) can be applied to detect predefined events based on a hierarchical event structure. The LLM pipeline can comprise multiple steps in addition to event detection, including voice transcription, correction of transcription errors based on, e.g., consumer packaged goods (CPG) brands, interaction parsing, trends and anomaly detection, threat word detection, etc. Following detection of transaction events, each detected event is categorized according to its event family, which represents a key operational aspect of the store environment.

[0194] A non-limiting example of an event hierarchy includes the following event families:• evt / red: reduce (e.g., fuel spills, product quality issues, etc.)• evt / eng: engage (e.g., sales tactics, loyalty program interactions, service gaps, above-and- beyond actions, etc.)• evt / exp: experience (e.g., checkout issues, long wait times, customer feedback, etc.)• evt / rtn: retain (e.g., employee altercations, training gaps, hiring inquiry responses, etc.)

[0195] For each detected event, the system can generate a specific event summary and populate additional fields, such as:Resolution: whether the event or issue was resolved.• Cashier influence: whether the cashier influenced the transaction (e.g. , through upselling, promotions, or loyalty engagement).• Sentiment: the sentiment evaluation score specific to the detected event (e.g., positive, neutral, or negative).

[0196] Customization Using RAG for chain-specific variability: The Engage event family can be highly customizable as events within this category often vary across retail settings and retail chains due to differences in, e.g., loyalty programs and promotions. In some instances, the system may use Retrieval-Augmented Generation (RAG) to dynamically tailor the event detection process to each retail setting or retail chain's specific offerings and employee expectations, for example:• evt / eng / abv (above and beyond): For events tagged as above-and-beyond behavior, such as an employee offering to carry groceries for an elderly customer, the system may use RAG to retrieve information from the chain's training manuals or Standard Operating Procedures (SOPs) to ensure that the detected behavior truly exceeds the chain’s baseline expectations.• evt / eng / svg (service gap): Similarly, for events tagged as service gaps, the system may use RAG to confirm whether the identified gap in behavior (e.g., lack of product knowledge) constitutes a breach of expected knowledge or service standards according to the chain's training materials or SOPs.• evt / eng / prm (promos): When identifying events related to promotions, the system leverages RAG to scrape the chain’s website or linked promotions database, dynamically retrieving up- to-date information on running promotions, including brands, prices, and start / end dates. This approach allows the system to accurately identify promotional engagement in real time and adjust the detection model to reflect the chain’s active offers.

[0197] Trend and anomaly detection: To identify long-term patterns and operational issues, in some instances the system applies trend models (e.g., linear regression models, Poisson distribution models) and / or anomaly detection models (e.g., K-nearest neighbors (KNN) models, isolation forest models, local outlier factor (LOF) models, time series anomaly detection, etc.) to the numerical data generated by event detection. These models analyze event frequency, location, time, and other key metrics to detect recurring issues, such as frequent product complaints, fuel spills, or security incidents. Trends may be identified based on, e.g., the number of occurrences of similar events as a function of time at a given retail location or store. Anomalies may be identified based on, e.g., adetermination that a specific location has a much lower or higher occurrence of a detected event than other, similar locations. Retailers are notified when trends or anomalies are detected, allowing them to take proactive measures to address operational inefficiencies or customer satisfaction issues before they escalate.

[0198] Summary aggregation (sum of summaries): The system can also include a “sum of summaries” feature, which aggregates data from a large number of interactions. This feature can be applied to subsets of data, whether generated through search results, trend analysis, anomaly detection, rule-based alerts, or scoring algorithms. The aggregated summary provides a high-level overview of hundreds of interactions, enabling retailers and decision-makers to glean insights from large datasets quickly and efficiently.

[0199] Customized large language model (LLM): In some instances, the disclosed systems may use a customized open-source LLM tailored specifically to, e.g., the fuel and convenience store environments. This model may be trained, for example, on a large corpus of industry-specific data (e.g., the LLM may be tuned using training data comprising thousands of human -tagged conversations that can vary by industry and geographical region due to business priorities and / or regional differences.), thereby ensuring more accurate detection of events, sentiments, and trends unique to the retail sector. By leveraging vector embeddings optimized for these interactions, the system can continuously improve its event detection, sentiment analysis, and recommendations as it leams from new data.

[0200] Privacy and security: In some instances, the system can be designed with robust privacy and security measures in mind, e.g., complying with SOC2 standards to ensure secure storage and processing of audio, transcript, and interaction data. All data may be encrypted both in transit and in storage. Retailers, for example, may be advised to display clearly visible signage at store entrances, informing customers that conversations are being recorded. This ensures that customer consent is implied upon entering the store, similar to the privacy practices used in conjunction with public video surveillance systems. Retailers can retain full control over the recorded data and can choose whether to anonymize data or share it with third parties, such as law enforcement, if necessary.

[0201] Key differentiators: Examples of key differentiators of the disclosed systems include, but are not limited to:• Domain-specific customization: In some implementations, the LLM may be specifically optimized for analyzing conversations in a specific commercial setting (e.g., fuel and convenience store interactions), thereby ensuring that event detection and insights are highly relevant and accurate.• Comprehensive event hierarchy: The system can be configured to provide granular event detection and categorization, thereby helping users, e.g., retailers, identify operational issues, employee actions, and / or customer feedback quickly and precisely.• Real-time processing: By using edge devices for transcription and initial interaction parsing, the system can be configured to deliver real-time insights with minimal latency, thereby allowing users, e.g., retailers, to respond to issues immediately.• Summarization and aggregation: The system’s ability to summarize and aggregate large datasets enables users, e.g., managers and other decision-makers, to derive insights from hundreds of interactions quickly, thereby facilitating faster, more informed decision-making.

[0202] Machine learning: Any of a variety of machine learning approaches & algorithms (where a machine learning model, as referred to herein, comprises a trained machine learning algorithm) may be used in implementing the disclosed methods and systems. For example, the machine learning model may comprise a supervised learning model (i.e., a model trained using labeled sets of training data), an unsupervised learning model (i.e., a model trained using unlabeled sets of training data), a semi-supervised learning model (i.e., a model trained using a combination of labeled and unlabeled training data), a self-supervised learning model, or any combination thereof. In some examples, the machine learning model can comprise a deep learning model (i.e., a model comprising many layers of coupled "nodes" that may be trained in a supervised, unsupervised, or semi-supervised manner). In some examples, the machine learning model can comprise a large language model (LLM) (e.g., a deep learning model based on a transformer network architecture that is trained on massive amountsof data; these networks can use self-attention to understand the relationships between, e.g., words and phrases in a sequence of text).

[0203] In some instances, one or more machine learning models (e.g., 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, or more than 10 machine learning models), or a combination thereof, may be utilized to implement the disclosed methods.

[0204] In some instances, the one or more machine learning models may comprise statistical methods for analyzing data. The machine learning models may be used for classification and / or regression of data. The machine learning models can include, for example, neural networks, support vector machines, decision trees, ensemble learning (e.g., bagging-based learning, such as random forest, and / or boosting-based learning), A nearest neighbors algorithms, linear regression-based models, and / or logistic regression-based models. The machine learning models can comprise regularization, such as LI regularization and / or L2 regularization. The machine learning models can include the use of dimensionality reduction techniques (e.g., principal component analysis, matrix factorization techniques, and / or autoencoders) and / or clustering techniques (e.g., hierarchical clustering, ic-means clustering, distribution-based clustering, such as Gaussian mixture models, or density-based clustering, such as DBSCAN or OPTICS). The one or more machine learning models can comprise solving, e.g., optimizing, an objective function over multiple iterations based on a training data set. The iterative solving approach can be used even when the machine learning model comprises a model for which there exists a closed-form solution (e.g., linear regression).

[0205] In some instances, the machine learning models can comprise artificial neural networks (ANNs), e.g., deep learning models. For example, the one or more machine learning models / algorithms used for implementing the disclosed methods may include an ANN which can comprise any of a variety of computational motifs / architectures known to those of skill in the art, including, but not limited to, feedforward connections (e.g., skip connections), recurrent connections, fully connected layers, convolutional layers, and / or pooling functions (e.g., attention, including self-attention). The artificial neural networks can comprise differentiable non-linear functions trained by backpropagation.

[0206] Artificial neural networks, e.g., deep learning models, generally comprise an interconnected group of nodes organized into multiple layers of nodes. For example, the ANN architecture may comprise at least an input layer, one or more hidden layers (i.e., intermediate layers), and an output layer. The ANN or deep learning model may comprise any total number of layers (e.g., 3, 4, 5, 6, 7, 8, 9, 10, 15, 20, or more than 20 layers in total), and any number of hidden layers (e.g., 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 15, 20, or more than 20 hidden layers), where the hidden layers function as trainable feature extractors that allow mapping of a set of input data to a preferred output value or set of output values. Each layer of the neural network comprises a plurality of nodes (e.g., at least 10, 25, 50, 75 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, 10,000, or more than 10,000 nodes). A node receives input data (e.g., genomic feature data (such as variant sequence data, methylation status data, etc.), non-genomic feature data (e.g., digital pathology image feature data), or other types of input data (e.g., patient-specific clinical data)) that comes either directly from one or more input data nodes or from the output of one or more nodes in previous layers, and performs a specific operation, e.g., a summation operation. In some cases, a connection from an input to a node is associated with a weight (or weighting factor). In some cases, the node may, for example, sum up the products of all pairs of inputs, , and their associated weights, W, In some cases, the weighted sum is offset with a bias, b. In some cases, the output of a node may be gated using a threshold or activation function, where f may be a linear or non-linear function. The activation function may be, for example, a rectified linear unit (ReLU) activation function or other function such as a saturating hyperbolic tangent, identity, binary step, logistic, arcTan, softsign, parameteric rectified linear unit, exponential linear unit, softPlus, bent identity, softExponential, Sinusoid, Sine, Gaussian, or sigmoid function, or any combination thereof.

[0207] The weighting factors, bias values, and threshold values, or other computational parameters of the neural network (or other machine learning architecture), can be "taught" or "learned" in a training phase using one or more sets of training data (e.g., 1, 2, 3, 4, 5, or more than 5 sets of training data) and a specified training approach configured to solve, e.g., minimize, a loss function. For example, the adjustable parameters for an ANN (e.g., deep learning model) may be determined based on input data from a training data set using an iterative solver (such as a gradient-basedmethod, e.g., backpropagation), so that the output value(s) that the ANN computes (e.g., a classification of a sample or a prediction of an outcome) are consistent with the examples included in the training data set. The training of the model (i.e., determination of the adjustable parameters of the model using an iterative solver) may or may not be performed using the same hardware as that used for deployment of the trained model.

[0208] In some instances, the disclosed methods may comprise retraining any of the machine learning models (e.g., iteratively retraining a previously trained model using one or more training data sets that differ from those used to train the model initially). In some instances, retraining the machine learning model may comprise using a continuous, e.g., online, machine learning model, i.e., where the model is periodically or continuously updated or retrained based on new training data. The new training data may be provided by, e.g., a single deployed local operational system, a plurality of deployed local operational systems, or a plurality of deployed, geographically- distributed operational systems. In some instances, the disclosed methods may employ, for example, pre-trained ANNs or LLMs, and the pre-trained ANNs or LLMs can be fine-tuned according to an additional dataset that is inputted into the pre-trained ANN or LLM.

[0209] Computing devices and systems: FIG. 16 illustrates an example of a computing device or system in accordance with one or more examples of the present disclosure. Device 1600 can be a host computer connected to a network. Device 1600 can be a client computer or a server. As shown in FIG. 16, device 1600 can be any suitable type of microprocessor-based device, such as a personal computer, workstation, server, or handheld computing device (portable electronic device), such as a phone or tablet. The device can include, for example, one or more of processor 1610, input device 1620, output device 1630, storage 1640, and communication device 1660. Input device 1620 and output device 1630 can generally correspond to those described above, and they can either be connectable or integrated with the computer.

[0210] Input device 1620 can be any suitable device that provides input, such as a touch screen, keyboard or keypad, mouse, or voice-recognition device. Output device 1630 can be any suitable device that provides output, such as a touch screen, haptics device, or speaker.

[0211] Storage 1640 can be any suitable device that provides storage, such as an electrical, magnetic, or optical memory including a RAM, cache, hard drive, or removable storage disk. Communication device 1660 can include any suitable device capable of transmitting and receiving signals over a network, such as a network interface chip or device. The components of the computer can be connected in any suitable manner, such as via a physical bus 1670 or wirelessly.

[0212] Software 1650, which can be stored in memory / storage 1640 and executed by processor 1610, can include, for example, the programming that embodies the functionality of the present disclosure (e.g., as embodied in the methods and systems described above).

[0213] Software 1650 can also be stored and / or transported within any non-transitory computer- readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as those described above, that can fetch instructions associated with the software from the instruction execution system, apparatus, or device and execute the instructions. In the context of this disclosure, a computer-readable storage medium can be any medium, such as storage 1640, that can contain or store programming for use by or in connection with an instruction execution system, apparatus, or device.

[0214] Software 1650 can also be propagated within any transport medium for use by or in connection with an instruction execution system, apparatus, or device, such as those described above, that can fetch instructions associated with the software from the instruction execution system, apparatus, or device and execute the instructions. In the context of this disclosure, a transport medium can be any medium that can communicate, propagate, or transport programming for use by or in connection with an instruction execution system, apparatus, or device. The transport readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, or infrared wired or wireless propagation medium.

[0215] Device 1600 may be connected to a network, which can be any suitable type of interconnected communication system. The network can implement any suitable communications protocol and can be secured by any suitable security protocol. The network can comprise networklinks of any suitable arrangement that can implement the transmission and reception of network signals, such as wireless network connections, T1 or T3 lines, cable networks, DSL, or telephone lines.

[0216] Device 1600 can implement any operating system suitable for operating on the network. Software 1650 can be written in any suitable programming language, such as C, C++, Java, or Python. In various embodiments, application software embodying the functionality of the present disclosure can be deployed in different configurations, such as in a client / server arrangement or through a web browser as a web-based application or web service, for example.NUMBERED EMBODIMENTS

[0217] Exemplary methods and systems, and computer-readable storage media are set out in the following list:1. A system comprising: at least one microphone positioned at a transaction location within a commercial setting, and configured to capture audio data from participants in a transaction conversation at the transaction location; an edge device configured to transcribe the captured audio data and generate a record of the transaction conversation; and at least one processor configured to run a machine learning model, wherein the machine learning model is configured to process the transaction conversation record and detect a transaction event.2. The system of embodiment 1, wherein the machine learning model is further configured to identify a participant in the transaction conversation.3. The system of embodiment 1 or embodiment 2, wherein the machine learning model is further configured to identify a category of participant for at least one participant in the transaction conversation.4. The system of any one of embodiments 1 to 3, wherein the machine learning model is further configured to identify a type of transaction based on the transaction event detected in the transaction conversation record.5. The system of any one of embodiments 1 to 4, wherein the machine learning model is further configured to generate a transaction event-specific summary of the transaction conversation.6. The system of any one of embodiments 1 to 5, wherein the machine learning model is further configured to generate a sentiment evaluation score for the transaction conversation.7. The system of any one of embodiments 1 to 6, further comprising a cloud-based server configured to store uploaded transaction conversation records and / or associated metadata.8. The system of embodiment 7, wherein the associated metadata comprises transaction type data, transaction event-specific summaries, transaction log (TLOG) data, loyalty program data, sentiment evaluation scores, or any combination thereof.9. The system of any one of embodiments 1 to 8, wherein the transaction conversation record comprises a transcript of the transaction conversation and / or a voiceprint for at least one participant in the transaction conversation.10. The system of any one of embodiments 1 to 9, wherein the machine learning model runs on a cloud-based computing platform.11. The system of any one of embodiments 1 to 10, wherein the at least one microphone is equipped with noise cancellation technology to filter out ambient noise from the commercial setting.12. The system of any one of embodiments 1 to 11, wherein the at least one microphone comprises a microphone array.13. The system of any one of embodiments 1 to 12, wherein the system comprises at least two microphones positioned in at least two transaction locations.14. The system of embodiment 13, wherein the at least two transaction locations are located at a same geographical location.15. The system of embodiment 13, wherein the at least two transaction locations are located in at least two different geographical locations.16. The system of any one of embodiments 1 to 15, wherein the edge device is configured to perform real-time transcription of the captured audio data using a speech recognition model.17. The system of any one of embodiments 1 to 16, wherein the edge device is configured to segment the captured audio data into predefined intervals and perform real-time transcription using a Whisper model.18. The system of any one of embodiments 1 to 17, wherein the machine learning model is trained on a training dataset comprising human-tagged audio data captured for a plurality of transaction conversations involving a plurality of different participants.19. The system of any one of embodiments 1 to 18, wherein the machine learning model is trained on a training dataset comprising domain-specific data for the commercial setting.20. The system of any one of embodiments 2 to 19, wherein identification of a participant in the transaction conversation by the machine learning model comprises a comparison of a voiceprint for a participant in the transaction conversation with a database of voiceprints to identify the participant.21. The system of embodiment 20, wherein the participant is identified if a match between a voiceprint for the participant to a voiceprint in the database of voiceprints exceeds a predetermined confidence threshold.22. The system of any one of embodiments 1 to 21, wherein the machine learning model comprises a large language model (LLM) configured to process transaction conversation records and detect transaction events, generate transaction event-specific summaries, and / or generate sentiment evaluation scores.23. The system of any one of embodiments 1 to 22, wherein the commercial setting is a retail setting.24. The system of embodiment 23, wherein the transaction location is a point-of-sale (POS) location.25. The system of any one of embodiments 3 to 24, wherein the commercial setting is a retail setting, and wherein the category of participant identified for the at least one participant is an employee, a cashier, a manager, a customer, a daily customer, a weekly customer, a monthly customer, or an irregular customer.26. The system of any one of embodiments 7 to 25, wherein the commercial setting is a retail setting, and wherein the associated metadata further comprises transaction data, timestamp data, point-of-sale (POS) register identifiers, or any combination thereof.27. The system of any one of embodiments 23 to 26, wherein the retail setting is a fuel station or convenience store setting.28. A method comprising: capturing audio data for a transaction conversation at a transaction location within a commercial setting using at least one microphone positioned at the transaction location; generating a transaction conversation record based on the captured audio data; andprocessing the transaction conversation record using at least one machine learning model, wherein the at least one machine learning model is configured to process the transaction conversation record and detect a transaction event.29. The method of embodiment 28, wherein the machine learning model is further configured to identify a participant in the transaction conversation.30. The method of embodiment 28 or embodiment 29, wherein the machine learning model is further configured to identify a category of participant for at least one participant in the transaction conversation.31. The method of any one of embodiments 28 to 30, wherein the machine learning model is further configured to identify a type of transaction based on the transaction event detected in the transaction conversation record.32. The method of any one of embodiments 28 to 31, wherein the machine learning model is further configured to generate a transaction event-specific summary of the transaction conversation.33. The method of any one of embodiments 28 to 32, wherein the machine learning model is further configured to generate a sentiment evaluation score for the transaction conversation.34. The method of any one of embodiments 28 to 33, wherein the at least one microphone is equipped with noise cancellation technology to filter out ambient noise from the commercial setting.35. The method of any one of embodiments 28 to 34, wherein the at least one microphone comprises a microphone array.36. The method of any one of embodiments 28 to 35, wherein the at least one microphone comprises at least two microphones positioned in at least two transaction locations.37. The method of embodiment 36, wherein the at least two transaction locations are located at a same geographical location.38. The method of embodiment 36, wherein the at least two transaction locations are located at least two different geographical locations.39. The method of any one of embodiments 28 to 38, wherein the transaction conversation record comprises a transcript of the transaction conversation and / or a voiceprint for at least one participant in the transaction conversation.40. The method of any one of embodiments 28 to 39, wherein the transaction conversation record is generated by an edge device configured to perform real-time transcription of the captured audio data using a speech recognition model.41. The method of any one of embodiments 28 to 40, wherein the transaction conversation record is generated by an edge device configured to segment the captured audio data into predefined intervals and perform real-time transcription using a Whisper model.42. The method of any one of embodiments 28 to 41, wherein the transaction conversation record is generated by an edge device configured to upload transaction conversation records and / or associated metadata to a cloud-based server.43. The method of embodiment 42, wherein the associated metadata comprises transaction type data, transaction event-specific summaries, transaction log (TLOG) data, loyalty program data, sentiment evaluation scores, or any combination thereof.44. The method of any one of embodiments 28 to 43, wherein the machine learning model runs on a cloud-based computing platform.45. The method of any one of embodiments 28 to 44, wherein the machine learning model is trained on a training dataset comprising human-tagged audio data captured for a plurality of transaction conversations involving a plurality of different participants.46. The method of any one of embodiments 28 to 45, wherein the machine learning model is trained on a training dataset comprising domain-specific data for the commercial setting.47. The method of any one of embodiments 29 to 46, wherein identification of a participant in the transaction conversation by the machine learning model comprises a comparison of a voiceprint for a participant in the transaction conversation with a database of voiceprints to identify the participant.48. The method of embodiment 47, wherein the participant is identified if a match between a voiceprint for the participant to a voiceprint in the database of voiceprints exceeds a predetermined confidence threshold.49. The method of any one of embodiments 28 to 48, wherein the machine learning model comprises a large language model (LLM) configured to process transaction conversation records and detect transaction events, generate transaction event-specific summaries, and / or generate sentiment evaluation scores.50. The method of any one of embodiments 28 to 49, wherein the commercial setting is a retail setting.51. The method of embodiment 50, wherein the transaction location is a point-of-sale (POS) location.52. The method of any one of embodiments 30 to 51, wherein the commercial setting is a retail setting, and wherein the category of participant identified for the at least one participant is an employee, a cashier, a manager, a customer, a daily customer, a weekly customer, a monthly customer, or an irregular customer.53. The method of any one of embodiments 42 to 52, wherein the commercial setting is a retail setting, and wherein the associated metadata further comprises transaction data, timestamp data, point-of-sale (POS) register identifiers, or any combination thereof.54. The method of any one of embodiments 50 to 53, wherein the retail setting is a fuel station or convenience store setting.55. A system for identifying participants in a transaction in a commercial setting, the system comprising: at least one microphone positioned at a transaction location within the commercial setting, and configured to capture audio data for a transaction conversation; an edge device configured to receive the captured audio data, perform diarization of the audio data, and generate a voiceprint for a participant in the transaction conversation; at least one processor configured to run a machine learning model, wherein the machine learning model is configured to classify the voiceprint generated for the participant as belonging to one of a predetermined set of participant roles; and at least one processor configured to run a matching algorithm, wherein the matching algorithm is configured to compare the voiceprint generated for the participant to a database of voiceprints associated with a participant role to which the generated voiceprint has been classified to identify the participant.56. The system of embodiment 55, further comprising a cloud-based server configured to upload, integrate, and / or store voiceprints and / or associated metadata.57. The system of embodiment 56, wherein the cloud-based server is configured to integrate voiceprints with timestamps, transaction location data, point-of-sale (POS) register identifiers, transaction log (TLOG) data, loyalty program data, or any combination thereof.58. The system of embodiment 57, wherein integration of voiceprints with timestamps, transaction location data, point-of-sale (POS) register identifiers, and / or transaction log (TLOG) data enables detailed transaction tracking and analysis.59. The system of embodiment 57, wherein integration of voiceprints with loyalty data enables automatic triggers for initiating personalized actions.60. The system of any one of embodiments 55 to 59, wherein the at least one processor configured to run the machine learning model and the at least one processor configured to run the matching algorithm are the same at least one processor.61. The system of any one of embodiments 55 to 60, wherein the at least one processor configured to run the machine learning model and the at least one processor configured to run the matching algorithm are part of a cloud-based computing platform.62. The system of any one of embodiments 55 to 61, further comprising a machine learning -based transaction parsing module configured to determine a beginning and end for each unique transaction based on transcription segments, audio volume, and participant proximity to the at least one microphone.63. The system of any one of embodiments 55 to 62, further comprising a privacy compliance feature.64. The system of embodiment 63, wherein the privacy compliance feature comprises signage displayed at an entrance to the commercial setting to inform entrants thereto that audio data is being recorded, with consent implied upon entry.65. The system of any one of embodiments 55 to 64, wherein the at least one microphone is equipped with noise cancellation technology to filter out ambient noise from the commercial setting.66. The system of any one of embodiments 55 to 65, wherein the at least one microphone comprises a microphone array.67. The system of any one of embodiments 55 to 66, wherein the system comprises at least two microphones positioned in at least two transaction locations.68. The system of embodiment 67, wherein the at least two transaction locations are located at a same geographical location.69. The system of embodiment 67, wherein the at least two transaction locations are located in at least two different geographical locations.70. The system of any one of embodiments 55 to 69, wherein the edge device is configured to perform transcription of the captured audio data in addition to voiceprint generation.71. The system of any one of embodiments 56 to 70, wherein the edge device is configured to upload transcriptions and / or voiceprints to the cloud-based server.72. The system of any one of embodiments 56 to 71, wherein the edge device is configured to process the captured audio data locally, and upload transcriptions and / or voiceprints to the cloudbased server for further analysis and / or long-term storage.73. The system of any one of embodiments 55 to 72, wherein the predetermined set of participant roles is associated with a specific commercial setting.74. The system of embodiment 73, wherein the predetermined set of participant roles comprises an employee type and a customer type.75. The system of any one of embodiments 55 to 74, wherein the machine learning model is trained on a training dataset comprising human-tagged voiceprint data generated for a plurality of transaction conversations involving a plurality of participants.76. The system of any one of embodiments 55 to 75, wherein the machine learning model is configured to separate overlapping transaction conversations and assign them to individual participants based on transcription segments and audio characteristics captured in voiceprints.77. The system of embodiment 76, wherein the audio characteristics comprise volume, amplitude, frequency, or a combination thereof.78. The system of any one of embodiments 55 to 77, wherein the machine learning model comprises a deep learning model.79. The system of any one of embodiments 55 to 78, wherein identification of the participant by the matching algorithm is based on a match between the voiceprint generated for the participant and a voiceprint in the database that satisfies a predetermined confidence score threshold.80. The system of embodiment 79, wherein the confidence score threshold is adjustable to ensure a specified level of accurate identification.81. The system of embodiment 79 or embodiment 80, wherein ambiguous matches having a confidence score of less than the predetermined confidence score threshold are flagged for further review.82. The system of embodiment 79 or embodiment 80, wherein ambiguous matches having a confidence score of less than the predetermined confidence score threshold are stored for future matching if confidence scores improve over time.83. The system of any one of embodiments 55 to 82, wherein the commercial setting is a retail setting.84. The system of any one of embodiments 55 to 83, wherein the transaction location is a point-of- sale (POS) location.85. The system of any one of embodiments 55 to 84, wherein the commercial setting is a retail setting, and wherein the predetermined set of participant roles includes an employee, a cashier, a manager, a customer, a daily customer, a weekly customer, a monthly customer, an irregular customer, or any combination thereof.86. The system of any one of embodiments 55 to 85, wherein the commercial setting is a retail setting, and wherein the personalized actions include special offers, discounts, or other notifications based on customer interactions at a point-of-sale (POS).87. The system of any one of embodiments 55 to 86, wherein the commercial setting is a fuel station or convenience store setting.88. A method for identifying participants in a transaction in a commercial setting, the method comprising: capturing audio data for a transaction conversation at a transaction location within the commercial setting using at least one microphone positioned at the transaction location; generating a voiceprint for a participant in the transaction conversation based on the captured audio data; classifying the voiceprint as belonging to one of a predetermined set of participant roles; and comparing the voiceprint generated for the participant to a database of voiceprints associated with a participant role to which the generated voiceprint has been classified to identify the participant.89. The method of embodiment 88, further comprising uploading the voiceprint and / or associated metadata to a cloud-based server.90. The method of embodiment 88 or embodiments 89, further comprising integrating the generated voiceprint with timestamps, transaction location data, point-of-sale (POS) register identifiers, transaction log (TLOG) data, loyalty program data, or any combination thereof.91. The method of any one of embodiments 88 to 90, wherein the classification of the voiceprint is performed using a machine learning model.92. The method of embodiment 91, wherein the machine learning model is configured to disambiguate overlapping transaction conversations and differentiate between simultaneous transaction conversations to ensure accurate voiceprint assignment to individual participants.93. The method of embodiment 91 or embodiment 92, wherein the machine learning model comprises a deep learning model.94. The method of any one of embodiments 88 to 93, wherein the predetermined set of participant roles is associated with a specific commercial setting.95. The system of embodiment 94, wherein the predetermined set of participant roles comprises an employee type and a customer type.96. The method of any one of embodiments 88 to 95, wherein comparison of the voiceprint generated for the participant to the database of voiceprints is performed using a matching algorithm.97. The method of embodiment 96, wherein identification of the participant by the matching algorithm is based on a match between the voiceprint generated for the participant and a voiceprint in the database that satisfies a predetermined confidence score threshold.98. The method of embodiment 97, wherein the confidence score threshold is adjustable to ensure a specified level of accurate identification.99. The method of any one of embodiments 88 to 98, wherein the at least one microphone is equipped with noise cancellation technology to filter out ambient noise from the commercial setting.100. The method of any one of embodiments 88 to 99, wherein the at least one microphone comprises a microphone array.101. The method of any one of embodiments 88 to 100, wherein the method is performed in compliance with privacy standards to ensure secure storage and processing of voiceprint data.102. The method of embodiment 101, wherein the privacy standard is SOC2.103. The method of any one of embodiments 88 to 102, wherein the commercial setting is a retail setting.104. The method of any one of embodiments 88 to 103, wherein the transaction location is a point- of-sale (POS) location.105. The method of any one of embodiments 88 to 104, wherein the commercial setting is a retail setting, and wherein the predetermined set of participant roles includes an employee, a cashier, amanager, a customer, a daily customer, a weekly customer, a monthly customer, an irregular customer, or any combination thereof.106. The method of any one of embodiments 90 to 105, wherein the commercial setting is a retail setting, and wherein integrating the generated voiceprint with transaction log (TLOG) data and loyalty program data enables linking customer transactions with specific purchases, promotions, and loyalty activities.107. The method of any one of embodiments 90 to 106, wherein the commercial setting is a retail setting, and wherein integrating the generated voiceprint with loyalty program data enables automatic triggers for initiating special offers, discounts, or other notifications based on customer interactions at a point-of-sale (POS).108. The method of any one of embodiments 88 to 107, wherein the commercial setting is a retail setting, and wherein voiceprints generated for participants in a transaction conversation are used to track and evaluate cashier performance by linking specific transaction conversations with transaction outcomes and customer feedback.109. The method of any one of embodiments 88 to 108, wherein the commercial setting is a retail setting, and wherein voiceprints generated for participants in a transaction conversation are used to enable real-time feedback mechanisms.110. The method of embodiment 109, wherein a customer complaint or customer feedback at a point-of-sale (POS) triggers an automated response.111. The method of embodiment 110, wherein the automated response comprises a special offer sent via SMS or another communication channel.112. The method of embodiment 111, wherein the special offer comprises a discount.113. The method of any one of embodiments 88 to 112, wherein the commercial setting is a fuel station or convenience store setting.114. A system for analyzing transaction conversations in a commercial setting, the system comprising: at least one microphone positioned at a transaction location within the commercial setting, and configured to capture audio data for at least one transaction conversation; an edge device configured to receive the captured audio data and perform real-time transcription of the audio data; at least one processor configured to run a first machine learning model, wherein the first machine learning model is configured to parse the transcribed audio data for the at least one transaction conversation into distinct transaction conversation transcripts; and at least one processor configured to run a second machine learning model, wherein the second machine learning model is configured to detect predefined events within the distinct transaction conversation transcripts and generate an event-specific summary of the at least one transaction conversation transcript.115. The system of embodiment 114, further comprising a cloud-based server configured to store audio data, transaction conversation transcription data, transaction conversation transcript type classifications, event-specific summaries, sentiment scores, or any combination thereof.116. The system of embodiment 114 or embodiment 115, wherein the at least one processor configured to run the first machine learning model and the at least one processor configured to run the second machine learning model are the same at least one processor.117. The system of any one of embodiments 114 to 116, wherein the at least one processor configured to run the first machine learning model and the at least one processor configured to run the second machine learning model are part of a cloud-based computing platform.118. The system of any one of embodiments 114 to 117, wherein the at least one microphone is equipped with noise cancellation technology to filter out ambient noise from the commercial setting.119. The system of any one of embodiments 114 to 118, wherein the at least one microphone comprises a microphone array.120. The system of any one of embodiments 114 to 119, wherein the system comprises at least two microphones positioned in at least two transaction locations.121. The system of embodiment 120, wherein the at least two transaction locations are located at a same geographical location.122. The system of embodiment 120, wherein the at least two transaction locations are located in at least two different geographical locations.123. The system of any one of embodiments 114 to 122, wherein the edge device is configured to upload both audio data and transcript data to a cloud-based server.124. The system of any one of embodiments 114 to 122, wherein the edge device is configured to process audio data locally and upload transcripts, transaction data, and event-specific summaries to a cloud-based server for long-term storage and / or further analysis.125. The system of any one of embodiments 114 to 124, wherein the real-time transcription of the audio data is performed using a speech recognition model.126. The system of any one of embodiments 114 to 125, wherein the edge device is further configured to segment audio recordings into predefined intervals and perform real-time transcription using a Whisper model.127. The system of any one of embodiments 114 to 126, wherein the first machine learning model is further configured to summarize each distinct transaction conversation transcript.128. The system of any one of embodiments 114 to 127, wherein the second machine learning model is further configured to detect predefined events within the distinct transaction conversation transcripts that are categorized by event families.129. The system of any one of embodiments 114 to 128, wherein the second machine learning model is further configured to generate a sentiment score for the at least one transaction conversation transcript.130. The system of any one of embodiments 114 to 129, wherein the second machine learning model is trained on domain-specific data for the commercial setting.131. The system of any one of embodiments 114 to 130, wherein the second machine learning model is trained on an event hierarchy comprising: customer experience events; employee engagement events; and / or operational issues.132. The system of embodiment 131, wherein the customer experience events include checkout issues, long wait times, customer feedback, or any combination thereof.133. The system of embodiment 131 or embodiment 132, wherein the employee engagement events comprise promotions, upselling, loyalty program engagement, or any combination thereof.134. The system of any one of embodiments 131 to 133, wherein the operational issues comprise product quality, out-of-stock issues, facility problems, or any combination thereof.135. The system of any one of embodiments 114 to 134, wherein the second machine learning model is a large language model (LLM).136. The system of embodiment 135, wherein the large language model (LLM) comprises a customized, open-source model optimized for the commercial setting.137. The system of any one of embodiments 114 to 136, further comprising a machine learningbased transaction parsing module configured to: determine the beginning and end of unique transaction conversations; and / orassign timestamps to individual transaction conversations, enabling multi-speaker separation within single or multiple recordings.138. The system of any one of embodiments 114 to 137, wherein the predefined events are associated with: a resolution status indicating whether a transaction issue was addressed; a transaction participant influence metric indicating whether or not the transaction participant’s actions influenced a transaction outcome; and / or a sentiment score specific to a transaction event.139. The system of embodiment 138, wherein the transaction participant is a cashier.140. The system of embodiment 138 or embodiment 139, wherein the sentiment score has a positive, neutral, or negative value.141. The system of any one of embodiments 115 to 140, wherein the cloud-based server further comprises a trend and anomaly detection module that: analyzes the frequency, location, and time of detected events to identify trends and anomalies across multiple transaction locations; and / or alerts commercial setting managers when recurring issues or anomalies are detected.142. The system of any one of embodiments 114 to 141, further comprising a summary aggregation feature that: aggregates event-specific summaries and event data for a plurality of transaction conversations based on user-defined parameters; and / or provides high-level summaries of transaction conversations to enable commercial setting managers to gain insights from large datasets.143. The system of embodiment 142, wherein the user-defined parameters include time frame, transaction location, geographical location, transaction event type, or any combination thereof.144. The system of any one of embodiments 114 to 143, wherein the system complies with privacy and / or data protection regulations.145. The system of embodiment 144, wherein the compliance with privacy and / or data protection regulations comprises a display of clearly visible signage informing customers that conversations may be recorded.146. The system of embodiment 144 or embodiment 145, wherein the compliance with privacy and / or data protection regulations comprises encryption of all audio data, transcription data, and transaction data.147. The system of any one of embodiments 114 to 146, wherein the commercial setting is a retail setting.148. The system of any one of embodiments 114 to 147, wherein the transaction location is a point- of-sale (POS) location.149. The system of any one of embodiments 114 to 148, wherein the commercial setting is a fuel station or convenience store setting.150. A method for analyzing transaction conversations in a commercial setting, the method comprising: capturing audio data for at least one transaction conversation at a transaction location within a commercial setting using at least one microphone positioned at the transaction location; transcribing the captured audio data for the at least one transaction conversation in real-time; parsing the transcribed audio data for the at least one transaction conversation into distinct transaction conversation transcripts using a first machine learning model; anddetecting predefined events within the distinct transaction conversation transcripts and generating an event-specific summary of the at least one transaction conversation transcript using a second machine learning model.151. The method of embodiment 150, wherein the at least one microphone is equipped with noise cancellation technology to filter out ambient noise from the commercial setting.152. The method of embodiment 150 or embodiment 151, wherein the at least one microphone comprises a microphone array.153. The method of any one of embodiments 150 to 152, wherein the system comprises at least two microphones positioned in at least two transaction locations.154. The method of embodiment 153, wherein the at least two transaction locations are located at a same geographical location.155. The method of embodiment 153, wherein the at least two transaction locations are located in at least two different geographical locations.156. The method of any one of embodiments 150 to 155, wherein the transcription of the captured audio data is performed using an edge device.157. The method of any one of embodiments 150 to 156, wherein the real-time transcription of the captured audio data is performed using a speech recognition model.158. The method of embodiment 157, wherein the speech recognition model comprises using a Whisper base. en model.159. The method of any one of embodiments 150 to 158, wherein parsing the transcribed audio data using the first machine learning model further comprises assigning timestamps to distinct transaction conversation transcripts.160. The method of any one of embodiments 150 to 159, further comprising use of a large language model (LLM) to summarize each transaction conversation.161. The method of embodiment 160, wherein the large language model (LLM) is trained on domain-specific data for the commercial setting.162. The method of any one of embodiments 150 to 161, wherein the second machine learning model is further configured to detect predefined events based on an event hierarchy and categorize them based on event families.163. The method of any one of embodiments 150 to 162, further comprising using a trend model and / or an anomaly detection model to: analyze a frequency and distribution of detected events across a plurality of transaction locations and / or geographical locations; and / or identify recurring operational issues, service gaps, and / or customer feedback trends.164. The method of any one of embodiments 150 to 163, wherein the second machine learning model is further configured to generate a sentiment score and resolution status for each detected event.165. The method of embodiment 164, wherein the second machine learning model is further configured to associate a detected event with an employee action.166. The method of embodiment 165, wherein the employee action comprises a cashier influence on a customer transaction.167. The method of any one of embodiments 150 to 166, further comprising providing a summary that aggregates data for a plurality of transaction conversations and generates high-level insights for commercial setting managers based on user-defined criteria.168. The method of any one of embodiments 150 to 167, further comprising ensuring compliance with privacy regulations and or data security regulations.169. The method of embodiment 168, wherein ensuring compliance with privacy and / or data security regulations comprises displaying signage at commercial location entrances to inform customers that transaction conversations will be recorded.170. The method of embodiment 168 or embodiment 169, wherein ensuring compliance with privacy and / or data security regulations comprises encrypting all audio data, transcription data, and / or transaction data for secure storage and / or transmission.171. The method of any one of embodiments 150 to 170, wherein the commercial setting is a retail setting.172. The method of any one of embodiments 150 to 171, wherein the transaction location is a point- of-sale (POS) location.173. The method of any one of embodiments 150 to 172, wherein the commercial setting is a fuel station or convenience store setting.EXAMPLES

[0218] The following examples are included for illustrative purposes only and are not intended to limit the scope of the present disclosure.Example 1 - Typical Countertop Retail Store Cashier’s Use of POS Registers

[0219] This prophetic example provides an illustration of the use of the disclosed methods and systems by a retailer during a typical busy day in which multiple cashiers are on duty concurrently and are available to ring up customers’ purchases and accept payment using one of the store’s POS registers. For operational efficiency, cashiers often share a single POS register login to ring up customers’ orders (despite each having their own unique login) to avoid having to repeatedlylogin / logout each time they swap using the register. Unfortunately, this practice obscures which of the cashiers performed which customer transaction.

[0220] The disclosed methods and systems address this problem by using cashier voiceprints to match payment transactions with the voiceprint of a specific cashier, and thereby properly identify which cashier was involved with what activity.Example 2 - Identification of a Person of Interest Involved in an Incident

[0221] This prophetic example illustrates how the disclosed methods and systems can be used to identify a person of interest involved in a specific incident. An incident occurs in the retail store and law enforcement wants to identify the person of interest.

[0222] The disclosed methods and systems provide a solution for discovering the identity of the person of interest by comparing their voiceprint (generated in real-time) against those of other voiceprints captured over a period of time which may include associated identifying metadata available which can be used to identify the person of interest.Example 3 - Remote Management and Orchestration of Software Updates

[0223] This prophetic example illustrates how the disclosed methods and systems can be used to manage biometric software updates remotely. An edge compute device running recorder and transcription software captures the conversations between cashiers in a countertop retail setting and their customers using microphone arrays that can detect the direction of the speaker and thereby identify whether the speaker is a cashier or customer. The planned introduction of an updated version of the software could include biometrics analytics which could be used to identify which cashier is speaking using voiceprint technology. Using centralized version management and orchestration, the updated software including the biometrics feature can be scheduled for deployment to the remote edge compute device containers, and the status of each remote edge compute device container update would be centrally available. With this orchestration approach, the operations team has direct insight into what the status is and which software version is running oneach remote edge compute device container. The benefit of this orchestration approach is that it enable effective troubleshooting in the event that an issue arises, based on fully understanding which version(s) of the applicable software are running.Example 4 - Monitoring Cashier Engagement with Customers

[0224] This prophetic example illustrates how the disclosed methods and systems can be used to monitor cashier engagement with customers. A customer approaches a cashier in a countertop retailer to pay for their items. One test for measuring how engaged the cashier is with customers is to monitor how the cashier greets customers. The greeting may range from, e.g., “welcome”, to “hi”, to “how are you today”, or to “next” in the case of a long line of customers waiting to buy lottery tickets when the lottery prize in huge. There are many more possible greetings than those just mentioned, and these greetings will differ regionally as well as by the demographics of those involved. Hence to measure how well a cashier is engaged with customers is difficult because a friendly greeting may have been missed by an incomplete SQL-like query. Significantly more accurate identification of friendly greetings from the cashier can be enabled by using adaptable LLMs trained on domain-specific data.

[0225] It should be understood from the foregoing that, while particular implementations of the disclosed methods, devices, and systems have been illustrated and described, various modifications can be made thereto and are contemplated herein. It is also not intended that the invention be limited by the specific examples provided within the specification. While the invention has been described with reference to the aforementioned specification, the descriptions and illustrations of the preferable embodiments herein are not meant to be construed in a limiting sense. Furthermore, it shall be understood that all aspects of the invention are not limited to the specific depictions, configurations or relative proportions set forth herein which depend upon a variety of conditions and variables. Various modifications in form and detail of the embodiments of the invention will be apparent to a person skilled in the art. It is therefore contemplated that the invention shall also cover any such modifications, variations and equivalents.

Claims

CLAIMSWhat is claimed is:

1. A system comprising: at least one microphone positioned at a transaction location within a commercial setting, and configured to capture audio data from participants in a transaction conversation at the transaction location; an edge device configured to transcribe the captured audio data and generate a record of the transaction conversation; and at least one processor configured to run a machine learning model, wherein the machine learning model is configured to process the transaction conversation record and detect a transaction event.

2. The system of claim 1, wherein the machine learning model is further configured to identify a participant in the transaction conversation.

3. The system of claim 1 or claim 2, wherein the machine learning model is further configured to identify a category of participant for at least one participant in the transaction conversation.

4. The system of any one of claims 1 to 3, wherein the machine learning model is further configured to identify a type of transaction based on the transaction event detected in the transaction conversation record.

5. The system of any one of claims 1 to 4, wherein the machine learning model is further configured to generate a transaction event-specific summary of the transaction conversation.

6. The system of any one of claims 1 to 5, wherein the machine learning model is further configured to generate a sentiment evaluation score for the transaction conversation.

7. The system of any one of claims 1 to 6, further comprising a cloud-based server configured to store uploaded transaction conversation records and / or associated metadata.

8. The system of claim 7, wherein the associated metadata comprises transaction type data, transaction event-specific summaries, transaction log (TLOG) data, loyalty program data, sentiment evaluation scores, or any combination thereof.

9. The system of any one of claims 1 to 8, wherein the transaction conversation record comprises a transcript of the transaction conversation and / or a voiceprint for at least one participant in the transaction conversation.

10. The system of any one of claims 1 to 9, wherein the machine learning model runs on a cloudbased computing platform.

11. The system of any one of claims 1 to 10, wherein the at least one microphone is equipped with noise cancellation technology to filter out ambient noise from the commercial setting.

12. The system of any one of claims 1 to 11, wherein the at least one microphone comprises a microphone array.

13. The system of any one of claims 1 to 12, wherein the system comprises at least two microphones positioned in at least two transaction locations.

14. The system of claim 13, wherein the at least two transaction locations are located at a same geographical location.

15. The system of claim 13, wherein the at least two transaction locations are located in at least two different geographical locations.

16. The system of any one of claims 1 to 15, wherein the edge device is configured to perform realtime transcription of the captured audio data using a speech recognition model.

17. The system of any one of claims 1 to 16, wherein the edge device is configured to segment the captured audio data into predefined intervals and perform real-time transcription using a Whisper model.

18. The system of any one of claims 1 to 17, wherein the machine learning model is trained on a training dataset comprising human-tagged audio data captured for a plurality of transaction conversations involving a plurality of different participants.

19. The system of any one of claims 1 to 18, wherein the machine learning model is trained on a training dataset comprising domain-specific data for the commercial setting.

20. The system of any one of claims 2 to 19, wherein identification of a participant in the transaction conversation by the machine learning model comprises a comparison of a voiceprint for a participant in the transaction conversation with a database of voiceprints to identify the participant.

21. The system of claim 20, wherein the participant is identified if a match between a voiceprint for the participant to a voiceprint in the database of voiceprints exceeds a predetermined confidence threshold.

22. The system of any one of claims 1 to 21, wherein the machine learning model comprises a large language model (LLM) configured to process transaction conversation records and detect transaction events, generate transaction event-specific summaries, and / or generate sentiment evaluation scores.

23. The system of any one of claims 1 to 22, wherein the commercial setting is a retail setting.

24. The system of claim 23, wherein the transaction location is a point-of-sale (POS) location.

25. The system of any one of claims 3 to 24, wherein the commercial setting is a retail setting, and wherein the category of participant identified for the at least one participant is an employee, a cashier, a manager, a customer, a daily customer, a weekly customer, a monthly customer, or an irregular customer.

26. The system of any one of claims 7 to 25, wherein the commercial setting is a retail setting, and wherein the associated metadata further comprises transaction data, timestamp data, point-of-sale (POS) register identifiers, or any combination thereof.

27. The system of any one of claims 23 to 26, wherein the retail setting is a fuel station or convenience store setting.

28. A method comprising: capturing audio data for a transaction conversation at a transaction location within a commercial setting using at least one microphone positioned at the transaction location; generating a transaction conversation record based on the captured audio data; and processing the transaction conversation record using at least one machine learning model, wherein the at least one machine learning model is configured to process the transaction conversation record and detect a transaction event.

29. The method of claim 28, wherein the machine learning model is further configured to identify a participant in the transaction conversation.

30. The method of claim 28 or claim 29, wherein the machine learning model is further configured to identify a category of participant for at least one participant in the transaction conversation.

31. The method of any one of claims 28 to 30, wherein the machine learning model is further configured to identify a type of transaction based on the transaction event detected in the transaction conversation record.

32. The method of any one of claims 28 to 31, wherein the machine learning model is further configured to generate a transaction event-specific summary of the transaction conversation.

33. The method of any one of claims 28 to 32, wherein the machine learning model is further configured to generate a sentiment evaluation score for the transaction conversation.

34. The method of any one of claims 28 to 33, wherein the at least one microphone is equipped with noise cancellation technology to filter out ambient noise from the commercial setting.

35. The method of any one of claims 28 to 34, wherein the at least one microphone comprises a microphone array.

36. The method of any one of claims 28 to 35, wherein the at least one microphone comprises at least two microphones positioned in at least two transaction locations.

37. The method of claim 36, wherein the at least two transaction locations are located at a same geographical location.

38. The method of claim 36, wherein the at least two transaction locations are located at least two different geographical locations.

39. The method of any one of claims 28 to 38, wherein the transaction conversation record comprises a transcript of the transaction conversation and / or a voiceprint for at least one participant in the transaction conversation.

40. The method of any one of claims 28 to 39, wherein the transaction conversation record is generated by an edge device configured to perform real-time transcription of the captured audio data using a speech recognition model.

41. The method of any one of claims 28 to 40, wherein the transaction conversation record is generated by an edge device configured to segment the captured audio data into predefined intervals and perform real-time transcription using a Whisper model.

42. The method of any one of claims 28 to 41, wherein the transaction conversation record is generated by an edge device configured to upload transaction conversation records and / or associated metadata to a cloud-based server.

43. The method of claim 42, wherein the associated metadata comprises transaction type data, transaction event-specific summaries, transaction log (TLOG) data, loyalty program data, sentiment evaluation scores, or any combination thereof.

44. The method of any one of claims 28 to 43, wherein the machine learning model runs on a cloudbased computing platform.

45. The method of any one of claims 28 to 44, wherein the machine learning model is trained on a training dataset comprising human-tagged audio data captured for a plurality of transaction conversations involving a plurality of different participants.

46. The method of any one of claims 28 to 45, wherein the machine learning model is trained on a training dataset comprising domain-specific data for the commercial setting.

47. The method of any one of claims 29 to 46, wherein identification of a participant in the transaction conversation by the machine learning model comprises a comparison of a voiceprint for a participant in the transaction conversation with a database of voiceprints to identify the participant.

48. The method of claim 47, wherein the participant is identified if a match between a voiceprint for the participant to a voiceprint in the database of voiceprints exceeds a predetermined confidence threshold.

49. The method of any one of claims 28 to 48, wherein the machine learning model comprises a large language model (LLM) configured to process transaction conversation records and detect transaction events, generate transaction event-specific summaries, and / or generate sentiment evaluation scores.

50. The method of any one of claims 28 to 49, wherein the commercial setting is a retail setting.

51. The method of claim 50, wherein the transaction location is a point-of-sale (POS) location.

52. The method of any one of claims 30 to 51, wherein the commercial setting is a retail setting, and wherein the category of participant identified for the at least one participant is an employee, a cashier, a manager, a customer, a daily customer, a weekly customer, a monthly customer, or an irregular customer.

53. The method of any one of claims 42 to 52, wherein the commercial setting is a retail setting, and wherein the associated metadata further comprises transaction data, timestamp data, point-of-sale (POS) register identifiers, or any combination thereof.

54. The method of any one of claims 50 to 53, wherein the retail setting is a fuel station or convenience store setting.

Images