Environmentally collaborative intelligent system and method

Abstract

A method, computer program product, and computing system for generating, via a video recording subsystem of an ACI calibration platform, a three-dimensional model of at least a portion of a three-dimensional space containing an ACI system; and generating, via an audio generation subsystem of the ACI calibration platform, one or more audio calibration signals for receipt by an audio recording system included within the ACI system.

Description

[0001] Related applications

[0002] This application claims the benefit of U.S. Non-Provisional Application No. 17 / 077,863, filed October 22, 2020, which claims the benefit of U.S. Provisional Application No. 62 / 988,337, filed March 11, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to intelligent systems and methods, and more particularly, to environmental collaborative intelligent systems and methods. Background Art

[0004] As known in the art, collaborative intelligence is the creation of reports and documents detailing the history of events / individuals. As expected, traditional documents include various types of data, examples of which may include, but are not limited to, paper documents and transcripts, as well as various images and charts.

[0005] As the world moves from paper to digital content, traditional documentation is also moving in this direction, with reports and documents gradually transitioning from geographically dispersed paper files across multiple locations / institutions to consolidated and easily accessible digital content. Summary of the Invention

[0006] In one implementation, a computer-implemented method is executed on a computing device and includes: generating, via a video recording subsystem of an ACI calibration platform, a three-dimensional model of at least a portion of a three-dimensional space that contains an ACI system; and generating, via an audio generation subsystem of the ACI calibration platform, one or more audio calibration signals for receipt by an audio recording system included within the ACI system.

[0007] One or more of the following features may be included. The ACI calibration platform can be autonomously positioned in a three-dimensional space via a mobile base assembly of the ACI calibration platform. At least a portion of the three-dimensional space can be autonomously cleaned via a cleaning assembly of the ACI calibration platform. The ACI calibration platform can be configured to be manually positioned in a three-dimensional space. The video recording system of the ACI calibration platform can be configured to interface with an object data source that defines a plurality of objects that can be located in the three-dimensional space. The three-dimensional model can be configured to define at least one of the following: one or more subspaces within the three-dimensional space; one or more objects within the three-dimensional space; one or more features within the three-dimensional space; one or more interaction zones within the three-dimensional space; and one or more noise sources within the three-dimensional space. The one or more audio calibration signals may include one or more of the following: a noise signal; a sinusoidal signal; and a multi-frequency signal.

[0008] In another implementation, a computer program product resides on a computer-readable medium and has a plurality of instructions stored thereon. The instructions, when executed by a processor, cause the processor to perform operations including: generating, via a video recording subsystem of an ACI calibration platform, a three-dimensional model of at least a portion of a three-dimensional space, the three-dimensional space containing an ACI system; and generating, via an audio generation subsystem of the ACI calibration platform, one or more audio calibration signals for receipt by an audio recording system included in the ACI system.

[0009] One or more of the following features may be included. The ACI calibration platform can be autonomously positioned in a three-dimensional space via a mobile base assembly of the ACI calibration platform. At least a portion of the three-dimensional space can be autonomously cleaned via a cleaning assembly of the ACI calibration platform. The ACI calibration platform can be configured to be manually positioned in a three-dimensional space. The video recording system of the ACI calibration platform can be configured to interface with an object data source that defines a plurality of objects that can be located in the three-dimensional space. The three-dimensional model can be configured to define at least one of the following: one or more subspaces within the three-dimensional space; one or more objects within the three-dimensional space; one or more features within the three-dimensional space; one or more interaction zones within the three-dimensional space; and one or more noise sources within the three-dimensional space. The one or more audio calibration signals may include one or more of the following: a noise signal; a sinusoidal signal; and a multi-frequency signal.

[0010] In another implementation, a computing system includes a processor and a memory configured to perform operations including: generating, via a video recording subsystem of an ACI calibration platform, a three-dimensional model of at least a portion of a three-dimensional space that contains an ACI system; and generating, via an audio generation subsystem of the ACI calibration platform, one or more audio calibration signals for receipt by an audio recording system included within the ACI system.

[0011] One or more of the following features may be included. The ACI calibration platform can be autonomously positioned in a three-dimensional space via a mobile base assembly of the ACI calibration platform. At least a portion of the three-dimensional space can be autonomously cleaned via a cleaning assembly of the ACI calibration platform. The ACI calibration platform can be configured to be manually positioned in a three-dimensional space. The video recording system of the ACI calibration platform can be configured to interface with an object data source that defines a plurality of objects that can be located in the three-dimensional space. The three-dimensional model can be configured to define at least one of the following: one or more subspaces within the three-dimensional space; one or more objects within the three-dimensional space; one or more features within the three-dimensional space; one or more interaction regions within the three-dimensional space; and one or more noise sources within the three-dimensional space. The one or more audio calibration signals may include one or more of the following: a noise signal; a sinusoidal signal; and a multi-frequency signal.

[0012] The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a schematic diagram of an ambient collaborative intelligent computing system and an ambient collaborative intelligent process coupled to a distributed computing network;

[0014] Figure 2 is included Figure 1 Schematic diagram of the modular ACI system of the environmental collaborative intelligent computing system;

[0015] Figure 3 is included in Figure 2 Schematic diagram of a hybrid media ACI device within a modular ACI system;

[0016] Figure 4 yes Figure 1 A flowchart of an implementation of an environmental collaborative intelligence process;

[0017] Figure 5 yes Figure 1 A flowchart of another implementation of the environmental collaborative intelligence process;

[0018] Figure 6 yes Figure 1 A flowchart of another implementation of the environmental collaborative intelligence process;

[0019] Figure 7 yes Figure 1 A flowchart of another implementation of the environmental collaborative intelligence process;

[0020] Figure 8 It is a schematic diagram of the ACI calibration platform;

[0021] Figure 9 is Figure 8 A flowchart of one implementation of a process performed by the ACI calibration platform; and

[0022] Figure 10 yes Figure 1 Flowchart of another implementation of the environmental collaborative intelligence process.

[0023] Like reference numerals in the various drawings represent like elements. DETAILED DESCRIPTION

[0024] System Overview

[0025] refer to Figure 1, shows an ambient collaborative intelligent process 10. As will be discussed in more detail below, the ambient collaborative intelligent process 10 can be configured to automate the collection and processing of meeting information to generate / store / distribute reports.

[0026] The environmental collaborative intelligent process 10 can be implemented as a server-side process, a client-side process, or a hybrid server-side / client-side process. For example, the environmental collaborative intelligent process 10 can be implemented as a pure server-side process via the environmental collaborative intelligent process 10s. Alternatively, the environmental collaborative intelligent process 10 can be implemented as a pure client-side process via one or more of the following: environmental collaborative intelligent process 10c1, environmental collaborative intelligent process 10c2, environmental collaborative intelligent process 10c3, and environmental collaborative intelligent process 10c4. Still alternatively, the environmental collaborative intelligent process 10 can be implemented as a hybrid server-side / client-side process via the environmental collaborative intelligent process 10s in combination with one or more of the following: environmental collaborative intelligent process 10c1, environmental collaborative intelligent process 10c2, environmental collaborative intelligent process 10c3, and environmental collaborative intelligent process 10c4.

[0027] Accordingly, the ambient collaborative intelligent process 10 used in the present disclosure may include any combination of the ambient collaborative intelligent process 10s, the ambient collaborative intelligent process 10c1, the ambient collaborative intelligent process 10c2, the ambient collaborative intelligent process 10c3, and the ambient collaborative intelligent process 10c4.

[0028] The ambient collaborative intelligence process 10s may be a server application and may reside on and be executed by an ambient collaborative intelligence (ACI) computing system 12, which may be connected to a network 14 (e.g., the Internet or a local area network). The ACI computing system 12 may include various components, examples of which may include, but are not limited to, a personal computer, a server computer, a series of server computers, a minicomputer, a mainframe computer, one or more network attached storage (NAS) systems, one or more storage area network (SAN) systems, one or more platform as a service (PaaS) systems, one or more infrastructure as a service (IaaS) systems, one or more software as a service (SaaS) systems, a cloud-based computing system, and a cloud-based storage platform.

[0029] As is known in the art, a SAN may include one or more of the following: a personal computer, a server computer, a series of server computers, a minicomputer, a mainframe computer, a RAID device, and a NAS system. The various components of the ACI computing system 12 may execute one or more operating systems, examples of which may include, but are not limited to, Microsoft Windows Server 2003, Microsoft Windows Server 2003, and Microsoft Windows Server 2003. tm; Redhat Linux tm , Unix, or a custom operating system.

[0030] The instruction sets and subroutines of the context collaborative intelligence process 10s, which may be stored on a storage device 16 coupled to the ACI computing system 12, may be executed by one or more processors (not shown) and one or more memory architectures (not shown) included within the ACI computing system 12. Examples of the storage device 16 may include, but are not limited to: a hard drive; a RAID device; random access memory (RAM); a read-only memory (ROM); and various forms of flash memory storage devices.

[0031] Network 14 may be connected to one or more auxiliary networks (eg, network 18), examples of which may include, but are not limited to, a local area network; a wide area network; or an intranet, for example.

[0032] Various IO requests (e.g., IO requests 20) may be sent from ambient collaborative intelligent process 10s, ambient collaborative intelligent process 10c1, ambient collaborative intelligent process 10c2, ambient collaborative intelligent process 10c3, and / or ambient collaborative intelligent process 10c4 to ACI computing system 12. Examples of IO requests 20 may include, but are not limited to, data write requests (i.e., requests to write content to ACI computing system 12) and data read requests (i.e., requests to read content from ACI computing system 12).

[0033] The instruction sets and subroutines of the ambient collaborative intelligence process 10c1, the ambient collaborative intelligence process 10c2, the ambient collaborative intelligence process 10c3, and / or the ambient collaborative intelligence process 10c4, which may be stored on storage devices 20, 22, 24, 26 (respectively) coupled to the ACI client electronic devices 28, 30, 32, 34 (respectively), may be executed by one or more processors (not shown) and one or more memory architectures (not shown) included in the ACI client electronic devices 28, 30, 32, 34 (respectively). The storage devices 20, 22, 24, 26 may include, but are not limited to: a hard drive; an optical drive; a RAID device; a random access memory (RAM); a read-only memory (ROM), and various forms of flash memory storage devices. Examples of ACI client electronic devices 28, 30, 32, 34 may include, but are not limited to, personal computing devices 28 (e.g., smartphones, personal digital assistants, laptops, notebook computers, and desktop computers), audio input devices 30 (e.g., handheld microphones, lavalier microphones, embedded microphones (such as those embedded in glasses, smartphones, tablet computers, and / or watches), and audio recording devices), display devices 32 (e.g., tablet computers, computer monitors, and smart TVs), machine vision input devices 34 (e.g., RGB imaging systems, infrared imaging systems, ultraviolet imaging systems, laser imaging systems, SONAR imaging systems, RADAR imaging systems, and thermal imaging systems), hybrid devices (e.g., a single device that includes the functionality of one or more of the above-referenced devices; not shown), audio presentation devices (e.g., a speaker system, a headphone system, or an earbud system; not shown), various medical devices (e.g., medical imaging devices, cardiac monitors, weight scales, thermometers, and blood pressure machines; not shown), and dedicated network devices (not shown).

[0034] Users 36, 38, 40, 42 may access ACI computing system 12 directly through network 14 or through auxiliary network 18. Additionally, ACI computing system 12 may be connected to network 14 through auxiliary network 18, as shown by link 44.

[0035] Various ACI client electronic devices (e.g., ACI client electronic devices 28, 30, 32, 34) can be coupled directly or indirectly to network 14 (or network 18). For example, personal computing device 28 is shown as being directly coupled to network 14 via a hardwired network connection. Additionally, machine vision input device 34 is shown as being directly coupled to network 18 via a hardwired network connection. Audio input device 30 is shown as being wirelessly coupled to network 14 via a wireless communication channel 46 established between audio input device 30 and a wireless access point (i.e., WAP) 48, which is shown as being directly coupled to network 14. WAP 48 can be, for example, an IEEE 802.11a, 802.11b, 802.11g, 802.11h, Wi-Fi, and / or Bluetooth device capable of establishing a wireless communication channel 46 between audio input device 30 and WAP 48. The display device 32 is shown as wirelessly coupled to the network 14 via a wireless communication channel 50 established between the display device 32 and a WAP 52 , which is shown as being directly coupled to the network 14 .

[0036] The various ACI client electronic devices (e.g., ACI client electronic devices 28, 30, 32, 34) may each execute an operating system, examples of which may include, but are not limited to, Microsoft Windows tm 、Apple Macintosh tm 、RedhatLinux tm , or a custom operating system, where the combination of various ACI client electronic devices (e.g., ACI client electronic devices 28, 30, 32, 34) and the ACI computing system 12 can form a modular ACI system 54.

[0037] Environmental Collaborative Intelligent System

[0038] Although the ambient collaborative intelligent process 10 will be described below as being used to automate the collection and processing of clinical encounter information to generate / store / distribute medical records, this is for illustrative purposes only and is not intended to limit the present disclosure, as other configurations are possible and are considered to be within the scope of the present disclosure.

[0039] Also refer to Figure 2, shows a simplified exemplary embodiment of a modular ACI system 54 configured for automated collaborative intelligence. The modular ACI system 54 may include: a machine vision system 100 configured to obtain machine vision encounter information 102 related to a patient encounter; an audio recording system 104 configured to obtain audio encounter information 106 related to the patient encounter; and a computing system (e.g., the ACI computing system 12) configured to receive the machine vision encounter information 102 and the audio encounter information 106 from the machine vision system 100 and the audio recording system 104 (respectively). The modular ACI system 54 may also include: a display presentation system 108 configured to present visual information 110; and an audio presentation system 112 configured to present audio information 114, wherein the ACI computing system 12 may be configured to provide the visual information 110 and the audio information 114 to the display presentation system 108 and the audio presentation system 112 (respectively).

[0040] Examples of the machine vision system 100 may include, but are not limited to, one or more ACI client electronic devices (e.g., ACI client electronic device 34, examples of which may include, but are not limited to, an RGB imaging system, an infrared imaging system, an ultraviolet imaging system, a laser imaging system, a SONAR imaging system, a RADAR imaging system, and a thermal imaging system). Examples of the audio recording system 104 may include, but are not limited to, one or more ACI client electronic devices (e.g., ACI client electronic device 30, examples of which may include, but are not limited to, a handheld microphone (e.g., an example of a body-worn microphone), a lavalier microphone (e.g., another example of a body-worn microphone), an embedded microphone, such as a microphone embedded in glasses, a smartphone, a tablet computer, and / or a watch (e.g., another example of a body-worn microphone), and an audio recording device). Examples of the display rendering system 108 may include, but are not limited to, one or more ACI client electronic devices (e.g., ACI client electronic device 32, examples of which may include, but are not limited to, a tablet computer, a computer monitor, and a smart TV). Examples of the audio rendering system 112 may include, but are not limited to, one or more ACI client electronic devices (e.g., audio rendering device 116, examples of which may include, but are not limited to, a speaker system, a headphone system, and an earbud system).

[0041] The ACI computing system 12 can be configured to access one or more data sources 118 (e.g., a plurality of individual data sources 120, 122, 124, 126, 128), examples of which can include, but are not limited to, one or more of the following: a user profile data source, a voice print data source, a voice characteristic data source (e.g., for adapting an ambient speech recognition model), a faceprint data source, a human shape data source, a speech identifier data source, a wearable token identifier data source, an interaction identifier data source, a medical condition symptom data source, a prescription compatibility data source, a medical insurance coverage data source, a physical event data source, and a home healthcare data source. Although five different examples of data sources 118 are shown in this particular example, this is for illustrative purposes only and is not intended to be limiting of the present disclosure, as other configurations are possible and are considered to be within the scope of the present disclosure.

[0042] As will be discussed in greater detail below, the modular ACI system 54 can be configured to monitor a monitored space (e.g., the monitored space 130) in a clinical environment, examples of which can include, but are not limited to, a doctor’s office, a medical facility, a medical practice, a medical laboratory, an urgent care facility, a medical clinic, an emergency room, an operating room, a hospital, a long-term care facility, a rehabilitation facility, a nursing home, and a hospice facility. Accordingly, the above-mentioned patient encounter can include, but is not limited to, a patient visit to one or more of the above-mentioned clinical environments (e.g., a doctor’s office, a medical facility, a medical practice, a medical laboratory, an urgent care facility, a medical clinic, an emergency room, an operating room, a hospital, a long-term care facility, a rehabilitation facility, a nursing home, and a hospice facility).

[0043] When the above-mentioned clinical environment is larger or requires a higher level of resolution, the machine vision system 100 can include a plurality of discrete machine vision systems. As mentioned above, examples of the machine vision system 100 can include, but are not limited to, one or more ACI client electronic devices (e.g., the ACI client electronic device 34, examples of which can include, but are not limited to, an RGB imaging system, an infrared imaging system, an ultraviolet imaging system, a laser imaging system, a SONAR imaging system, a RADAR imaging system, and a thermal imaging system). Accordingly, the machine vision system 100 can include one or more of each of an RGB imaging system, an infrared imaging system, an ultraviolet imaging system, a laser imaging system, a SONAR imaging system, a RADAR imaging system, and a thermal imaging system.

[0044] When the clinical environment is larger or requires a higher level of resolution, the audio recording system 104 may include multiple separate audio recording systems. As described above, examples of the audio recording system 104 may include, but are not limited to, one or more ACI client electronic devices (e.g., ACI client electronic device 30, examples of which may include, but are not limited to, handheld microphones, lavalier microphones, embedded microphones (e.g., microphones embedded in glasses, smartphones, tablet computers, and / or watches), and audio recording devices). Accordingly, the machine vision system 100 may include one or more of each of the handheld microphone, lavalier microphone, embedded microphone (e.g., microphones embedded in glasses, smartphones, tablet computers, and / or watches), and audio recording devices.

[0045] When the clinical environment is larger or requires a higher level of resolution, the display presentation system 108 may include multiple separate display presentation systems. As described above, examples of the display presentation system 108 may include, but are not limited to, one or more ACI client electronic devices (e.g., ACI client electronic device 32, examples of which may include, but are not limited to, tablet computers, computer monitors, and smart TVs). Accordingly, the display presentation system 108 may include one or more of each of the tablet computers, computer monitors, and smart TVs.

[0046] When the clinical environment is larger or requires a higher level of resolution, the audio rendering system 112 may include multiple discrete audio rendering systems. As described above, examples of the audio rendering system 112 may include, but are not limited to, one or more ACI client electronic devices (e.g., audio rendering device 116, examples of which may include, but are not limited to, a speaker system, a headphone system, or an earbud system). Accordingly, the audio rendering system 112 may include one or more of each of the speaker system, the headphone system, or the earbud system.

[0047] The ACI computing system 12 may include multiple discrete computing systems. As described above, the ACI computing system 12 may include various components, examples of which may include, but are not limited to, a personal computer, a server computer, a series of server computers, a minicomputer, a mainframe computer, one or more network attached storage (NAS) systems, one or more storage area network (SAN) systems, one or more platform as a service (PaaS) systems, one or more infrastructure as a service (IaaS) systems, one or more software as a service (SaaS) systems, a cloud-based computing system, and a cloud-based storage platform. Accordingly, the ACI computing system 12 may include one or more of each of a personal computer, a server computer, a series of server computers, a minicomputer, a mainframe computer, one or more network attached storage (NAS) systems, one or more storage area network (SAN) systems, one or more platform as a service (PaaS) systems, one or more infrastructure as a service (IaaS) systems, one or more software as a service (SaaS) systems, a cloud-based computing system, and a cloud-based storage platform.

[0048] microphone array

[0049] Also refer to Figure 3 , the audio recording system 104 may include a microphone array 200 having a plurality of discrete microphone assemblies. For example, the audio recording system 104 may include a plurality of discrete audio capture devices (e.g., audio capture devices 202, 204, 206, 208, 210, 212, 214, 216, 218) that may form the microphone array 200. As will be discussed in greater detail below, the modular ACI system 54 may be configured to form one or more audio recording bundles (e.g., audio recording bundles 220, 222, 224) via the discrete audio capture devices (e.g., audio capture devices 202, 204, 206, 208, 210, 212, 214, 216, 218) included within the audio recording system 104.

[0050] For example, the modular ACI system 54 can also be configured to direct one or more audio recording beams (e.g., audio recording beams 220, 222, 224) toward one or more encounter participants (e.g., encounter participants 226, 228, 230) of the patient encounter. Examples of encounter participants (e.g., encounter participants 226, 228, 230) can include, but are not limited to, medical professionals (e.g., doctors, nurses, physician assistants, laboratory technicians, physical therapists, scribes (e.g., transcriptionists), and / or staff involved in the patient encounter), patients (e.g., people visiting the clinical environment for the patient encounter), and third parties (e.g., friends of the patient, relatives of the patient, and / or acquaintances of the patient involved in the patient encounter).

[0051] Accordingly, the modular ACI system 54 and / or the audio recording system 104 can be configured to form an audio recording beam using one or more of the audio capture devices (e.g., audio capture devices 202, 204, 206, 208, 210, 212, 214, 216, 218). For example, the modular ACI system 54 and / or the audio recording system 104 can be configured to form an audio recording beam 220 using various audio capture devices to capture audio (e.g., speech) generated by a meeting participant 226 (because the audio recording beam 220 is directed toward (i.e., directed toward) the meeting participant 226). Additionally, the modular ACI system 54 and / or the audio recording system 104 can be configured to form an audio recording beam 222 using various audio capture devices to capture audio (e.g., speech) generated by a meeting participant 228 (because the audio recording beam 222 is directed toward (i.e., directed toward) the meeting participant 228). Additionally, the modular ACI system 54 and / or the audio recording system 104 can be configured to utilize various audio capture devices to form an audio recording beam 224, thereby enabling capture of audio (e.g., speech) produced by the meeting participant 230 (because the audio recording beam 224 is directed toward (i.e., directed toward) the meeting participant 230).

[0052] In addition, the modular ACI system 54 and / or the audio recording system 104 can be configured to utilize null-steering precoding to eliminate interference between speakers and / or noise. As is known in the art, null-steering precoding is a spatial signal processing method by which a multi-antenna transmitter can null out multi-user interference signals in wireless communications, wherein null-steering precoding can mitigate the effects of background noise and unknown user interference. In particular, null-steering precoding can be a method of narrowband signal beamforming that can compensate for the delay in receiving signals from a particular source at different elements of an antenna array. Typically, and to improve the performance of an antenna array, incoming signals can be summed and averaged, where certain signals can be weighted and signal delays can be compensated.

[0053] The machine vision system 100 and the audio recording system 104 may be separate devices (e.g. Figure 2). Additionally / alternatively, the machine vision system 100 and the audio recording system 104 can be combined into a single package to form a hybrid-media ACI device 232. For example, the hybrid-media ACI device 232 can be configured to be mounted to structures (e.g., walls, ceilings, beams, columns) within the aforementioned clinical environments (e.g., physician offices, medical institutions, medical practices, medical laboratories, urgent care facilities, medical clinics, emergency rooms, operating rooms, hospitals, long-term care facilities, rehabilitation facilities, nursing homes, and hospice facilities), thereby allowing for easy installation thereof. Furthermore, when the aforementioned clinical environments are larger or require a higher level of resolution, the modular ACI system 54 can be configured to include multiple hybrid-media ACI devices (e.g., the hybrid-media ACI device 232).

[0054] The modular ACI system 54 can also be configured to direct one or more audio recording beams (e.g., audio recording beams 220, 222, 224) toward one or more interview participants (e.g., interview participants 226, 228, 230) of a patient interview based at least in part on the machine vision interview information 102. As described above, the hybrid media ACI device 232 (and the machine vision system 100 / audio recording system 104 included therein) can be configured to monitor one or more interview participants (e.g., interview participants 226, 228, 230) of a patient interview.

[0055] Specifically, and as will be discussed in greater detail below, the machine vision system 100 (as a standalone system or as a component of the hybrid-media ACI device 232) can be configured to detect humanoid shapes within the aforementioned clinical environments (e.g., a doctor's office, medical institution, medical practice, medical laboratory, urgent care facility, medical clinic, emergency room, operating room, hospital, long-term care facility, rehabilitation facility, nursing home, and hospice facility). When the machine vision system 100 detects these humanoid shapes, the modular ACI system 54 and / or the audio recording system 104 can be configured to utilize one or more of the discrete audio capture devices (e.g., audio capture devices 202, 204, 206, 208, 210, 212, 214, 216, 218) to form an audio recording beam (e.g., audio recording beams 220, 222, 224) directed toward each of the detected humanoid shapes (e.g., meeting participants 226, 228, 230).

[0056] As described above, the ACI computing system 12 can be configured to receive machine vision meeting information 102 and audio meeting information 106 from the machine vision system 100 and the audio recording system 104 (respectively); and can be configured to provide visual information 110 and audio information 114 to the display presentation system 108 and the audio presentation system 112 (respectively). Depending on how the modular ACI system 54 (and / or the hybrid-media ACI device 232) is configured, the ACI computing system 12 can be included within the hybrid-media ACI device 232 or external to the hybrid-media ACI device 232.

[0057] Environmental Collaborative Intelligent Process

[0058] As described above, the ACI computing system 12 may execute all or a portion of the ambient collaborative intelligence process 10, wherein the instruction set and subroutines of the ambient collaborative intelligence process 10 (which may be stored, for example, on one or more of the storage devices 16, 20, 22, 24, 26) may be executed by the ACI computing system 12 and / or one or more of the ACI client electronic devices 28, 30, 32, 34.

[0059] As described above, the ambient collaborative intelligent process 10 can be configured to automate the collection and processing of clinical encounter information to generate / store / distribute medical records. Figure 4 And also refer to Figure 4 , the ambient collaborative intelligent process 10 can be configured to obtain 300 encounter information (e.g., machine vision encounter information 102 and / or audio encounter information 106) of a patient encounter (e.g., a visit to a doctor's office). The ambient collaborative intelligent process 10 can also be configured to process 302 the encounter information (e.g., machine vision encounter information 102 and / or audio encounter information 106) to generate an encounter transcript (e.g., encounter transcript 234), wherein the ambient collaborative intelligent process 10 can then process 304 at least a portion of the encounter record (e.g., encounter transcript 234) to populate at least a portion of a medical record (e.g., medical record 236) associated with the patient encounter (e.g., a visit to a doctor's office). The encounter transcript 234 and / or medical record 236 can be reviewed by medical professionals involved in the patient encounter (e.g., a visit to a doctor's office) to determine its accuracy and / or make corrections thereto.

[0060] For example, a scribe participating in (or assigned to) a patient encounter (e.g., a visit to a doctor’s office) can review the encounter transcript 234 and / or the medical record 236 to confirm its accuracy and / or to correct it. In the event that the encounter transcript 234 and / or the medical record 236 are corrected, the ambient collaborative intelligence process 10 can use these corrections for training / tuning purposes (e.g., adjust various profiles associated with participants of the patient encounter) to enhance future accuracy / efficiency / performance of the ambient collaborative intelligence process 10.

[0061] Alternatively / additionally, a doctor participating in a patient encounter (e.g., a visit to a doctor’s office) can review the encounter transcript 234 and / or the medical record 236 to confirm its accuracy and / or to correct it. In the event that the encounter transcript 234 and / or the medical record 236 are corrected, the ambient collaborative intelligence process 10 can use these corrections for training / tuning purposes (e.g., adjust various profiles associated with participants of the patient encounter) to enhance future accuracy / efficiency / performance of the ambient collaborative intelligence process 10.

[0062] For example, assume a patient (e.g., an encounter participant 228) visits a clinical environment (e.g., a doctor’s office) because they feel unwell. They have a headache, a fever, chills, a cough, and some difficulty breathing. In this particular example, a monitored space (e.g., the monitored space 130) within the clinical environment (e.g., the doctor’s office) can be equipped with a machine vision system 100 configured to obtain machine vision encounter information 102 about the patient encounter (e.g., the encounter participant 228 visiting the doctor’s office) and an audio recording system 104 configured to obtain audio encounter information 106 about the patient encounter (e.g., the encounter participant 228 visiting the doctor’s office) via one or more audio sensors (e.g., the audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218).

[0063] As noted above, if the monitored space (e.g., monitored space 130) within the clinical environment (e.g., physician's office) is larger or requires a higher level of resolution, the machine vision system 100 can include multiple discrete machine vision systems, where examples of the machine vision system 100 can include, but are not limited to, an RGB imaging system, an infrared imaging system, an ultraviolet imaging system, a laser imaging system, a SONAR imaging system, a RADAR imaging system, and a thermal imaging system. Accordingly, in certain instances / embodiments, the machine vision system 100 can include one or more of each of an RGB imaging system, an infrared imaging system, an ultraviolet imaging system, a laser imaging system, a SONAR imaging system, a RADAR imaging system, and a thermal imaging system positioned throughout the monitored space 130, where each of these systems can be configured to provide data (e.g., machine vision meeting information 102) to the ACI computing system 12 and / or the modular ACI system 54.

[0064] Also as noted above, if the monitored space (e.g., monitored space 130) within the clinical environment (e.g., physician's office) is larger or requires a higher level of resolution, the audio recording system 104 can include multiple discrete audio recording systems, where examples of the audio recording system 104 can include, but are not limited to, a handheld microphone, a lavalier microphone, an embedded microphone (e.g., those embedded in eyeglasses, smartphones, tablet computers, and / or watches), and an audio recording device. Accordingly, in certain instances / embodiments, the audio recording system 104 can include one or more of each of a handheld microphone, a lavalier microphone, an embedded microphone (e.g., those embedded in eyeglasses, smartphones, tablet computers, and / or watches), and an audio recording device positioned throughout the monitored space, where each of these microphones / devices can be configured to provide data (e.g., audio meeting information 106) to the ACI computing system 12 and / or the modular ACI system 54.

[0065] Because the machine vision system 100 and the audio recording system 104 can be positioned throughout the monitored space 130, all interactions between medical professionals (e.g., meeting participant 226), patients (e.g., meeting participant 228), and third parties (e.g., meeting participant 230) that occur during a patient encounter (e.g., meeting participant 228 visiting a physician’s office) within a monitored space (e.g., monitored space 130) of a clinical environment (e.g., physician’s office) can be monitored / recorded / processed. Accordingly, a patient “check-in” area within the monitored space 130 can be monitored for meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) during this pre-visit portion of a patient encounter (e.g., meeting participant 228 visiting a physician’s office). Further, various rooms within the monitored space 130 can be monitored for meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) during these different portions of a patient encounter (e.g., when meeting with a physician, when obtaining vital signs and statistics, and when conducting imaging). Further still, a patient “check-out” area within the monitored space 130 can be monitored for meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) during this post-visit portion of a patient encounter (e.g., meeting participant 228 visiting a physician’s office). Additionally and via the machine vision meeting information 102, visual speech recognition (via visual lip-reading functionality) can be leveraged by the environment-cooperative intelligence process 10 to further carry out the collection of audio meeting information 106.

[0066] Accordingly and when obtaining 300 meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106), the environmental collaborative intelligence process 10 can: obtain 306 meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) from a medical professional (e.g., meeting participant 226); obtain 308 meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) from a patient (e.g., meeting participant 228); and / or obtain 310 meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) from a third party (e.g., meeting participant 230). Further and when obtaining 300 meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106), the environmental collaborative intelligence process 10 can obtain 300 meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) from prior (related or unrelated) patient meetings. For example, if the current patient meeting is in fact a third visit by the patient regarding, for example, shortness of breath, meeting information from the previous two visits (i.e., the previous two patient meetings) can be highly relevant and can be obtained 300 by the environmental collaborative intelligence process 10.

[0067] When the environmental collaborative intelligence process 10 obtains 300 meeting information, the environmental collaborative intelligence process 10 can prompt 312, with a virtual assistant (e.g., virtual assistant 238), the patient (e.g., meeting participant 228) to provide at least a portion of the meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) during a pre-visit portion (e.g., patient intake portion) of the patient meeting (e.g., meeting participant 228 visits a doctor’s office).

[0068] Further and when the environmental collaborative intelligence process 10 obtains 300 meeting information, the environmental collaborative intelligence process 10 can prompt 314, with a virtual assistant (e.g., virtual assistant 238), the patient (e.g., meeting participant 228) to provide at least a portion of the meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) during a post-visit portion (e.g., patient follow-up portion) of the patient meeting (e.g., meeting participant 228 visits a doctor’s office).

[0069] Automated transcript generation

[0070] The environmental collaborative intelligence process 10 can be configured to process the meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) to generate a meeting transcript 234 that can be automatically formatted and punctuated.

[0071] According to Figure 5 And also with reference to Figure 5Accordingly and continuing with the above example, the environmental collaborative intelligence process 10 can be configured to obtain 300 meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) of a patient meeting (e.g., a visit to a physician’s office).

[0072] The environmental collaborative intelligence process 10 can process 350 the meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) to: associate a first portion of the meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) with a first meeting participant, and associate at least a second portion of the meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) with at least a second meeting participant.

[0073] As noted above, the modular ACI system 54 can be configured to form one or more audio recording beams (e.g., audio recording beams 220, 222, 224) via discrete audio acquisition devices (e.g., discrete audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218) included within the audio recording system 104, where the modular ACI system 54 can also be configured to direct the one or more audio recording beams (e.g., audio recording beams 220, 222, 224) toward one or more meeting participants (e.g., meeting participants 226, 228, 230) of the above-described patient meeting.

[0074] Accordingly and continuing with the above example, the modular ACI system 54 can direct the audio recording beam 220 toward the meeting participant 226, can direct the audio recording beam 222 toward the meeting participant 228, and can direct the audio recording beam 224 toward the meeting participant 230. Accordingly and due to the directionality of the audio recording beams 220, 222, 224, the audio meeting information 106 can include three components, namely audio meeting information 106A (obtained via the audio recording beam 220), audio meeting information 106B (obtained via the audio recording beam 222), and audio meeting information 106C (obtained via the audio recording beam 220).

[0075] Further and as noted above, the ACI computing system 12 can be configured to access one or more data sources 118 (e.g., a plurality of individual data sources 120, 122, 124, 126, 128), examples of which can include, but are not limited to, one or more of the following: a user profile data source, a voiceprint data source, a voice characteristic data source (e.g., for tuning automated speech recognition models), a faceprint data source, a body shape data source, a speech identifier data source, a wearable token identifier data source, an interaction identifier data source, a medical condition symptom data source, a prescription compatibility data source, a medical insurance coverage data source, a physical event data source, and a home healthcare data source.

[0076] Accordingly, the environmental collaborative intelligence process 10 can process 350 the meeting information (e.g., the machine vision meeting information 102 and / or the audio meeting information 106) to: associate a first portion (e.g., the meeting information 106A) of the meeting information (e.g., the audio meeting information 106) with a first meeting participant (e.g., the meeting participant 226), and associate at least a second portion (e.g., the meeting information 106B, 106C) of the meeting information (e.g., the audio meeting information 106) with at least a second meeting participant (e.g., the meeting participants 228, 230, respectively).

[0077] Further and when processing 350 the meeting information (e.g., the audio meeting information 106A, 106B, 106C), the environmental collaborative intelligence process 10 can compare each of the audio meeting information 106A, 106B, 106C to the voiceprints defined in the voiceprint data source described above, whereby the identity of the meeting participants 226, 228, 230 can be determined (respectively). Accordingly, if the voiceprint data source includes a voiceprint corresponding to one or more of: the voice of the meeting participant 226 (as heard in the audio meeting information 106A), the voice of the meeting participant 228 (as heard in the audio meeting information 106B), or the voice of the meeting participant 230 (as heard in the audio meeting information 106C), the identity of one or more of the meeting participants 226, 228, 230 can be defined. And in the event that the voice heard in one or more of the audio meeting information 106A, the audio meeting information 106B, or the audio meeting information 106C is unidentifiable, the one or more particular meeting participants can be defined as “unknown participants”.

[0078] Once the voices of the meeting participants 226, 228, 230 are processed 350, the environmental collaborative intelligence process 10 can generate 302 a meeting transcript (e.g., the meeting transcript 234) based at least in part on the first portion of the meeting information (e.g., the audio meeting information 106A) and the at least second portion of the meeting information (e.g., the audio meeting information 106B, 106C).

[0079] Automated role assignment

[0080] The environmental collaborative intelligence process 10 can be configured to automatically define the roles of the meeting participants (e.g., the meeting participants 226, 228, 230) in a patient meeting (e.g., a visit to a doctor’s office).

[0081] Accordingly and also with reference to Figure 6, the environmental collaborative intelligence process 10 can be configured to obtain 300 meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) of a patient meeting (e.g., visiting a physician office).

[0082] Then, the environmental collaborative intelligence process 10 can process 400 the meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) to associate a first portion of the meeting information with a first meeting participant (e.g., meeting participant 226), and assign 402 a first role to the first meeting participant (e.g., meeting participant 226).

[0083] While processing 400 the meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) to associate a first portion of the meeting information with a first meeting participant (e.g., meeting participant 226), the environmental collaborative intelligence process 10 can process 404 the meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) to associate a first portion of the audio meeting information (e.g., audio meeting information 106A) with the first meeting participant (e.g., meeting participant 226).

[0084] In particular and while processing 404 the meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) to associate a first portion of the audio meeting information (e.g., audio meeting information 106A) with the first meeting participant (e.g., meeting participant 226), the environmental collaborative intelligence process 10 can compare 406 one or more voiceprints (defined within a voiceprint data source) to one or more voices defined within the first portion of the audio meeting information (e.g., audio meeting information 106A); and can compare 408 one or more utterance identifiers (defined within an utterance data source) to one or more utterances defined within the first portion of the audio meeting information (e.g., audio meeting information 106A); wherein the comparisons 406, 408 can allow the environmental collaborative intelligence process 10 to assign 402 the first role to the first meeting participant (e.g., meeting participant 226). For example, if the identity of the meeting participant 226 can be defined via a voiceprint, then in the case where that defined identity is associated with a role (e.g., the identity defined for the meeting participant 226 is Dr. Susan Jones), the role of the meeting participant 226 can be assigned 402. Further, if the utterance of the meeting participant 226 is "I am Dr. Susan Jones," then that utterance can allow the role of the meeting participant 226 to be assigned 402.

[0085] When processing 400 meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) to associate a first portion of the meeting information with a first meeting participant (e.g., meeting participant 226), the ambient collaborative intelligent process 10 may process 410 meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106) to associate a first portion of the machine vision meeting information (e.g., machine vision meeting information 102A) with a first meeting participant (e.g., meeting participant 226).

[0086] In particular, and when processing 404 the meeting information (e.g., the machine vision meeting information 102 and / or the audio meeting information 106) to associate the first portion of the machine vision meeting information (e.g., the machine vision meeting information 102A) with the first meeting participant (e.g., the meeting participant 226), the ambient collaborative intelligent process 10 may compare 412 one or more faceprints (defined within the faceprint data source) with one or more faces defined within the first portion of the machine vision meeting information (e.g., the machine vision meeting information 102A); compare 414 one or more wearable token identifiers (defined within the faceprint data source); The process 10 may further comprise comparing 412, 414, and 416 one or more interaction identifiers (defined in the interaction identifier data source) with one or more wearable tokens defined in the first portion of the machine vision meeting information (e.g., machine vision meeting information 102A); and comparing 416 one or more interaction identifiers (defined in the interaction identifier data source) with one or more humanoid interactions defined in the first portion of the machine vision meeting information (e.g., machine vision meeting information 102A); wherein the comparisons 412, 414, and 416 may allow the ambient collaborative intelligence process 10 to assign 402 a first role to a first meeting participant (e.g., meeting participant 226). For example, if the identity of meeting participant 226 can be defined via a faceprint, then if the defined identity is associated with a role (e.g., the identity defined for meeting participant 226 is Dr. Susan Jones), then the role of meeting participant 226 may be assigned 402. Furthermore, if the wearable token worn by meeting participant 226 can be identified as a wearable token assigned to Dr. Susan Jones, then the role of meeting participant 226 may be assigned 402. Additionally, if an interaction performed by the meeting participant 226 corresponds to a type of interaction performed by a physician, the presence of the interaction may allow for the assignment 402 of a role for the meeting participant 226 .

[0087] Examples of such wearable tokens can include, but are not limited to, wearable devices that can be worn by medical professionals while they are within the monitored space 130 (or after they leave the monitored space 130). For example, these wearable tokens can be worn by medical professionals, e.g., when they are moving between monitored rooms within the monitored space 130, to and from the monitored space 130, and / or outside of the monitored space 130 (e.g., at home).

[0088] Additionally, the environment-coordinated intelligent process 10 can process 418 the meeting information (e.g., the machine-vision meeting information 102 and / or the audio meeting information 106) to associate at least a second portion of the meeting information with at least a second meeting participant; and can assign 420 at least a second role to the at least a second meeting participant.

[0089] In particular, the environment-coordinated intelligent process 10 can process 418 the meeting information (e.g., the machine-vision meeting information 102 and / or the audio meeting information 106) to associate at least a second portion of the meeting information with at least a second meeting participant. For example, the environment-coordinated intelligent process 10 can process 418 the meeting information (e.g., the machine-vision meeting information 102 and / or the audio meeting information 106) to associate the audio meeting information 106B and the machine-vision meeting information 102B with the meeting participant 228, and can associate the audio meeting information 106C and the machine-vision meeting information 102C with the meeting participant 230.

[0090] Further, the environment-coordinated intelligent process 10 can assign 420 at least a second role to the at least a second meeting participant. For example, the environment-coordinated intelligent process 10 can assign 420 a role to the meeting participant 228, 230.

[0091] Automated movement tracking

[0092] The environment-coordinated intelligent process 10 can be configured to track movement and / or interactions of human shapes within a monitored space (e.g., the monitored space 130) during a patient meeting (e.g., a visit to a doctor’s office) such that, for example, the environment-coordinated intelligent process 10 knows when a meeting participant (e.g., one or more of the meeting participants 226, 228, 230) enters, leaves, or crosses a path within the monitored space 130.

[0093] According to Figure 7 and also with reference to Figure 7environmentally collaborative intelligent process 10 can process 450 machine vision meeting information (e.g., machine vision meeting information 102) to identify one or more human shapes. As noted above, examples of machine vision systems 100 (and, in particular, examples of ACI client electronic devices 34) can include, but are not limited to, one or more of an RGB imaging system, an infrared imaging system, an ultraviolet imaging system, a laser imaging system, a SONAR imaging system, a RADAR imaging system, a thermal imaging system.

[0094] When an ACI client electronic device 34 includes a visible light imaging system (e.g., an RGB imaging system), the ACI client electronic device 34 can be configured to monitor various objects within a monitored space 130 by recording motion video in the visible light spectrum of those various objects. When an ACI client electronic device 34 includes a non-visible light imaging system (e.g., a laser imaging system, an infrared imaging system, and / or an ultraviolet imaging system), the ACI client electronic device 34 can be configured to monitor various objects within a monitored space 130 by recording motion video in the non-visible light spectrum of those various objects. When an ACI client electronic device 34 includes an X-ray imaging system, the ACI client electronic device 34 can be configured to monitor various objects within a monitored space 130 by recording energy in the X-ray spectrum of those various objects. When an ACI client electronic device 34 includes a SONAR imaging system, the ACI client electronic device 34 can be configured to monitor various objects within a monitored space 130 by emitting sound waves that can reflect off of those various objects. When an ACI client electronic device 34 includes a RADAR imaging system, the ACI client electronic device 34 can be configured to monitor various objects within a monitored space 130 by emitting radio waves that can reflect off of those various objects. When an ACI client electronic device 34 includes a thermal imaging system, the ACI client electronic device 34 can be configured to monitor various objects within a monitored space 130 by tracking thermal energy of those different objects.

[0095] As noted above, the ACI computing system 12 can be configured to access one or more data sources 118 (e.g., a plurality of individual data sources 120, 122, 124, 126, 128), examples of which can include, but are not limited to, one or more of: a user profile data source, a voiceprint data source, a voice characteristic data source (e.g., for tuning automated speech recognition models), a facial print data source, a human shape data source, a speech identifier data source, a wearable token identifier data source, an interaction identifier data source, a medical condition symptom data source, a prescription compatibility data source, a medical insurance coverage data source, a physical event data source, and a home healthcare data source.

[0096] Accordingly, and when processing 450 machine vision meeting information (e.g., machine vision meeting information 102) to identify one or more humanoid shapes, the ambient collaborative intelligence process 10 can be configured to compare humanoid shapes defined within one or more data sources 118 with potential humanoid shapes within the machine vision meeting information (e.g., machine vision meeting information 102).

[0097] When processing 450 machine vision meeting information (e.g., machine vision meeting information 102) to identify one or more humanoid shapes, the ambient collaborative intelligence process 10 may track 452 the movement of the one or more humanoid shapes within a monitored space (e.g., monitored space 130). For example, and while tracking 452 the movement of the one or more humanoid shapes within the monitored space 130, the ambient collaborative intelligence process 10 may add 454 new humanoid shapes to the one or more humanoid shapes when new humanoid shapes enter the monitored space (e.g., monitored space 130) and / or may remove 456 existing humanoid shapes from the one or more humanoid shapes when existing humanoid shapes leave the monitored space (e.g., monitored space 130).

[0098] For example, assume that a lab technician (e.g., meeting participant 242) temporarily enters the monitored space 130 to chat with meeting participant 230. Accordingly, the ambient collaboration intelligent process 10 can add 454 the meeting participant 242 to the one or more humanoid shapes being tracked 452 upon the new humanoid shape (i.e., meeting participant 242) entering the monitored space 130. Furthermore, assume that the lab technician (e.g., meeting participant 242) leaves the monitored space 130 after chatting with meeting participant 230. Accordingly, the ambient collaboration intelligent process 10 can remove 456 the meeting participant 242 from the one or more humanoid shapes being tracked 452 upon the humanoid shape (i.e., meeting participant 242) leaving the monitored space 130.

[0099] Furthermore, when tracking 452 the movement of one or more humanoid shapes within the monitored space 130, the ambient collaborative intelligence process 10 may monitor the trajectories of the various humanoid shapes within the monitored space 130. Accordingly, assume that the meeting participant 242 walks in front of (or behind) the meeting participant 226 when leaving the monitored space 130. As the ambient collaborative intelligence process 10 is monitoring (in this example) the trajectories of the meeting participant 242 (e.g., who is moving from left to right) and the meeting participant 226 (e.g., who is stationary), the identities of the two humanoid shapes may not be confused by the ambient collaborative intelligence process 10 when the meeting participant 242 passes in front of (or behind) the meeting participant 226.

[0100] The ambient collaborative intelligence process 10 may be configured to obtain 300 encounter information for a patient encounter (eg, a visit to a doctor's office), which may include machine vision encounter information 102 (in the manner described above) and / or audio encounter information 106 .

[0101] The ambient collaborative intelligence process 10 can direct 458 one or more audio recording beams (e.g., audio recording beams 220, 222, 224) toward one or more humanoid shapes (e.g., meeting participants 226, 228, 230) to capture audio meeting information (e.g., audio meeting information 106), where the audio meeting information 106 can be included within the meeting information (e.g., machine vision meeting information 102 and / or audio meeting information 106).

[0102] In particular, and as described above, the ambient collaborative intelligence process 10 (via the modular ACI system 54 and / or the audio recording system 104) can utilize one or more discrete audio capture devices (e.g., audio capture devices 202, 204, 206, 208, 210, 212, 214, 216, 218) to form an audio recording beam. For example, the modular ACI system 54 and / or the audio recording system 104 can be configured to utilize various audio capture devices to form an audio recording beam 220, thereby enabling the capture of audio (e.g., speech) generated by a meeting participant 226 (because the audio recording beam 220 is directed toward (i.e., directed toward) the meeting participant 226). Additionally, the modular ACI system 54 and / or the audio recording system 104 can be configured to utilize various audio capture devices to form an audio recording beam 222, thereby enabling the capture of audio (e.g., speech) generated by a meeting participant 228 (because the audio recording beam 222 is directed toward (i.e., directed toward) the meeting participant 228). Additionally, the modular ACI system 54 and / or the audio recording system 104 can be configured to form an audio recording beam 224 utilizing various audio capture devices, thereby enabling the capture of audio (e.g., speech) produced by the meeting participant 230 (because the audio recording beam 224 is directed toward (i.e., directed toward) the meeting participant 230).

[0103] Once obtained, the ambient collaborative intelligent process 10 can process 302 the encounter information (e.g., machine vision encounter information 102 and / or audio encounter information 106) to generate an encounter transcript 234, and can process 304 at least a portion of the encounter transcript 234 to populate at least a portion of a medical record (e.g., medical record 236) associated with the patient encounter (e.g., a visit to a doctor's office).

[0104] Fully automatic / semi-automatic scanning:

[0105] As mentioned above and as Figure 2As shown, the modular ACI system 54 can be configured as automated collaborative intelligence, wherein the modular ACI system 54 can include: a machine vision system 100, configured to obtain machine vision encounter information 102 about a patient encounter; an audio recording system 104, configured to obtain audio encounter information 106 about a patient encounter; and a computing system (e.g., the ACI computing system 12), configured to receive the machine vision encounter information 102 and the audio encounter information 106 from the machine vision system 100 and the audio recording system 104 (respectively).

[0106] As also described above, the machine vision system 100 may include, but is not limited to, one or more ACI client electronic devices (e.g., ACI client electronic device 34, examples of which may include, but are not limited to, RGB imaging systems, infrared imaging systems, ultraviolet imaging systems, laser imaging systems, SONAR imaging systems, RADAR imaging systems, thermal imaging systems).

[0107] As also described above, the audio recording system 104 may include, but is not limited to, one or more ACI client electronic devices (e.g., the ACI client electronic device 30, examples of which may include, but are not limited to, a handheld microphone (e.g., one example of a body-worn microphone), a lavalier microphone (e.g., another example of a body-worn microphone), an embedded microphone, such as a microphone embedded in glasses, a smartphone, a tablet computer, and / or a watch (e.g., another example of a body-worn microphone), and an audio recording device).

[0108] In addition and as Figure 3 As shown, the machine vision system 100 and the audio recording system 104 can be combined into one package to form a hybrid-media ACI device 232. For example, the hybrid-media ACI device 232 can be configured to be mounted to structures (e.g., walls, ceilings, beams, columns) within the aforementioned clinical environments (e.g., doctors' offices, medical offices, medical practices, medical laboratories, urgent care facilities, medical clinics, emergency rooms, operating rooms, hospitals, long-term care facilities, rehabilitation facilities, nursing homes, and hospice facilities), thereby allowing for easy installation thereof.

[0109] The modular ACI system 54 can also be configured to direct one or more audio recording beams (e.g., audio recording beams 220, 222, 224) toward one or more interview participants (e.g., interview participants 226, 228, 230) of a patient interview based at least in part on the machine vision interview information 102. As described above, the hybrid media ACI device 232 (and the machine vision system 100 / audio recording system 104 included therein) can be configured to monitor one or more interview participants (e.g., interview participants 226, 228, 230) of a patient interview.

[0110] In particular, and as described above, the machine vision system 100 (as a standalone system or as a component of the hybrid-media ACI device 232) can be configured to detect humanoid shapes within the aforementioned clinical environments (e.g., a doctor's office, a medical office, a medical practice, a medical laboratory, an urgent care facility, a medical clinic, an emergency room, an operating room, a hospital, a long-term care facility, a rehabilitation facility, a nursing home, and a hospice facility). When the machine vision system 100 detects these humanoid shapes, the modular ACI system 54 and / or the audio recording system 104 can be configured to utilize one or more of the discrete audio capture devices (e.g., audio capture devices 202, 204, 206, 208, 210, 212, 214, 216, 218) to form an audio recording beam (e.g., audio recording beams 220, 222, 224) directed toward each detected humanoid shape (e.g., meeting participant 226, 228, 230).

[0111] Accordingly, it is contemplated that one or more of the systems / devices included within the modular ACI system 54 (e.g., the machine vision system 100, the audio recording system 104, the mixed media ACI device 232, and / or the audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218) may require calibration (e.g., initially and / or subsequently recalibrated).

[0112] Also refer to Figure 8-Figure 9 Such calibration of one or more of these systems / devices (e.g., machine vision system 100, audio recording system 104, mixed media ACI device 232, and / or audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218) included within modular ACI system 54 may be performed via the ACI calibration platform 500.

[0113] The ACI calibration platform 500 may include a video recording subsystem 502 configured to generate 550 a three-dimensional model (e.g., three-dimensional model 504) of at least a portion of a three-dimensional space (e.g., monitored space 130) that contains an ACI system (e.g., modular ACI system 54). The ACI calibration platform 500 (in general) and the video recording subsystem 502 (in particular) may include (or interface with) machine vision technology, examples of which may include, but are not limited to, RGB imaging systems, infrared imaging systems, ultraviolet imaging systems, laser imaging systems, SONAR imaging systems, RADAR imaging systems, and thermal imaging systems.

[0114] As described above, examples of monitored spaces 130 may include, but are not limited to, clinical settings (e.g., physicians' offices, medical offices, medical practices, medical laboratories, urgent care facilities, medical clinics, emergency rooms, operating rooms, hospitals, long-term care facilities, rehabilitation facilities, nursing homes, and hospice facilities).

[0115] The three-dimensional model (e.g., three-dimensional model 504) generated by the video recording subsystem 502 of the ACI calibration platform 500 can be configured to define one or more of the following:

[0116] Subspaces: One or more subspaces within a three-dimensional space (e.g., monitored space 130) may be defined within three-dimensional model 504, where examples of such subspaces may include, but are not limited to, visitor waiting space 506 (shown as including meeting participants 230, 242).

[0117] Objects: One or more objects within the three-dimensional space (eg, monitored space 130 ) may be defined within the three-dimensional model 504 , where examples of these objects may include, but are not limited to, a physician table 508 and an examination table 510 .

[0118] Features: One or more features within the three-dimensional space (eg, monitored space 130 ) may be defined within the three-dimensional model 504 , where examples of such features may include, but are not limited to, a window 512 .

[0119] Interaction zones: One or more interaction zones within the three-dimensional space (e.g., monitored space 130) may be defined within the three-dimensional model 504, where examples of such interaction zones may include, but are not limited to, an inspection zone 514 (i.e., an area proximate to the inspection table 510).

[0120] Noise Sources: One or more noise sources within the three-dimensional space (eg, monitored space 130 ) may be defined within the three-dimensional model 504 , where examples of such noise sources may include, but are not limited to, HVAC supply vents 516 .

[0121] The ACI calibration platform 500 may be wirelessly coupled to one or more external systems (e.g., modular ACI system 54) and / or one or more external resources (e.g., one or more of data sources 120, 122, 124, 126, 128), thereby enabling data to be transferred between the ACI calibration platform 500 and these external resources and / or data sources.

[0122] Accordingly and by way of this wireless connectivity, the three-dimensional model 504 can be wirelessly transmitted from the ACI calibration platform 500 to the modular ACI system 54 for processing, which will be discussed in greater detail below. Alternatively, the three-dimensional model 504 can be transmitted from the ACI calibration platform 500 to the modular ACI system 54 for processing via a wired transmission method (e.g., a USB drive; not shown), which will be discussed in greater detail below.

[0123] The video recording system 502 can be configured to interface with an object data source (e.g., the object data source 518), which can define a plurality of objects that can be located within a three-dimensional space (e.g., the monitored space 130). For example, the object data source 518 can define what a table “looks like,” what an exam table “looks like,” what an HVAC vent “looks like,” and what a window “looks like.” This functionality can be implemented in a similar manner as the facial recognition system uses to know what a face “looks like.” Depending on how the ACI calibration platform 500 is configured, the object data source 518 can be a locally accessible data source that resides on the ACI calibration platform 500. Alternatively, the object data source 518 can be a remotely accessible data source that resides on the modular ACI system 54.

[0124] Accordingly and by way of the use of the object data source 518, the ACI calibration platform 500 can produce a three-dimensional model (e.g., the three-dimensional model 504) in which the objects included / defined can be of a known type (e.g., the physician desk 508, the exam table 510, the window 512, the HVAC supply vent 516), which can be done via tagging / metadata.

[0125] The ACI calibration platform 500 can include an audio generation subsystem 520 configured to generate 552 one or more audio calibration signals (e.g., the audio calibration signal 522) for receipt by an audio recording system (e.g., the audio recording system 104) included within an ACI system (e.g., the modular ACI system 54). The ACI calibration platform 500 (generally) and the audio generation subsystem 520 (specifically) can include (or interface with) audio presentation technology, examples of which can include, but are not limited to, a speaker assembly.

[0126] The one or more audio calibration signals (e.g., the audio calibration signal 522) can include one or more of the following:

[0127] Noise signal: Examples of audio calibration signal 522 may include, but are not limited to, a white noise signal. As is known in the art, a white noise signal is a random signal with equal intensity at all frequencies, giving it a constant power spectral density. The term is used with this or similar meanings in many scientific and technical disciplines, including physics, acoustic engineering, telecommunications, and statistical prediction. White noise refers to a statistical model of signals and signal sources, rather than any specific signal.

[0128] Sinusoidal signal: Examples of audio calibration signal 522 may include, but are not limited to, a sinusoidal signal. A sinusoidal signal is a signal completely characterized by a mathematical function describing a smooth, periodic oscillation with a fixed frequency. It is named after the function sine. Sinusoidal curves frequently appear in pure and applied mathematics, as well as in physics, engineering, and signal processing.

[0129] Multi-frequency signal: Examples of audio calibration signal 522 may include, but are not limited to, a swept sine signal. A swept sine signal is a signal fully characterized by a mathematical function that describes a smooth periodic oscillation with a frequency that varies over time (typically between two frequencies, such as a logarithmic sweep from 20 Hz to 20 kHz in acoustic applications).

[0130] Impulse function: Examples of the audio calibration signal 522 may include, but are not limited to, an impulse function. An impulse function is a function that is zero everywhere except at the origin, where the amplitude is infinitely high.

[0131] The ACI calibration platform 500 may include a mobile base assembly 524 configured to autonomously position 554 the ACI calibration platform 500 within a three-dimensional space (e.g., the monitored space 130). Accordingly, the ACI calibration platform 500 may be configured to move within the monitored space 130 in an automated and controlled manner (e.g., in a manner similar to a robotic autonomous vacuum cleaner). For example, the ACI calibration platform 500 may include the aforementioned machine vision technology to enable the ACI calibration platform 500 to navigate through the monitored space 130 via the use of the mobile base assembly 524. Additionally, the ACI calibration platform 500 (generally) and the mobile base assembly 524 (specifically) may include one of a plurality of impact sensors (e.g., impact sensors 526, 528) that sense impacts with various objects (e.g., walls, doors, furniture) within the monitored space 130, such that upon sensing such impacts, the direction of travel of the ACI calibration platform 500 (e.g., reverse direction) may be adjusted.

[0132] The ACI calibration platform 500 may include a cleaning component 530 configured to autonomously clean 556 at least a portion of a three-dimensional space (e.g., the monitored space 130). Examples of the cleaning component 530 may include:

[0133] Vacuum cleaner assembly: For example, the cleaning assembly 530 can be configured to vacuum the floor of the monitored space 130 .

[0134] Mopping assembly: For example, the cleaning assembly 530 can be configured to mop the floor of the monitored space 130 .

[0135] • Disinfection assembly: For example, the cleaning assembly 530 can be configured to disinfect the floor of the monitored space 130 via, for example, steam generation or ultraviolet light.

[0136] While the ACI calibration platform 500 is described as being capable of autonomous movement within the monitored space 130 (via the mobile base assembly 524), this is for illustrative purposes only and is not intended to limit the present disclosure, as other configurations are possible and are considered within the scope of the present disclosure. For example, the ACI calibration platform 500 can be configured to be manually positioned within a three-dimensional space (e.g., the monitored space 130). Thus, the ACI calibration platform 500 can be included within (or a portion of) a handheld client electronic device (such as a smartphone or tablet computer).

[0137] As is known in the art, such client electronic devices typically include machine vision technology (e.g., a visible light camera) and audio rendering technology (such as one or more speaker assemblies) that are capable of generating 550, 552 the three-dimensional model 504 and / or the audio calibration signal 522. In such an implementation, the ACI calibration platform 500 may be manually manipulated (i.e., moved / positioned) by a user within the monitored space 130, wherein the user may move the ACI calibration platform 500 within the monitored space 130 to generate 550 the three-dimensional model 504 and / or generate 552 the audio calibration signal 522 from all appropriate / desired locations within the monitored space 130.

[0138] ACI System Calibration

[0139] As described above, the ACI calibration platform 500 can be wirelessly coupled to one or more external systems (e.g., the modular ACI system 54) and / or one or more external resources (e.g., one or more of the data sources 120, 122, 124, 126, 128), thereby enabling data to be transferred between the ACI calibration platform 500 and these external resources and / or data sources. Accordingly, and through this wireless connectivity, the three-dimensional model 504 can be wirelessly transferred from the ACI calibration platform 500 to the modular ACI system 54 for processing. Alternatively, the three-dimensional model 504 can be transferred from the ACI calibration platform 500 to the modular ACI system 54 for processing via a non-wireless transfer method, such as via a portable data transfer device (e.g., a USB drive; not shown).

[0140] Accordingly, the ambient collaborative intelligence process 10 can obtain 600 calibration information (e.g., calibration information 532) for a three-dimensional space (e.g., monitored space 130) containing an ACI system (e.g., modular ACI system 54). As described above, the calibration information can be obtained from an ACI calibration platform (e.g., ACI calibration platform 500). The calibration information (e.g., calibration information 532) can include the three-dimensional model 504 and one or more audio calibration signals (e.g., audio calibration signal 522).

[0141] As described above, the ACI calibration platform 500 can generate a three-dimensional model 504 for at least a portion of a monitored space 130 containing an ACI system (e.g., the modular ACI system 54), wherein the three-dimensional model 504 can be configured to define one or more of the following:

[0142] Subspaces: One or more subspaces within a three-dimensional space (e.g., monitored space 130) may be defined within three-dimensional model 504, where examples of such subspaces may include, but are not limited to, visitor waiting space 506 (shown as including meeting participants 230, 242).

[0143] Objects: One or more objects within the three-dimensional space (eg, monitored space 130 ) may be defined within the three-dimensional model 504 , where examples of these objects may include, but are not limited to, a physician table 508 and an examination table 510 .

[0144] Features: One or more features within the three-dimensional space (eg, monitored space 130 ) may be defined within the three-dimensional model 504 , where examples of such features may include, but are not limited to, a window 512 .

[0145] Interaction zones: One or more interaction zones within the three-dimensional space (e.g., monitored space 130) may be defined within the three-dimensional model 504, where examples of such interaction zones may include, but are not limited to, an inspection zone 514 (i.e., an area proximate to the inspection table 510).

[0146] Noise Sources: One or more noise sources within the three-dimensional space (eg, monitored space 130 ) may be defined within the three-dimensional model 504 , where examples of such noise sources may include, but are not limited to, HVAC supply vents 516 .

[0147] As described above, these one or more audio calibration signals (e.g., audio calibration signal 522) may include one or more of the following:

[0148] Noise signal: Examples of audio calibration signal 522 may include, but are not limited to, a white noise signal. As is known in the art, a white noise signal is a random signal with equal intensity at all frequencies, giving it a constant power spectral density. The term is used with this or similar meanings in many scientific and technical disciplines, including physics, acoustic engineering, telecommunications, and statistical prediction. White noise refers to a statistical model of signals and signal sources, rather than any specific signal.

[0149] Sinusoidal signal: Examples of audio calibration signal 522 may include, but are not limited to, a sinusoidal signal. A sinusoidal signal is a signal completely characterized by a mathematical function describing a smooth, periodic oscillation with a fixed frequency. It is named after the function sine. Sinusoidal curves frequently appear in pure and applied mathematics, as well as in physics, engineering, and signal processing.

[0150] Multi-frequency signal: Examples of audio calibration signal 522 may include, but are not limited to, a swept sine signal. A swept sine signal is a signal fully characterized by a mathematical function that describes a smooth periodic oscillation with a frequency that varies over time (typically between two frequencies, such as a logarithmic sweep from 20 Hz to 20 kHz in acoustic applications).

[0151] Impulse function: Examples of the audio calibration signal 522 may include, but are not limited to, an impulse function. An impulse function is a function that is zero everywhere except at the origin, where the amplitude is infinitely high.

[0152] In particular, the three-dimensional model 504 can be transmitted wirelessly (or via wired transmission) from the ACI calibration platform 500 to the modular ACI system 54, for example, in the manner described above. Additionally, one or more audio calibration signals (e.g., audio calibration signals 522) can be acoustically transmitted from the ACI calibration platform 500 to the modular ACI system 54. For example, the audio calibration signals 522 can be presented via, for example, a speaker assembly included within the ACI calibration platform 500, where the audio calibration signals 522 can be acoustically transmitted through the air of the monitored space 130 and “heard” by the modular ACI system 54 via, for example, the audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218.

[0153] Once calibration information (eg, calibration information 532 ) is obtained 600 , the ambient collaborative intelligence process 10 may process 602 the calibration information (eg, calibration information 532 ) to (eg, initially or subsequently) calibrate an ACI system (eg, modular ACI system 54 ).

[0154] In particular, one or more audio calibration signals (e.g., audio calibration signal 522) may be utilized, in whole or in part, to calibrate one or more audio acquisition devices (e.g., audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218) within a three-dimensional space (e.g., monitored space 130).

[0155] As is known in the art, the performance of audio acquisition devices (e.g., audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218) can change over time. Naturally, such audio acquisition devices may completely fail and no longer function at all (which is relatively easy to detect). However, such audio acquisition devices may not completely fail and may only experience performance drift, where the ability of older audio acquisition devices to detect, for example, higher frequency signals decreases (in a manner similar to how human hearing declines with age). Accordingly, and by using one or more audio calibration signals (e.g., audio calibration signal 522), the performance of audio acquisition devices (e.g., audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218) can be determined / measured and compensated (if necessary).

[0156] As described above, the modular ACI system 54 can be configured to direct one or more audio recording beams (e.g., audio recording beams 220, 222, 224) toward one or more encounter participants (e.g., encounter participants 226, 228, 230) of the patient encounter, wherein the modular ACI system 54 can be configured to form these audio recording beams (e.g., audio recording beams 220, 222, 224) using one or more discrete audio capture devices (e.g., audio capture devices 202, 204, 206, 208, 210, 212, 214, 216, 218).

[0157] As noted above, an example of the audio calibration signal 522 can be a white noise signal (e.g., a random signal with equal intensity at all frequencies). Accordingly and as the ACI calibration platform 500 moves (and continuously repositions) within the monitored space 130, the audio generation subsystem 520 within the ACI calibration platform 500 can present the audio calibration signal 522, which (in this example) is a white noise signal with equal spectral intensity from 20 Hz to 20 kHz. It is further assumed that the audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218 are designed to have a flat frequency response (i.e., equally sensitive) to signals in the 100 Hz to 5 kHz range. Accordingly and to test the performance of the audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218, the ACI calibration platform 500 can move (or be moved) to a sufficient number of positions within the monitored space 130 to ensure spatial coverage of the acoustic environment (e.g., by measuring a number of acoustic paths from a typical interaction zone to various ACI devices (e.g., the audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218) within the monitored space 130).

[0158] Since (in this example) the audio calibration signal 522 is a white noise signal with equal spectral intensity from 20 Hz to 20 kHz, the frequency response from 100 Hz to 5 kHz should be flat (i.e., have the same intensity) for each of the audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218. In the event that one of the audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218 does not produce any signal, that particular audio acquisition device can have failed and can need to be replaced.

[0159] And in the event that one of the audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218 performs abnormally, that particular audio acquisition device can need to be compensated. For example:

[0160] • If the audio acquisition device 202 is too sensitive at lkz and is producing an output signal that is 6 db higher than expected at the @1 kHz location, the environment collaborative intelligence process 10 can attenuate the output signal provided by the audio acquisition device 202 at @1 kHz by a factor of 6 db.

[0161] If the audio capture device 206 is insensitive at 3 kHz and is producing an output signal that is 8 dB lower than expected @ 3 kHz, the ambient collaborative intelligence process 10 may be configured to amplify the output signal provided by the audio capture device 206 @ 3 kHz by a factor of 8 dB.

[0162] Furthermore, a three-dimensional model (eg, three-dimensional model 504 ) may be utilized in whole or in part to direct one or more audio recording beams (eg, audio recording beams 220 , 222 , 224 ) within a three-dimensional space (eg, monitored space 130 ).

[0163] As described above and using the object data source 518, the ACI calibration platform 500 can generate a three-dimensional model (e.g., three-dimensional model 504), wherein the objects included / defined therein can be of known types (e.g., doctor's table 508, examination table 510, window 512, HVAC supply vent 516), which can be accomplished via tagging / metadata. Accordingly, the ambient collaborative intelligence process 10 can determine acoustic propagation channel information for use in, for example, robust automatic speech recognition, signal enhancement, audio recording beamforming, acoustic echo cancellation, nulling, and blind source separation. This acoustic path information can be associated with the spatial information defined within the three-dimensional model 504.

[0164] As is known in the art, echo cancellation is a method for improving signal quality by removing echo after it already exists. This method may be referred to as acoustic echo suppression (AES) and acoustic echo cancellation (AEC), and is more commonly known in the context of telecommunications line echo cancellation (LEC). In some cases, these terms are more accurate, as the types and causes of echo are diverse and have unique characteristics, including acoustic echo (sound from a speaker reflected by a microphone, coupled to the microphone, and recorded by the microphone, which may vary significantly over time) and line echo (electrical echo signals caused by, for example, coupling between transmit and receive lines, impedance mismatches, electrical reflections, etc., which vary much less than acoustic echo). Accordingly, and in this configuration, such echo cancellation methods can be utilized, for example, to reject a coupled signal from a second speaker that appears in an audio recording beam directed near a first speaker, while also rejecting a coupled signal from the first speaker that appears in an audio recording beam directed near a second speaker.

[0165] As is known in the art, null-steering precoding is a spatial signal processing method by which a multi-antenna transmitter can null out multi-user interference signals in wireless communications, where null-steering precoding can mitigate the effects of background noise and unknown user interference. Specifically, null-steering precoding can be a method of narrowband signal beamforming that can compensate for the delay in receiving signals from a particular source at different elements of an antenna array. Typically, and to improve the performance of an antenna array, incoming signals can be summed and averaged, where certain signals can be weighted and signal delays can be compensated.

[0166] As known in the art, blind source separation is to separate the set of source signal from the set of mixed signal without the help of the information (or very little information) about source signal or mixed process.Because the main difficulty of blind source separation is its incomplete uncertainty, the method for blind source separation is usually sought to reduce the set of possible solutions in the mode of unlikely getting rid of desired solution.In a method taking principal component analysis and independent component analysis as an example, people seek the minimum relevant or maximum independent source signal in probability or information theory sense.The second method taking non-negative matrix decomposition as an example is to impose structural constraint on source signal.

[0167] As described above, the three-dimensional model 504 may be configured to define one or more of the following:

[0168] Subspaces: One or more subspaces within a three-dimensional space (e.g., monitored space 130) may be defined within three-dimensional model 504, where examples of such subspaces may include, but are not limited to, visitor waiting space 506 (shown as including meeting participants 230, 242).

[0169] Objects: One or more objects within the three-dimensional space (eg, monitored space 130 ) may be defined within the three-dimensional model 504 , where examples of these objects may include, but are not limited to, a physician table 508 and an examination table 510 .

[0170] Features: One or more features within the three-dimensional space (eg, monitored space 130 ) may be defined within the three-dimensional model 504 , where examples of such features may include, but are not limited to, a window 512 .

[0171] Interaction zones: One or more interaction zones within the three-dimensional space (e.g., monitored space 130) may be defined within the three-dimensional model 504, where examples of such interaction zones may include, but are not limited to, an inspection zone 514 (i.e., an area proximate to the inspection table 510).

[0172] Noise Sources: One or more noise sources within the three-dimensional space (eg, monitored space 130 ) may be defined within the three-dimensional model 504 , where examples of such noise sources may include, but are not limited to, HVAC supply vents 516 .

[0173] Each of the particular subspaces, objects, features, interaction zones, and noise sources defined in the three-dimensional model 504 may have a positive / negative impact on the manner in which audio recording beams (e.g., audio recording beams 220, 222, 224) may be directed within the three-dimensional space (e.g., the monitored space 130).

[0174] For example and with respect to the visitor waiting space 506 (which is shown as including the meeting participants 230, 242), the ambient collaborative intelligence process 10 may not be conducive to directing the audio recording beams (e.g., the audio recording beams 220, 222, 224) toward the visitor waiting space 506 because this is a waiting area and there is less likelihood of extracting substantive information from conversations occurring in this area.

[0175] Conversely, and with respect to the physician's table 508 and the examination table 510 , the ambient collaborative intelligence process 10 may facilitate directing audio recording beams (e.g., audio recording beams 220 , 222 , 224 ) toward the physician's table 508 and the examination table 510 because it is more likely that substantive information can be extracted from conversations occurring in these areas.

[0176] With respect to window 512, the ambient collaborative intelligence process 10 may not be conducive to directing audio recording beams (e.g., audio recording beams 220, 222, 224) toward window 512 because this is a hard surface and it is likely that high levels of reflections / echoes / noise may be included in the information captured using the audio recording beams directed toward window 512.

[0177] Conversely, and with respect to inspection area 514 (i.e., areas proximate inspection table 510), ambient collaborative intelligence process 10 may facilitate directing audio recording beams (e.g., audio recording beams 220, 222, 224) toward inspection area 514 because it is more likely that substantive information can be extracted from conversations occurring in these areas.

[0178] From the calibration steps described above, a model of locating objects within a three-dimensional space (e.g., the monitored space 130) may be defined, wherein the model may assist the natural language understanding functionality of the ambient collaborative intelligence process 10. For example, if the ambient collaborative intelligence process 10 identifies a sound coming from a location corresponding to an examination table, this information may be very useful to the natural language understanding functionality of the ambient collaborative intelligence process 10, particularly if the modular ACI system 54 does not include machine vision.

[0179] With respect to the HVAC supply vent 516, the ambient collaborative intelligence process 10 may be unfavorable in directing audio recording beams (e.g., audio recording beams 220, 222, 224) toward the HVAC supply vent 516 because it is a noisy object and it is more likely that high levels of noise may be included in the information captured using the audio recording beams directed toward the HVAC supply vent 516.

[0180] Non-medical applications:

[0181] As mentioned above, although the ambient collaborative intelligent process 10 is described above as being used to automate the collection and processing of clinical encounter information to generate / store / distribute medical records, this is for illustrative purposes only and is not intended to limit the present disclosure, as other configurations are possible and are considered within the scope of the present disclosure. Accordingly, such encounter information may include, but is not limited to, the following examples.

[0182] Financial information:

[0183] For example, the ambient collaborative intelligent process 10 generally (and / or the ACD system 54 specifically) can be configured to automate the collection and processing of financial data generated during meetings to discuss financial information. An example of such a meeting may include, but is not limited to, a meeting between an individual and a financial advisor. For example, the ambient collaborative intelligent process 10 can be configured to supplement the financial advisor's knowledge by recommending products, answering questions, and providing quotes based on conversations the financial advisor has with the client in substantially real time, as well as completing various forms, mortgage applications, stock purchase and sale orders, estate planning documents, and the like.

[0184] Benefits: When configured to process financial information, the benefits that ambient collaborative intelligent process 10 can achieve can be substantial. For example, it is understood that a financial advisor may not know everything about finances and investment vehicles. Accordingly, the ambient collaborative intelligent process 10 (when configured to process financial information) can monitor conversations between the financial advisor and the client. The ambient collaborative intelligent process 10 can then utilize natural language processing and artificial intelligence to identify issues / questions within the conversation and leverage collective knowledge to provide relevant information to the financial advisor.

[0185] For example, suppose a client visits a financial advisor seeking financial advice regarding tax-free / tax-deferred retirement savings. Accordingly, and using the various systems described above (e.g., audio input device 30, display device 32, machine vision input device 34, and audio rendering device 116), the ambient collaborative intelligent process 10 (when configured to process financial information) can monitor the conversation between the financial advisor and the client. Assuming this is the client's first meeting with their financial advisor, the information obtained during this initial meeting can be parsed and used to populate various fields on a client intake form. For example, the client can identify themselves, and their name can be entered on the client intake form. Additionally, the ambient collaborative intelligent process 10 can be configured to define a voiceprint and / or facial print for the client, so that, for example, in the future, when the client seeks access to their data, this voiceprint and / or facial print can be utilized to authenticate the client's identity. Additionally, when the client indicates, for example, their age, their marital status, their spouse's name, their spouse's age, and whether they have children and, if so, their children's ages, all of this information can be used to populate the client intake form.

[0186] Continuing with the above example, let's assume the client is inquiring about tax-free / tax-deferred retirement savings plans. The financial advisor might then ask what their income was last year. Since the ambient collaborative intelligent process 10 may be monitoring this conversation via the audio input device 30, the ambient collaborative intelligent process 10 can "hear" that the client is interested in tax-free / tax-deferred retirement savings plans and their income level. Accordingly, and using the above-described natural language processing and artificial intelligence, the ambient collaborative intelligent process 10 can determine whether the client is eligible for a 401(k) retirement plan, a pre-tax / post-tax traditional IRA plan, and / or a pre-tax / post-tax Roth IRA plan. After making such a determination, the ambient collaborative intelligent process 10 can provide additional information to the financial advisor so that the financial advisor can provide guidance to the client.

[0187] For example, the ambient collaborative intelligent process 10 may present (on the display device 32) a list of tax-free / tax-deferred retirement savings plans that the client is eligible to participate in. Additionally / alternatively, this information may be presented in audio form (e.g., discreetly into earbuds worn by the financial advisor) so that the financial advisor can provide such information to the client.

[0188] Accordingly and by using such a system, the ambient collaborative intelligent process 10 (when configured to process financial information) can monitor (in this example) a conversation between a financial advisor and a client to, for example, collect information and populate a client intake form, generate a voice and / or face print for client authentication, listen to inquiries posed by the client, and respond to those inquiries so that the financial advisor can provide guidance to the client.

[0189] Additionally, the ambient collaborative intelligent process 10 can be configured to monitor the advice that the financial advisor is providing to the client and confirm its accuracy, wherein if the financial advisor makes an error (e.g., telling the client that they qualify for a retirement plan when they are not), a covert correction / notification can be provided to the financial advisor.

[0190] Furthermore, the ambient collaborative intelligent process 10 can be configured to provide such guidance to the financial advisor / client even if such guidance is not sought. For example, if the client indicates that they have children, the ambient collaborative intelligent process 10 can prompt the financial advisor to inquire about what college savings plans (e.g., 529s) they have in place for their children. If these are not in place, the financial advisor can be prompted to explain the tax advantages of such plans.

[0191] In addition, the ambient collaborative intelligent process 10 can be configured to covertly provide the financial advisor with information that may help build a relationship between the financial advisor and the client. For example, suppose the client (during the first meeting between the client and the financial advisor) explains that his wife's name is Jill and the client explains that he and his wife will be visiting Italy in the summer. Assume that the client returns to meet with the financial advisor in the fall. During the first visit, the ambient collaborative intelligent process 10 can (as described above) populate a client intake form that identifies the client's spouse as Jill. In addition, the ambient collaborative intelligent process 10 can make a note that the client and Jill will be visiting Italy in the summer of 2020. Assuming that this follow-up meeting is after the summer of 2020, the ambient collaborative intelligent process 10 can covertly prompt the financial advisor to ask the client if he and Jill enjoyed Italy, thereby enabling the establishment of goodwill between the client and the financial advisor.

[0192] The ambient collaborative intelligent process 10 can also be configured to automatically populate forms that may be required based on the needs of the client. For example, if the client needs to fill out a tax form regarding an IRA deferral, the ambient collaborative intelligent process 10 can be configured to obtain the necessary information based on a conversation between the financial advisor and the client and / or proactively obtain the required information from data sources accessible to the ambient collaborative intelligent process 10, populate the appropriate forms required to execute, for example, the IRA deferral using the data obtained from the data sources, and present (e.g., print) the populated forms so that the client can execute the deferral.

[0193] The ambient collaborative intelligent process 10 can also be configured to implement the functionality of a digital assistant, where the ambient collaborative intelligent process 10 can monitor (in this example) the conversation between the financial advisor and the client so that the items mentioned can be marked for follow-up processing. For example, suppose that in the above-mentioned conversation between the financial advisor and the client, the client expressed that they are interested in establishing a 529 college savings account for their children, and they asked the financial advisor to provide them with information about establishing such a college savings account. Accordingly, the ambient collaborative intelligent process can enter (e.g., into a client-specific to-do list) "Send 529 information to the Smith family." Additionally, if the client says that they would like to schedule a follow-up meeting in three weeks to discuss issues regarding the 529, the ambient collaborative intelligent process 10 can schedule a meeting within the financial advisor's calendar for such discussion.

[0194] Legal Information:

[0195] For example, the ambient collaborative intelligent process 10 generally (and / or the ACD system 54 specifically) can be configured to automate the collection and processing of legal data generated during meetings where legal information is discussed. Examples of such meetings may include, but are not limited to, meetings between legal professionals and the individuals they represent. For example, the ambient collaborative intelligent process 10 can be configured to: recommend strategies, answer questions, and provide advice based on conversations that legal professionals have with clients in substantially real time, as well as complete hearing / deposition transcripts, warrants, court orders / judgments, and various applications for the above and other items, etc., to supplement / supplement the knowledge of legal professionals.

[0196] Benefits: When configured to process legal information, the benefits that can be realized by the ambient collaborative intelligent process 10 can be considerable. For example, it is understood that legal professionals may not know everything about, for example, various legal situations, events, and proceedings. Accordingly, the ambient collaborative intelligent process 10 (when configured to process legal information) can monitor conversations between legal professionals and clients. The ambient collaborative intelligent process 10 can then utilize natural language processing and artificial intelligence to identify topics / issues within the conversation and leverage collective knowledge to provide relevant information to the legal professional.

[0197] For example, assume that a deposition is taking place while the defendant in a lawsuit (represented by a first group of attorneys) is being questioned by the plaintiff in the lawsuit (represented by a second group of attorneys). Accordingly, by utilizing the various systems described above (e.g., audio input device 30, display device 32, machine vision input device 34, and audio rendering device 116), the ambient collaborative intelligent process 10 (when configured to process legal information) can monitor the conversation between the defendant / first group of attorneys and the plaintiff / second group of attorneys. In this case, the ambient collaborative intelligent process 10 (when configured to process legal information) can be configured to perform the functionality of a courtroom transcriber.

[0198] For example, participants in a testimony may be asked to identify themselves (e.g., provide name and title). The ambient collaborative intelligent process 10 may use this information to populate an attendance log for the testimony and may be configured to define a voiceprint and / or faceprint for each attendee of the testimony.

[0199] Accordingly, once the deposition actually begins, the ambient collaborative intelligent process 10 can monitor the deposition and transcribe it (via the aforementioned voice / faceprint), essentially replicating the functionality of a courtroom transcriber. Essentially, the ambient collaborative intelligent process 10 can generate a deposition journal that reads like a movie script, where, for example, each verbal statement is transcribed and the speaker of that verbal statement is identified (via the voice / faceprint).

[0200] Additionally and by using the above-described natural language processing and artificial intelligence, traditional legal tasks can be performed efficiently. For example, suppose an objection is raised (during testimony) and a case law is cited as the basis for the objection. If the non-opposing attorney believes that the case law is no longer valid (e.g., because it was overturned by a higher court), the non-opposing attorney can query the ambient collaborative intelligent process 10 (when configured to process legal information) to determine the status of the case law relied upon (i.e., whether the case law is still valid or has been overturned). The ambient collaborative intelligent process can then provide a response to the non-opposing attorney (e.g., the case is still valid or the case was overturned by the First Circuit Court of Appeals in 2016 and confirmed by the U.S. Supreme Court in 2017).

[0201] Telecommunications information:

[0202] For example, the ambient collaborative intelligent process 10 generally (and / or the ACD system 54 specifically) can be configured to automate the collection and processing of telecommunications data generated during a meeting between a caller and a sales / service representative. Examples of such meetings can include, but are not limited to, telephone and / or chat sessions between a sales / service representative and a customer experiencing a problem with cable television service. For example, the ambient collaborative intelligent process 10 can be configured to supplement / supplement the knowledge of a service representative by recommending plans / products, troubleshooting procedures, answering questions, and providing advice based on conversations the service representative has with the customer in substantially real time.

[0203] Benefits: When configured to process telecom information, the benefits that can be realized by the ambient collaborative intelligent process 10 can be considerable. For example, and understandably, a sales / service representative may not know everything about, for example, various service plans, available products, troubleshooting procedures, and warranty coverage. Accordingly, the ambient collaborative intelligent process 10 (when configured to process telecom information) can monitor the conversation (e.g., voice or text) between the service representative and the caller. The ambient collaborative intelligent process 10 can then utilize natural language processing and artificial intelligence to identify topics / questions in the conversation and leverage collective knowledge to provide relevant information to the telecom salesperson.

[0204] For example, suppose a user of a cable TV service has difficulty tuning into one of the premium channels in their cable TV channel lineup. Accordingly, the user can call (or send a message) their cable TV service and chat with a customer service representative. The ambient collaborative intelligent process 10 (when configured to process telecommunications information) can, for example, utilize caller ID, IP address, and / or voiceprint to identify the caller and obtain information about their account, location, device, service plan, etc.

[0205] For this example, assume that a caller explains to a service representative that they are unable to tune their cable box to a desired channel. The ambient collaborative intelligent process 10 may, for example, first confirm that their current service plan includes the channel the caller is attempting to access. In the event that the service plan does not include such a channel, the ambient collaborative intelligent process 10 may notify the service representative (e.g., via a text-based message visible on a display accessible to the service representative or via an earbud) that the channel is not included in their service plan. The ambient collaborative intelligent process 10 may then provide the service representative with information about which service plans include the channel the caller is inquiring about, to see, for example, whether they would like to upgrade / change their plan to one that includes the channel in question.

[0206] If the channel is indeed included in the caller's current service plan, the ambient collaborative intelligent process 10 can begin to provide the service representative with tips on troubleshooting procedures that can be used to identify the problem. For example, the ambient collaborative intelligent process 10 (via, for example, a display or earbuds) can provide the service representative with a series of steps that the caller can perform to (hopefully) correct the situation. For example, the service representative can instruct the caller to first unplug the cable box from the power outlet, let it sit for 30 seconds, and then plug it back in so that it can restart. If this process does not solve the problem, the list provided by the ambient collaborative intelligent process 10 can instruct the service representative to send a reset signal to the cable box in question. If this process does not solve the problem, the ambient collaborative intelligent process 10 can determine that a new cable box is needed and can assist the service representative in scheduling a service call so that a service technician can replace the faulty cable box.

[0207] Retail information:

[0208] For example, the ambient collaborative intelligent process 10 generally (and / or the ACD system 54 specifically) can be configured to automate the collection and processing of retail data generated during meetings to discuss retail information. Examples of such meetings can include, but are not limited to, meetings between sales associates at department stores and individuals interested in purchasing specific products. For example, the ambient collaborative intelligent process 10 can be configured to: recommend products, answer questions, and provide advice based on conversations the sales associate has with customers in substantially real time, as well as enable checkouts, completion of work order forms, financial / sales agreements, product order forms, warranty forms, etc., to supplement / supplement the sales associate's knowledge.

[0209] Benefits: When configured to process retail information, the benefits that can be realized by the ambient collaborative intelligent process 10 can be considerable. For example, it is understood that a salesperson may not know everything about, for example, the types and locations of products offered. Accordingly, the ambient collaborative intelligent process 10 (when configured to process retail information) can monitor the conversation between the salesperson and the customer. The ambient collaborative intelligent process 10 can then utilize natural language processing and artificial intelligence to identify topics / questions in the conversation and leverage collective knowledge to provide relevant information to the salesperson.

[0210] For example, assume a customer goes to a local department store and they are looking for several items, including a drill. So the customer approaches a sales associate and asks if they sell drills, and if so, where the drills are located. The environmental collaborative intelligence process 10 (when configured to process retail information) can monitor this conversation and identify the issue that needs to be addressed by using the natural language processing and artificial intelligence described above. For example, the environmental collaborative intelligence process 10 can identify the word "drill" in the customer's statement and can check the department store's inventory records and determine that the department store does sell drills. In addition, the environmental collaborative intelligence process 10 can determine that the customer is asking where the drills are located, and after checking the department store's product inventory chart, can determine that the drills are located in the hardware section (aisle 23, racks 16-20).

[0211] Additionally, the environmental collaborative intelligence process 10 can be configured to address additional questions that the customer can have, such as "which drills cost less than $30?", "which drill has the longest warranty?", "do you have any DeWalt drills?", and "do you have drill bits for drilling into concrete?". When providing answers to these questions that the customer has asked, the environmental collaborative intelligence process 10 can publicly provide the information onto a display screen (such as a handheld electronic device) so that the customer can view the information. Alternatively, the environmental collaborative intelligence process 10 can provide the information covertly in an earpiece so that the sales associate can verbally provide the information to the customer.

[0212] In addition, assume a family goes to a local wireless carrier store and asks about cell phones and cell phone plans. Accordingly, by using the various systems described above (e.g., the audio input device 30, the display device 32, the machine vision input device 34, and the audio presentation device 116), the environmental collaborative intelligence process 10 (when configured to process retail information) can monitor the conversation between the family and the sales associate and provide guidance and insights about such conversations by using the natural language processing and artificial intelligence described above. For example, assume the family asks the sales associate if there are any sales / promotions for the latest model iPhone. If so, the environmental collaborative intelligence process 10 (when configured to process retail information) can covertly provide the sales / promotion list to the sales associate via, for example, an earpiece assembly, or can publicly provide the sales / promotion list to the sales associate via, for example, a client electronic device (e.g., a smartphone, a tablet computer, a notebook computer, or a display).

[0213] Additionally, suppose a family asks what the best phone and / or best data plan is for extensive international travel. Accordingly, the ambient collaborative intelligent process 10 (when configured to process retail information) can present a list of suitable phone / data plans, for example, on a client electronic device (e.g., a smartphone, tablet, laptop, or display) so that a salesperson can review such options. Furthermore, if the ambient collaborative intelligent process 10 determines that one or more family members are interested in a cellular phone that is not compatible with cellular networks in various countries around the world, the ambient collaborative intelligent process 10 can prompt the salesperson to ask whether the family member is traveling to, for example, country A, B, or C.

[0214] Additionally, because the ambient collaborative intelligent process 10 may be monitoring the conversation between the family and the salesperson, the ambient collaborative intelligent process 10 can determine the number of cellular phones they are interested in purchasing. The ambient collaborative intelligent process 10 can then review the various promotional plans offered by the cellular phone manufacturer, as well as any available data plan options, so that the ambient collaborative intelligent process 10 can propose the phone and data plan that is most beneficial to the family.

[0215] Additionally, the ambient collaborative intelligent process 10 can monitor the conversation between the family and the salesperson to identify and / or correct any errors or misstatements that the salesperson may have inadvertently made. For example, if the user states that they frequently travel to country X and they are purchasing cell phone Y (which does not work in country X), the ambient collaborative intelligent process 10 can covertly inform the salesperson (e.g., via an earbud) that cell phone Y will not function properly in country X.

[0216] In general:

[0217] As will be appreciated by those skilled in the art, the present disclosure may be embodied as a method, system, or computer program product. Accordingly, the present disclosure may take the form of a fully hardware embodiment, a fully software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, which are collectively referred to herein as a "circuit," "module," or "system." Furthermore, the present disclosure may take the form of a computer program product on a computer-usable storage medium having a computer-usable program code contained therein.

[0218] Any suitable computer-usable or computer-readable medium may be used. A computer-usable or computer-readable medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, apparatus, or propagation medium. More specific examples of computer-readable media (a non-exhaustive list) may include the following: an electrical connection having one or more wires, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a transmission medium (such as a transmission medium supporting the Internet or an intranet), or a magnetic storage device. A computer-usable or computer-readable medium may also be paper or other suitable medium on which the program is printed, since the program can be captured electronically, for example, by optical scanning of the paper or other medium, and then compiled, interpreted, or processed in a suitable manner as necessary, and then stored in a computer memory. In the context of this document, a computer-usable or computer-readable medium may be any medium that can contain, store, communicate, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The computer usable medium may include a propagated data signal in baseband or as part of a carrier wave, which contains the computer usable program code. The computer usable program code may be transmitted using any appropriate medium, including but not limited to the Internet, wired, optical cable, RF, etc.

[0219] The computer program code for performing the operations of the present disclosure can be written in an object-oriented programming language such as Java, Smalltalk, C++, etc. However, the computer program code for performing the operations of the present disclosure can also be written in a traditional procedural programming language, such as the "C" programming language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via a local area network / wide area network / internet (e.g., network 14).

[0220] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer / special-purpose computer / other programmable data processing device, so that instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in one or more blocks of the flowcharts and / or block diagrams.

[0221] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function / act specified in the flowchart and / or block diagram block or blocks.

[0222] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0223] The flow and block diagrams in the figures can illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or combinations of special purpose hardware and computer instructions.

[0224] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0225] The corresponding structure, material, acts, and equivalents of all means or step plus function elements in the claims that follow, if any, are intended to include any structure, material, or act for performing the function in combination with other claimed

[0226] Many implementations have been described. The disclosure of this application has been so detailed as to enable those skilled in the art to make and use various embodiments, and it will be apparent to those skilled in the art that modifications and variations can be made without departing from the scope of the disclosure as defined by the appended claims.

Claims

1. A computer-implemented method executed on a computer device, the method comprising: generating, via a video recording subsystem of an ACI calibration platform, a three-dimensional model of at least a portion of a three-dimensional space, the three-dimensional space containing the ACI system; generating, via an audio generation subsystem of the ACI calibration platform, one or more audio calibration signals for receipt by an audio recording system included within the ACI system, wherein the one or more audio calibration signals include the three-dimensional model of at least the portion of the three-dimensional space, wherein the video recording subsystem of the ACI calibration platform is configured to interface with an object data source, the object data source defining at least one of one or more static objects located within the three-dimensional space, wherein the three-dimensional model is further configured to define one or more interaction zones within the three-dimensional space, wherein the one or more interaction zones include a patient examination zone proximate to the at least one of the one or more static objects within the three-dimensional space; as well as Autonomously positioning the ACI calibration platform within the three-dimensional space via a mobile base assembly of the ACI calibration platform based at least in part on the patient examination area within the three-dimensional space proximate to the at least one of the one or more static objects in the three-dimensional model obtained from the video recording subsystem.

2. The computer-implemented method of claim 1 , further comprising: At least a portion of the three-dimensional space is autonomously cleaned via a cleaning component of the ACI calibration platform. 3 . The computer-implemented method of claim 1 , wherein the ACI calibration platform is configured to be manually positioned within the three-dimensional space.

4. The computer-implemented method of claim 1 , wherein the three-dimensional model is further configured to define at least one of: one or more subspaces within the three-dimensional space; one or more objects within the three-dimensional space; One or more features within the three-dimensional space; and One or more noise sources within the three-dimensional space.

5. The computer-implemented method of claim 1 , wherein the one or more audio calibration signals include one or more of: Noise signal; Sinusoidal signals; and multi-frequency signal.

6. A computer program product residing on a non-transitory computer-readable medium having stored thereon a plurality of instructions that, when executed by a processor, cause the processor to perform operations comprising: generating, via a video recording subsystem of an ACI calibration platform, a three-dimensional model of at least a portion of a three-dimensional space, the three-dimensional space containing the ACI system; as well as generating, via an audio generation subsystem of the ACI calibration platform, one or more audio calibration signals for receipt by an audio recording system included within the ACI system, wherein the one or more audio calibration signals include the three-dimensional model of at least the portion of the three-dimensional space, wherein the video recording subsystem of the ACI calibration platform is configured to interface with an object data source, the object data source defining at least one of one or more static objects located within the three-dimensional space, wherein the three-dimensional model is further configured to define one or more interaction zones within the three-dimensional space, wherein the one or more interaction zones include a patient examination zone proximate to the at least one of the one or more static objects within the three-dimensional space; as well as Autonomously positioning the ACI calibration platform within the three-dimensional space via a mobile base assembly of the ACI calibration platform based at least in part on the patient examination area within the three-dimensional space proximate to the at least one of the one or more static objects in the three-dimensional model obtained from the video recording subsystem.

7. The computer program product of claim 6, further comprising: At least a portion of the three-dimensional space is autonomously cleaned via a cleaning component of the ACI calibration platform.

8. The computer program product of claim 6, wherein the ACI calibration platform is configured to be manually positioned within the three-dimensional space.

9. The computer program product of claim 6, wherein the three-dimensional model is configured to define at least one of: one or more subspaces within the three-dimensional space; one or more objects within the three-dimensional space; One or more features within the three-dimensional space; and One or more noise sources within the three-dimensional space.

10. The computer program product of claim 6, wherein the one or more audio calibration signals include one or more of: Noise signal; Sinusoidal signals; and multi-frequency signal.

11. An environmental collaborative intelligent ACI calibration platform, comprising: a video recording subsystem configured to generate a three-dimensional model of at least a portion of a three-dimensional space containing the ACI system; an audio generation subsystem configured to generate one or more audio calibration signals for receipt by an audio recording system included in the ACI system, wherein the one or more audio calibration signals include the three-dimensional model of at least the portion of the three-dimensional space, wherein the video recording subsystem of the ACI calibration platform is configured to interface with an object data source, the object data source defining at least one of one or more static objects located within the three-dimensional space, wherein the three-dimensional model is further configured to define one or more interaction zones within the three-dimensional space, wherein the one or more interaction zones include a patient examination zone proximate to at least one of the one or more static objects within the three-dimensional space; as well as and a mobile base assembly configured to autonomously position the ACI calibration platform within the three-dimensional space based at least in part on the patient examination area proximate to the at least one of the one or more static objects within the three-dimensional space of the three-dimensional model obtained from the video recording subsystem.

12. The ACI calibration platform according to claim 11, further comprising: A cleaning component is configured to autonomously clean at least a portion of the three-dimensional space. 13 . The ACI calibration platform of claim 11 , wherein the ACI calibration platform is configured to be manually positioned within the three-dimensional space.

14. The ACI calibration platform of claim 11 , wherein the three-dimensional model is further configured to define at least one of the following: one or more subspaces within the three-dimensional space; one or more objects within the three-dimensional space; One or more features within the three-dimensional space; and One or more noise sources within the three-dimensional space.

15. The ACI calibration platform of claim 11 , wherein the one or more audio calibration signals include one or more of the following: Noise signal; Sinusoidal signals; and multi-frequency signal.

Images