Computerized system and method for location management

Abstract

Systems and methods are disclosed that provide a novel framework for centralized management of a location (e.g., a worksite) and / or users working there. The disclosed framework is operable to monitor the performance of tasks performed by users at a location, thereby determining how effective, efficient, compliant, and / or safe the users are. Such information can be used to train machine learning and / or artificial intelligence (ML / AI) models that can be recursively applied to ensure performance levels and safety standards for each user's work at a location. The framework can be further applied to compile learned activity behaviors into computer-executable instructions that can be used to automate specific tasks at a location by specially configured real-world and / or digital assets (e.g., machines and robots on the worksite).

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Provisional Application No. 63 / 428,000, filed November 25, 2022, which is incorporated herein by reference in its entirety.

[0002] The present disclosure relates generally to location monitoring and operations systems, and more particularly to a decision intelligence (DI)-based computerized framework that deterministically monitors and tracks user activities at a location and enables management and execution of such activities based thereon. [Background technology]

[0003] Traditional mechanisms and protocols for managing work sites and the workers (or employees) working there are typically results-based, meaning that the overall efficiency of a work site and the performance of its workers are traditionally based on which tasks are completed and which tasks remain incomplete. Summary of the Invention

[0004] Traditionally, worker and worksite performance is typically judged by managerial evaluations, meaning that a site worker and / or department manager or other supervisory personnel can monitor how the workers are performing and make judgments about the performance and operational status of the worker and / or task based on the captured movements of each worker and the tasks completed.

[0005] Most workplaces today may be equipped with cameras that capture or record user activity, but these cameras are simply placed around the facility to capture worker movements and do not provide insight into each worker's level or state of performance (e.g., whether the worker is fatigued, working slower than normal, or exhibiting movements consistent with injury, stress, or other forms of discomfort).

[0006] In some cases, a workplace may have security and / or safety protocols implemented. However, such protocols are not only completely separate from performance monitoring but are also manually applied by management and / or security personnel. Indeed, conventional systems may utilize, for example, cameras and / or motion sensors to detect unsafe and / or risky behavior. However, the implementation is entirely based on user monitoring and / or user providing input / feedback as to whether captured images of workers correspond to unsafe situations. For example, security personnel may monitor worker activity on the workplace by viewing a closed-loop feed of the workplace.

[0007] The disclosed system and method addresses such shortcomings and others by automatically capturing and recording users' tracked movements associated with tasks performed at a location (e.g., a worksite) and determining therefrom compliance and performance indicators and safety indicators for each worker. As described herein, the disclosed framework enables coordination between compliance, security, and safety monitoring at a location and worker performance evaluation. The disclosed framework allows worker activity to be tracked, monitored, and ensured in accordance with compliance regulations and security / safety standards, which may be related to the type of tasks the worker is performing and / or how (e.g., in what manner) the worker is performing those tasks. Thus, according to some embodiments, operation of the disclosed framework as described herein can ensure that workers and / or the overall operations of a location (e.g., a worksite) are in compliance with required, established, and / or applied compliance regulations (e.g., by worksite and / or industry-wide), performance indicators, and / or safety / security standards / regulations, thereby ensuring a safe, secure, lawful, and efficient worksite.

[0008] As will become apparent from the disclosure herein, such performance and compliance monitoring and safety measures can have benefits beyond simply determining whether a task is completed safely. According to some embodiments, as described in more detail below, the identified (determined) performance indicators can provide information related to, but not limited to, worker behavior, individual worker patterns specific to a particular task, task progress / performance, worker fatigue, worker efficiency, security and / or safety risks associated with worker performance, whether a worker is complying with regulations, and the like, or combinations thereof.

[0009] In some embodiments, as described below, performance indicators can be utilized to generate and communicate real-time alerts, which can be sent to specific workers and / or managers. For example, if a worker is detected not performing a task with a certain level of efficiency and / or safety, an alert can be automatically sent to a manager in the worker's vicinity.

[0010] In some embodiments, the worker's device can generate an output (e.g., an audio alert, a haptic effect, etc.) that can alert the user to a particular situation the user is currently engaged in. For example, if a user is attempting to lift a box of a particular size and weight and the user's performance indicates that the user is currently working at a level that indicates fatigue, an alert can be triggered and sent to the user's device (e.g., the user's smartphone and / or wearable sensor) to alert the user to the serious situation the user is about to embark on. In some embodiments, such an alert can also be sent to the user's manager and / or other users determined to be near the user's current location (e.g., to encourage them to assist the user).

[0011] In another example, if a user is operating a machine in the direction of a known / detected hazard (e.g., a liquid spill), an alert may be automatically generated to notify the user of the hazard (and in some embodiments, reroute the user / machine, as described below).

[0012] Additionally, according to some embodiments, the collected and analyzed data may be further compiled and utilized to perform operational tasks at a location (e.g., a work site) where a real-world asset (e.g., a computer-operated machine) is located. Thus, as described in more detail below, according to some embodiments, the disclosed framework may leverage tracked user activity and behavior to generate computer-executable instructions by which a machine can operate, thereby enabling automated performance of a task by the machine. Thus, in some embodiments, worker kinematics captured, learned, located, detected, or otherwise identified as described herein may be compiled and transferred to a robotic worker, thereby enabling the robotic worker to perform automated tasks based on actions instructed / provided via the kinematics.

[0013] References herein to a "user" should be understood to correspond to a person, worker, employee, or employee working at a location, which may correspond to, but is not limited to, a work site, facility, building, factory, plant, home, etc., and / or other type of geographic area where a user's performance may be monitored.

[0014] Thus, as described herein, the disclosed systems and methods provide centralized management of a location and / or users working at a location based on detected, analyzed, and monitored behavior that can be utilized to determine the performance and / or safety of such users, as well as automate certain activities for performance by a computer-operated machine.

[0015] According to some embodiments, a method is disclosed for a DI-based computerized framework that deterministically monitors and tracks user activities at a location and enables management and execution of such activities based thereon. According to some embodiments, the present disclosure provides a non-transitory computer-readable storage medium for performing the above-described technical steps of the framework's functionality. The non-transitory computer-readable storage medium tangibly stores or tangibly encodes computer-readable instructions that, when executed by a device, cause at least one processor to perform the method for the DI-based computerized framework, which deterministically monitors and tracks user activities at a location and enables management and execution of such activities based thereon.

[0016] According to one or more embodiments, a system is provided, the system including one or more processors and / or computing devices configured to provide functionality according to the embodiments. According to one or more embodiments, the functionality is embodied in method steps performed by at least one computing device. According to one or more embodiments, program code (or program logic) executed by a processor(s) of a computing device to implement functionality according to one or more embodiments is embodied in, by, and / or on a non-transitory computer-readable storage medium. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a block diagram of an exemplary configuration in which the systems and methods disclosed herein may be implemented in accordance with some embodiments of the present disclosure. [Figure 2] FIG. 1 is a block diagram illustrating components of an exemplary system according to some embodiments of the present disclosure. [Figure 3A]FIG. 2 illustrates an exemplary data flow according to some embodiments of the present disclosure. [Figure 3B] FIG. 2 illustrates an exemplary data flow according to some embodiments of the present disclosure. [Figure 4] FIG. 2 illustrates an exemplary data flow according to some embodiments of the present disclosure. [Figure 5] FIG. 2 illustrates an exemplary data flow according to some embodiments of the present disclosure. [Figure 6] FIG. 1 illustrates an example implementation of an architecture according to some embodiments of the present disclosure. [Figure 7] FIG. 1 illustrates an example implementation of an architecture according to some embodiments of the present disclosure. [Figure 8] FIG. 1 is a block diagram illustrating a computing device that illustrates an example of a client device or server device for use in various embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0018] Features and advantages of the present disclosure will become apparent from the following description of the embodiments illustrated in the accompanying drawings, in which reference characters refer to the same parts throughout. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the present disclosure.

[0019] The present disclosure will now be described in more detail with reference to the accompanying drawings. The accompanying drawings, which form a part of this specification, illustrate, by way of non-limiting example, certain exemplary embodiments. However, because the subject matter can be embodied in a variety of different forms, the subject matter covered or claimed herein is not intended to be construed as limited to the exemplary embodiments described herein, which exemplary embodiments are provided for illustrative purposes only. Likewise, a reasonably broad scope is intended for the subject matter claimed or covered. In particular, for example, the subject matter may be embodied as a method, device, component, or system. Thus, embodiments can take the form of, for example, hardware, software, firmware, or any combination thereof (excluding software itself). Therefore, the following detailed description is not intended to be construed in a limiting sense.

[0020] Throughout this specification and the claims, terms may have nuanced meanings beyond their explicitly stated meanings as suggested or implied by context. Similarly, the phrase "in one embodiment" as used herein does not necessarily refer to the same embodiment, and the phrase "in another embodiment" as used herein does not necessarily refer to a different embodiment. For example, claimed subject matter is intended to include all or some combinations of the example embodiments.

[0021] Generally, terms can be understood, at least in part, from their usage in context. For example, terms such as "and," "or," and "and / or" as used herein can include a variety of meanings that can depend, at least in part, on the context in which such terms are used. Generally, when "or" is used to link a list, such as A, B, or C, it is intended to mean A, B, and C used in an inclusive sense herein, as well as A, B, or C used in an exclusive sense. Furthermore, the term "one or more" as used herein may be used to describe a feature, structure, or characteristic in a singular sense, or may be used to describe a combination of features, structures, or characteristics in a plural sense, depending, at least in part, on the context. Similarly, terms such as "a," "an," and "the" may be understood to convey singular usage or to convey plural usage, depending, at least in part, on the context. Furthermore, the term "based on" is not necessarily intended to convey an exclusive set of factors, but instead may allow for the presence of additional factors not necessarily explicitly described, depending, at least in part, on the context.

[0022] The present disclosure is described below with reference to block diagrams and operational diagrams of methods and devices. It is understood that each block of the block diagrams or operational diagrams, and combinations of blocks in the block diagrams or operational diagrams, can be implemented by analog or digital hardware and computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer to modify its functionality as described herein, or to a processor of a special-purpose computer, ASIC, or other programmable data processing device, where the instructions executed by the processor of the computer or programmable data processing device implement the functions / acts specified in the block diagrams or operational blocks. In some alternative implementations, the functions / acts noted in the blocks may not occur in the order noted in the operational diagrams. For example, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may be executed in the reverse order, depending on the functions / acts involved.

[0023] For purposes of this disclosure, a non-transitory computer-readable medium (or computer-readable storage medium / media) stores computer data in machine-readable form, which may include computer program code (or computer-executable instructions) executable by a computer. By way of example and not limitation, a computer-readable medium may include a computer-readable storage medium for tangible or fixed storage of data or a communication medium for transitory interpretation of signals containing code. As used herein, a computer-readable storage medium refers to physical or tangible storage (as opposed to signals) and includes, without limitation, volatile, non-volatile, removable, and non-removable media implemented in any method or technology for the tangible storage of information, such as computer-readable instructions, data structures, program modules, or other data. A computer-readable storage medium includes, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technology, optical storage, cloud storage, magnetic storage devices, or other physical or material media that can be used to tangibly store desired information, data, or instructions and that can be accessed by a computer or processor.

[0024] For purposes of this disclosure, the term "server" should be understood to refer to a service point that provides processing, database, and / or communication capabilities. By way of example and not limitation, the term "server" can refer to a single physical processor with associated communication, data storage, and / or database capabilities, or it can refer to a networked or clustered complex of processors and associated network and storage devices, as well as operating software, one or more database systems, and application software that support the services provided by the server. A cloud server is one example.

[0025] For purposes of this disclosure, a "network" should be understood to refer to a network that can connect devices so that communications can be exchanged, such as, for example, between a server and a client device or other type of device, or between wireless devices linked via a wireless network. A network can also include mass storage, such as, for example, a network-attached storage (NAS), a storage area network (SAN), a content delivery network (CDN), or other forms of computer- or machine-readable media. A network can include the Internet, one or more local area networks (LANs), one or more wide area networks (WANs), a wired network, a wireless network, a cellular network, or a combination thereof. Similarly, subnetworks employing or conforming to or compatible with different protocols can interoperate within a larger network.

[0026] For purposes of this disclosure, a "wireless network" should be understood to connect client devices to a network. A wireless network can employ a standalone ad-hoc network, a mesh network, a wireless LAN (WLAN) network, a cellular network, etc. A wireless network can also employ multiple network access technologies, including Wi-Fi, Long Term Evolution (LTE), wireless router mesh, or second-, third-, fourth-, or fifth-generation (2G, 3G, 4G, 5G) cellular technologies, mobile edge computing (MEC), Bluetooth, 802.11b / g / n, etc. Network access technologies enable widespread coverage for devices, such as client devices, with varying degrees of mobility.

[0027] That is, a wireless network may include virtually any type of wireless communication mechanism by which signals may be communicated between devices, such as client devices or computing devices, between networks, or within networks.

[0028] A computing device may send and receive signals, such as over a wired or wireless network, or may process or store signals, such as in memory as physical memory states, and thus may operate as a server. Devices that can operate as servers may include, by way of example, dedicated rack-mounted servers, desktop computers, laptop computers, set-top boxes, integrated devices that combine various functions, such as the functions of two or more of the above-mentioned devices, and the like.

[0029] For purposes of this disclosure, a client (or user, entity, subscriber, or customer) device may include a computing device capable of sending and receiving signals, such as over a wired or wireless network. A client device may include, for example, a desktop computer or a portable device, such as a mobile phone, smartphone, radio frequency (RF) device, infrared (IR) device, near field communication (NFC) device, personal digital assistant (PDA), handheld computer, tablet computer, phablet, laptop computer, set-top box, wearable computer, smart watch, integrated or distributed device that combines various functions, such as those of the above devices, and the like.

[0030] Client devices may vary in functionality and features. The claimed subject matter is intended to cover a wide range of potential variations, such as web-enabled client devices or the devices described above, including, for example, high-resolution screens (e.g., HD or 4K), one or more physical or virtual keyboards, mass storage, one or more accelerometers, one or more gyroscopes, a global positioning system (GPS) or other location-determining type functionality, or displays with advanced functionality, such as touch-sensitive color 2D or 3D displays.

[0031] Specific embodiments and principles are described in more detail with reference to the drawings. According to some embodiments, the disclosed framework as described herein provides novel capabilities for automatically capturing and recording tracked user movements associated with tasks performed at a location (e.g., a job site) and identifying (determining) performance and safety metrics for each worker therefrom. As described herein, the disclosed framework enables coordination of location security and safety monitoring with worker performance evaluation, so that user safety can be tracked, monitored, and ensured in relation to the types of tasks users are performing and / or how (e.g., in what manner) they are performing such tasks.

[0032] Thus, as described herein, the disclosed systems and methods can be utilized to provide centralized management of a location and / or users working at such a location based on detected, analyzed, and monitored behavior, to determine the performance and / or safety of such users, and can also automate certain activities for performance by computer-operated machines.

[0033] By way of non-limiting example, the disclosed framework can be utilized / implemented for, but not limited to, ranking task, worker, and / or worksite performance; third-party worker monitoring to ensure performance, compliance, and / or safety (e.g., which may be performed via peer devices, monitoring devices, and / or third-party devices); high-resolution evidence collection; determining whether task, performance, and / or other characteristics / features / attributes of workers / worksites comply with local / regional and universal guidelines, regulations, and / or laws; worker scheduling or routing (e.g., relocating or rebalancing the workforce across project zones and / or worksites); and the like, or some combination thereof.

[0034] 1, system 100 is shown that may operate and / or be configured in association with a location. As noted above, a location may correspond to, but is not limited to, a site, facility, building, factory, plant, home, etc., and / or other type of geographic area where real-world or digital tasks may be performed / completed.

[0035] According to some embodiments, system 100 includes UE 102 (e.g., a client device, as described above and below in connection with FIG. 8 ), sensor(s) 112, peripheral device 110, network 104, cloud system 106, database 108, operations engine 200, and imaging device(s) 114. While system 100 is shown to include these components, it should not be construed as limiting. Those skilled in the art will readily appreciate that various numbers of UEs, peripheral devices, sensors, cloud systems, databases, networks, and / or imaging devices may be utilized without departing from the scope of the present disclosure. However, for purposes of explanation, system 100 will be described with reference to the example shown in FIG. 1 .

[0036] According to some embodiments, the UE 102 may be any type of device, such as, but not limited to, a mobile phone, a tablet, a laptop, a sensor, a wearable device, a wearable camera, an Internet of Things (IoT) device, an autonomous machine, or any other type of modern device. In some embodiments, the UE 102 may be a device associated with an individual (or set of individuals) for whom security / safety services are provided. In some embodiments, the UE 102 may represent a device of a security entity (e.g., a security provider, in which case the device is a security panel and has corresponding sensors 112 as described herein).

[0037] In some embodiments, the UE 102 may represent a reflective marker whose operational data may be tracked via the imaging device 114, as described below.

[0038] In some embodiments, the peripheral device 110 may be any type of peripheral device that can be connected to the UE 102. Examples of such peripheral devices include, but are not limited to, wearable devices (e.g., smart watches), printers, sensors, etc. In some embodiments, the peripheral device 110 may be any type of device that can be connected to the UE 102 via any known or future known pairing mechanism, including, but not limited to, Bluetooth, Bluetooth Low Energy (BLE), NFC, etc.

[0039] According to some embodiments, the sensor 112 may correspond to a sensor associated with the location of the system 100. In some embodiments, the UE 102 may have multiple sensors 112 associated therewith to collect data from a user. By way of non-limiting example, the sensor 112 may include sensors on the UE 102 (e.g., a smartphone) and / or on a peripheral device (e.g., a paired smartwatch). For example, the sensor 112 may be, but is not limited to, an accelerometer or gyroscope that tracks patient movement. For example, an accelerometer may measure acceleration, which is the rate of change of an object's velocity, in meters per second squared (m / s 2 ) or G-force (g). Thus, for example, the collected sensor data may indicate patient movement, breathing, restlessness, convulsions, temporary pauses, or other detected movement and / or inactivity that may commonly occur while performing a task. In some embodiments, the sensors 112 may also track and / or collect x, y, z coordinates of the user and / or the UE 102 to detect user movement.

[0040] According to some embodiments, the sensors 112 may be specially configured for positional placement relative to the user. For example, the sensors 112 may be located on the user's extremities (e.g., arms and legs) or configured on the user's chest (e.g., body cameras such as hand-worn, foot-worn, and / or head / helmet-mounted cameras). Such sensors 112 may be secured to the user using bands, adhesives, straps, etc., or some combination thereof. For example, the sensors may be fabric wristbands (or other types of materials / garments) having contrast points for detection by an imaging modality (e.g., imaging device 114 and / or a camera associated with UE 102, etc.).

[0041] According to some embodiments, one or more of the sensors 112 may include, but are not limited to, a temperature sensor, a thermal gradient sensor, a barometer, an altimeter, an accelerometer, a gyroscope, a humidity sensor, a magnetometer, an inclinometer, an oximeter, a colorimetric monitor, a sweat analysis sensor, a galvanic skin response sensor, an interface pressure sensor, a flow sensor, a stretch sensor, a microphone, or the like, and / or any combination thereof.

[0042] According to some embodiments, sensors 112 may be integrated into the operation of the UE 102 to monitor the status of the user. In some embodiments, data acquired by sensors 112 may be used to train machine learning and / or artificial intelligence (ML / AI) algorithms used by the UE 102 and / or the artificial intelligence controlling the UE 102. According to some embodiments, such ML / AI may include, but is not limited to, computer vision, neural network analysis, etc., as described below.

[0043] In some embodiments, sensors 112 may be located at specific positions (or sublocations) of the location. Such sensors enable tracking of user location, movement, and / or inactivity, as described herein. In some embodiments, such sensors may be associated with security sensors, such as cameras, motion detectors, door and window contacts, heat and smoke detectors, passive infrared (PIR) sensors, etc. In some embodiments, sensors may be associated with devices associated with the location of system 100, such as lights, smart locks, garage doors, smart appliances (e.g., thermostats, refrigerators, televisions, personal assistants (e.g., Alexa®, Nest®, etc.), smartphones, smartwatches or other wearable devices, tablets, personal computers, etc., and some combination thereof.

[0044] According to some embodiments, imaging device 114 refers to a device used to obtain, capture, and / or record images (e.g., take photographs and / or record videos). For example, imaging device 114 may achieve image capture through any type of mechanism now known or hereafter known. For example, imaging device 114 may be, but is not limited to, a camera, an infrared camera, a thermal camera, etc. (e.g., any type of camera now known or hereafter known that is sensitive to the visible and non-visible spectrum). Thus, imaging device 114 may include any mechanical, digital, and / or electrical device capable of capturing and / or recording a visual image or set of visual images (e.g., image bursts, video frames, etc.).

[0045] Thus, in some embodiments, the imaging device 114 may receive or generate imaging data from multiple imaging devices 106, including, but not limited to, cameras worn by a user (e.g., body cameras (e.g., hand-worn, foot-worn, and / or head / helmet-mounted cameras)), cameras mounted on a ceiling or other structure around, above, or below the work site and / or machinery, cameras mounted on a tripod or other independent mounting device, cameras that may be incorporated into wearable devices (UE 102) such as augmented reality devices such as Google® Glass or Microsoft® Honolens, cameras that may be integrated into machinery, or any camera or other imaging device that may be present at the work site.

[0046] In some embodiments, network 104 may be any type of network, such as a wireless network (as described above), a cellular network, the Internet, etc. Network 104 facilitates the connection of components of system 100, as shown in FIG.

[0047] According to some embodiments, cloud system 106 may be any type of cloud operating platform and / or network-based system on which applications, operations, and / or other forms of network resources may be located. For example, system 106 may be a service provider and / or network provider on which services and / or applications may be accessed, sourced, or executed. For example, system 106 may represent a cloud-based architecture associated with a security system provider, with associated network resources hosted on the Internet or a private network (e.g., network 104), that enables worker safety management (via engine 200) as described herein.

[0048] In some embodiments, cloud system 106 may be a private cloud, where access is restricted by network isolation, such as preventing outside access, or by using encryption to limit access to only authorized users. Alternatively, cloud system 106 may be a public cloud that is widely accessible via the Internet. Public clouds may be unsecured or have limited security features.

[0049] In some embodiments, the cloud system 106 may include servers and / or databases accessible via the network 104. In some embodiments, the database 108 of the cloud system 106 may store data sets of data and metadata associated with users of the UEs 102 / devices 110 and local and / or network information related to the UEs 102 / devices 110, sensors 102, imaging devices 114, and services and applications provided by the cloud system 106 and / or operations engine 200.

[0050] In some embodiments, the cloud system 106 may provide a private / proprietary management platform whereby the engine 200, described below, accommodates the novel functionality that the system 106 enables, hosts, and provides to the network 104 and other devices / platforms operating thereon.

[0051] 6 and 7, in some embodiments, the exemplary computer-based system / platform, exemplary computer-based device, and / or exemplary computer-based component of the present disclosure may be specially configured to operate in a cloud computing / architecture 106 using a web browser, mobile app, thin client, terminal emulator, or other endpoint 704. The cloud computing / architecture 106 may be, but is not limited to, an Infrastructure as a Service (IaaS) 710, a Platform as a Service (PaaS) 708, and / or a Software as a Service (SaaS) 706. FIGS. 6 and 7 are schematic diagrams of non-limiting implementations of cloud computing / architectures in which the exemplary computer-based system for management customization and control of network-hosted APIs of the present disclosure may be specially configured to operate.

[0052] 1, according to some embodiments, database 108 may represent data storage for one platform (e.g., a network-hosted platform such as cloud system 106 described above) or multiple platforms. Database 108 may receive storage instructions / requests, for example, from engine 200 (and associated microservices), which may be in any type of format now known or to be known, such as, for example, Standard Query Language (SQL).

[0053] According to some embodiments, database 108 may represent a distributed ledger of a distributed network. In some embodiments, the distributed network may include multiple distributed network nodes, each of which includes and / or represents a computing device associated with at least one entity (e.g., an entity associated with cloud system 106 described above). In some embodiments, each distributed network node may include at least one distributed network data store configured to store distributed network-based data objects for at least one entity. For example, database 108 may correspond to a blockchain, and the distributed network-based data objects may include, but are not limited to, account information, medical information, entity identification information, wallet information, device information, network information, credentials, security information, permissions, identifiers, smart contracts, transaction history, etc., or any other type of data / metadata now known or hereafter known related to entity and / or user information, structure, business, and / or demographic information, among others.

[0054] In some embodiments, a blockchain can include one or more private and / or private permissioned cryptographically secured distributed databases, such as, but not limited to, blockchain (distributed ledger technology), Ethereum (Ethereum Foundation, Zug, Switzerland), and / or other similar distributed data management technologies. For example, as used herein, a distributed database such as a distributed ledger ensures data integrity by generating a digital chain of data blocks linked to each other by cryptographic hashes of the data records within the data blocks. For example, a cryptographic hash of at least some of the data records within an initial block, possibly combined with some of the data records within a previous block, is used to generate a block address of a new digital identity block following the initial block. As data blocks stored in one or more data blocks are updated, new block data is generated containing each updated data record and linked to the previous block with an address based on the cryptographic hash of at least some of the data records within the previous block. In other words, the linked blocks form a blockchain that includes a traceable address sequence that can be used to track updates to the data records contained therein. The linked blocks (or blockchain) are distributed among multiple network nodes in a computer network, and each node may maintain a copy of the blockchain. A malicious network node attempting to compromise the integrity of the database would have to recreate and redistribute the blockchain faster than legitimate network nodes, which is computationally infeasible in most cases. In other words, data integrity is ensured by multiple network nodes in the network having copies of the same blockchain. In some embodiments, as used herein, a central trusted authority for sensor data management may not be required to ensure the integrity of a distributed database hosted by multiple nodes in the network.

[0055] In some embodiments, implementations of the exemplary distributed blockchain-type ledgers of the present disclosure with associated devices may be configured to affect transactions involving Bitcoin and other cryptocurrencies with each other and with so-called FIAT money or currencies, and vice versa.

[0056] In some embodiments, exemplary distributed blockchain-type ledger implementations of the present disclosure with associated devices may be configured to utilize smart contracts, which are computer processes that facilitate, verify, and / or enforce the negotiation and / or execution of one or more specific activities between users / parties. For example, exemplary smart contracts may be configured to be partially or fully self-executing and / or self-enforcing. In some embodiments, exemplary asset tokenization distributed blockchain-type ledger implementations of the present disclosure may utilize a smart contract architecture that may be implemented with a replicated asset registry and contract execution using cryptographic hash chains and Byzantine fault-tolerant replication. For example, each node in a peer-to-peer or blockchain distributed network may act as a title registry and escrow, thereby effecting changes of ownership and implementing a predetermined set of rules governing transactions on the network. For example, each node may check the work of other nodes and, in some cases, may act as a miner or validator, as described above.

[0057] The operations engine 200 may include components for the disclosed functionality as described above and in more detail below. According to some embodiments, the operations engine 200 may be a dedicated machine or processor and may be hosted by a device on the network 104, within the cloud system 106, and / or on the UE 102 (and / or peripheral device 11). In some embodiments, the operations engine 200 may be hosted by a server and / or set of servers associated with the cloud system 106.

[0058] According to some embodiments, as described in more detail below, operations engine 200 may be configured to implement and / or control multiple services and / or multiple microservices, each of which is configured to execute multiple workflows related to performing the disclosed security controls. Non-limiting examples of such workflows are provided below in connection with at least Figures 3A, 3B, 4, and 5.

[0059] According to some embodiments, as described above, the operations engine 200 may function as an application provided by the cloud system 106. In some embodiments, the operations engine 200 may function as an application installed on a server, network location, and / or other type of network resource associated with the system 106. In some embodiments, the operations engine 200 may function as an application running via an edge device (not shown) at a location associated with the system 100. In some embodiments, the engine 200 may function as an application installed on and / or executed on the UE 102. In some embodiments, such an application may be a web-based application accessed over the network 104 from the cloud system 106 by devices associated with the UE 102, peripheral devices 110, and / or sensors 112. In some embodiments, the engine 200 may be configured and / or installed as an extension script, extension program, or extension application (e.g., a plug-in or extension) to another application or program provided by the cloud system 106 and / or running on the UE 102, peripheral devices 110, and / or sensors 112.

[0060] 2, according to some embodiments, operations engine 200 includes identification module 202, analysis module 204, identification module 206, and output module 208. It should be understood that the engines and modules described herein are not exhaustive, and that additional or fewer engines and / or modules (or sub-modules) may be applicable to the system and module embodiments described herein. Details of the operation, configuration, and functionality of engine 200 and each of its modules, and their role in embodiments of the present disclosure, are described below.

[0061] 3A, process 300 provides a non-limiting exemplary embodiment of the disclosed framework. According to some embodiments, process 300 provides a computerized mechanism for centrally managing a location and the workers (also referred to as users) working there through detection, analysis, and monitored movements of the workers.

[0062] According to some embodiments, steps 302-308 of process 300 may be performed by the identification module 202 of the operations engine 200. Steps 310-314 (and sub-steps 350-364 of step 314 in FIG. 3B) may be performed by the analysis module 204. Step 316 may be performed by the identification module 206. Steps 318-322 may be performed by the output module 208.

[0063] According to some embodiments, process 300 begins at step 302, where engine 200 may identify a user at a location (e.g., a work site, as described above). In some embodiments, the identification may be in response to, but not limited to, a request from another user (e.g., a supervisor at the location), motion detection by a camera, inactivity detected by a camera associated with a user, a request by a user to be monitored, a schedule for monitoring specific staff at the location, a type of task associated with a user, or some combination thereof.

[0064] In some embodiments, the operation, execution, and implementation of engine 200 may be associated with a device (e.g., a locally operated device) at the location. In some embodiments, engine 200 may operate on a remotely located device that can remotely access data collected about the location and perform the operational steps outlined herein in connection with process 300 (and / or processes 400, 500, described below).

[0065] In step 304, engine 200 may connect to a sensor. A sensor, as described above, may be associated with a user and / or a particular location at or around the location. Such connection may be performed via at least the mechanisms described above in connection with FIG. 1. In some embodiments, a connection between engine 200 and a sensor may already be established. As such, step 304 may include identifying the sensor, and in some embodiments, sending a ping message to check the connection.

[0066] In some embodiments, the connecting of step 304 may include configuring each identified sensor and pairing / connecting it with the engine 200 and / or with each other. Thus, in some embodiments, referring to FIG. 1 , for example, the sensors 112 may be paired with each other, with the engine 200, and / or with the UE 102 via a connection protocol provided and / or enabled via the engine 200. For example, a sensor 112 may be paired / connected with another sensor 112, the engine 200, the UE 102, and / or the peripheral device 110 via BLE technology. In some embodiments, the sensor 112 may be paired and / or connected with another sensor 112, the engine 200, the UE 102, and / or the peripheral device 110 via a physical wired connection (e.g., fiber optic, Ethernet, coaxial, and / or other type of wiring known or hereafter known that wires a location for network connectivity of devices operating therein). In some embodiments, a sensor 112 may be paired and / or connected with another sensor 112, the engine 200, the UE 102, and / or a peripheral device 110 via a cloud-to-cloud (C2C) connection (e.g., establishing a connection with a third-party cloud that interfaces with the cloud system 106). In some embodiments, a sensor 112 may be paired / connected via a combination of networking, a wired connection, and / or C2C. In some embodiments, a sensor 112 may be paired to extend the sensor's configured range to detect certain types of events.

[0067] In some embodiments, the sensor 112 may be paired / connected with the imaging device 114 as described below in connection with at least step 396 .

[0068] At step 306, engine 200 may identify a camera at the location (e.g., a camera located at the location and / or associated with UE 102, as described above). In some embodiments, the identification may include connecting the camera via network 104 and / or any of the pairing mechanisms described above. In some embodiments, step 306 may include identifying the camera (and, in some embodiments, sending a Ping message to verify camera connection and / or response). In some embodiments, step 306 may include pairing / connecting the camera with engine 200, sensor 112, UE 102, and / or peripheral device 114, which may occur via at least any of the mechanisms described above in connection with step 304.

[0069] In step 308, engine 200 may identify tasks assigned to the user. According to some embodiments, the assigned tasks may be provided by the user, provided by a manager or other user at the location, identified in step 302, extracted from a worksite manifest, identified or located from a worker activity log, identified via a captured image of the user, etc., without limitation.

[0070] It should be understood that the description herein of a single task with a single user at a single location should not be construed as limiting, and those skilled in the art will readily appreciate that the application of the disclosed engine 200 features and capabilities may be extended to multiple tasks, multiple users, and multiple locations without departing from the scope of the present disclosure.

[0071] According to some embodiments, step 308 may include identifying a task schedule (or manifest) for the worksite. The schedule may correspond to a particular shift, worker, type of task, location within the location, type of machine used, etc., and combinations thereof. In some embodiments, step 308 may include engine 200 searching a storage (e.g., database 108) where the schedule is stored to identify the user's task schedule. The search may include a query including an identifier for the user identified in step 302. In some embodiments, step 308 may include extracting task information according to the schedule from an electronic document that includes at least the user's schedule.

[0072] In some embodiments, step 308 may include analyzing the user's activity in real time to identify what tasks the user is performing. In some embodiments, such analysis may include capturing a set of images of the user (e.g., a single image or multiple images) and analyzing those images to identify what activities the user is performing in the images. The output of the analysis may be compared with the user's schedule information to identify (and, in some embodiments, confirm) the user's specific activities.

[0073] In some embodiments, by way of non-limiting example, engine 200 may utilize any type of artificial intelligence or machine learning algorithm or technique now known or hereafter known, including, but not limited to, computer vision, classifiers, feature vector analysis, decision trees, boosting, support vector machines, neural networks (e.g., convolutional neural networks (CNNs), recurrent neural networks (RNNs), etc.), nearest neighbor algorithms, naive Bayes, bagging, random forests, logistic regression, etc.

[0074] In some embodiments, and optionally in combination with any of the above or below embodiments, the neural network technique may be, but is not limited to, a feedforward neural network, a radial basis function network, a recurrent neural network, a convolutional network (e.g., U-net), or any other suitable network. In some embodiments, and optionally in combination with any of the above or below embodiments, the neural network implementation may be performed as follows: a. Define neural network architectures / models. b. Transfer the input data to the neural network model. c. Train the model incrementally. d. Determine the accuracy for a particular number of time steps. e. Apply the trained model to process newly received input data. f. Optionally, and in parallel, continue training the trained model at a predetermined cycle.

[0075] In some embodiments, and optionally in combination with any of the above or below embodiments, the trained neural network model may specify the neural network by at least a neural network topology, a set of activation functions, and connection weights. For example, the topology of the neural network may include the configuration of nodes in the neural network and the connections between such nodes. In some embodiments, and optionally in combination with any of the above or below embodiments, the trained neural network model may be specified to include other parameters, including, but not limited to, a bias value / function and / or an aggregation function. For example, the activation function of a node may be a step function, a sine function, a continuous or piecewise linear function, a sigmoid function, a hyperbolic tangent function, or any other type of mathematical function that indicates a threshold at which the node is activated. In some embodiments, and optionally in combination with any of the above or below embodiments, the aggregation function may be a function that combines (e.g., sum, product, etc.) the input signals to the node. In some embodiments, and optionally in combination with any of the above or below embodiments, the output of the aggregation function may be used as input to the activation function. In some embodiments, and optionally in combination with any of the above or below embodiments, the bias may be a constant value or a function that may be used by the aggregation function and / or activation function to make a node more or less likely to be activated.

[0076] For example, engine 200 can capture images of a user (e.g., images can be captured according to a predetermined time period) and input them into software defined, for example, by computer vision. The output can be compared to a schedule to identify / confirm the user's activities. In some embodiments, such output can be converted into an n-dimensional feature vector, and the nodes and edges of the output vector can be compared to the feature vector of the schedule. In some embodiments, if the output matches a task (e.g., a node) on the feature vector of the schedule to at least a threshold degree (e.g., which can be determined by a similarity analysis performed by engine 200 executing a similarity analysis algorithm (e.g., cosine similarity)), then a task assigned to the user can be identified / confirmed.

[0077] In step 310, engine 200 may monitor user activity related to the performance of assigned tasks. In some embodiments, monitoring may be enabled by engine 200 collecting and analyzing data collected via sensors 102 and / or imaging devices 114 identified / connected in steps 304 and 306, respectively.

[0078] According to some embodiments, the disclosed monitoring may be performed according to settings / criteria, including, but not limited to, detection of user activity, detection of user presence (e.g., via sensors / cameras), user identification (e.g., in step 302), request from another user to perform monitoring, time of day, date, continuous, predetermined intervals, dynamically determined intervals (which may be based on the type of activity identified in step 308), etc., or some combination thereof. For example, if a task is determined / identified as a dangerous task (e.g., handling hazardous materials), the monitoring cycle / interval may be increased depending on the determined / perceived risk of the task.

[0079] Thus, in some embodiments, monitoring enables the capture of sensor and / or camera data (e.g., via sensors connected / identified in steps 304 and 306, respectively), as described below.

[0080] In step 312, engine 200 may capture data corresponding to the monitored activity. According to some embodiments, the captured data may be stored in a database in association with a user identifier (ID) and / or a task (and / or location) identifier (ID).

[0081] According to some embodiments, the captured data may correspond to data that is live streamed / collected by the sensors 112 and / or cameras 114, previously streamed / collected and stored data, and / or delayed streamed data.

[0082] According to some embodiments, engine 200 may operate to trigger identified sensors and / or cameras to begin collecting data. According to some embodiments, sensor data may be collected continuously and / or according to predetermined periods or intervals. In some embodiments, sensor data may be collected based on detected events. In some embodiments, the type and / or amount of sensor data may be directly related to the type of sensor. For example, a motion detection sensor may collect sensor data only if motion is detected within the field of view of the motion detection sensor. As another non-limiting example, a gyroscope sensor on a user's smartphone may detect whether the user is moving, and the type and / or indicators of such motion.

[0083] Thus, in some embodiments, the camera data may correspond to captured images. As noted above, images may be captured by the camera on demand, continuously, at predetermined intervals, etc., but are not limited to such.

[0084] In step 314, engine 200 may analyze the captured data using a trained AI / ML algorithm. According to some embodiments, the AI / ML-based analysis may be performed using at least the AI / ML algorithms described above in connection with step 308. For example, engine 200 may perform step 314 using any type of AI / ML algorithm or technique known or hereafter known, including, but not limited to, computer vision, classifiers, feature vector analysis, decision trees, boosting, support vector machines, neural networks (e.g., convolutional neural networks (CNNs), recurrent neural networks (RNNs), etc.), nearest neighbor algorithms, naive Bayes, bagging, random forests, logistic regression, etc.

[0085] According to some embodiments, the analysis performed by step 314 may be performed by the substeps outlined in FIG. 3B (eg, steps 350-364).

[0086] Referring to FIG. 3B, a non-limiting exemplary embodiment of the computational analysis applied and / or performed by engine 200 in some embodiments of step 314 in relation to the data captured in step 312 is shown.

[0087] According to some embodiments, processing of step 314 begins with sub-step 350, in which engine 200 may execute a classifier algorithm to identify the type of activity performed by the user. In some embodiments, the input for sub-step 350 may be captured data, as described above.

[0088] According to some embodiments, the applied classifier algorithm may be any type of computational analytical classifier capable of analyzing the captured sensor / camera data to identify the type of activity. For example, the engine 200 may execute a TensorFlow algorithm.

[0089] In sub-step 352, engine 200 may output modeled data based on execution of a classifier (e.g., a TensorFlow algorithm). Thus, for example, engine 200 may identify the type of activity a user is performing in an image captured by a camera and / or identify the movement performed based on sensor data.

[0090] In substep 354, the determined output from substep 354 may be stored in a database in association with the user's ID and / or the task's (and / or the location's) ID. In some embodiments, such output may be provided for further training of the applied AI / ML model (e.g., substep 364 shown in FIG. 3B).

[0091] In sub-step 356, engine 200 may execute a kinematics algorithm. In some embodiments, the kinematics algorithm may include, but is not limited to, serial and / or parallel manipulator analysis related to the captured data. In some embodiments, the input to sub-step 356 may be, but is not limited to, the captured data and / or the output from sub-step 352.

[0092] According to some embodiments, sub-step 356 may output data regarding the user's movements. This data may be related to a particular movement of a particular body part (e.g., but is not limited to, what movement the arm made, what movement the fingers made, what the user's posture or stance was, the angle and / or speed of the user's movement (and / or at what angle and speed the particular body part, e.g., the user's arm, moved), the start position of the user / body part, the end position of the user / body part, etc., or some combination thereof).

[0093] In substep 358, the determined output from substep 356 may be stored in a database in association with the user's ID and / or the task's (and / or the location's) ID. In some embodiments, such output may be provided for further training of the applied AI / ML model (e.g., substep 364 shown in FIG. 3B).

[0094] In sub-step 360, engine 200 may execute a graphical information system (GIS) algorithm (or model). According to some embodiments, input to the GIS algorithm may be the captured data, the output from sub-step 352, and / or the output from sub-step 356, or some combination thereof.

[0095] According to some embodiments, execution of the GIS algorithm by engine 200 enables mapping of the location and / or proximate area around the user at said location (e.g., 3D mapping of predetermined positions around the user and the user's movements, e.g., 3D mapping of a 2 meter space around the user in the x, y, and z planes). In some embodiments, the mapping further enables tracking of the user's movements in 3D space as represented in the captured data and analyzed by sub-steps 353 and 357. In some embodiments, the output of the GIS algorithm includes 2D or 3D representations of real-world elements as graphical elements. The graphical elements may be a grid-space (e.g., raster) model or a line-based (e.g., vector) model.

[0096] In substep 362, the determined output from substep 360 may be stored in a database in association with the user's ID and / or the task's (and / or the location's) ID. In some embodiments, such output may be provided for further training of the applied AI / ML model (e.g., substep 364 shown in FIG. 3B).

[0097] Thus, step 314 can identify the user's movements (from general movements specific to the user to movements of specific limbs / body parts) in the 3D representation space of the location and generate a mapping. For example, step 314 can determine a 3D mapping of the displacements of the user's trunk joints from initial positions to new positions. The mapping can indicate displacement velocity, displacement acceleration, displacement angular velocity, overall time taken to perform the displacement (or task), etc., or some combination thereof. Thus, according to some embodiments, the mapping can provide user kinematics, which can include information about the user's actions, information about the user's identity, information about the user's demographic attributes, information about the user's biometrics, etc., or some combination thereof.

[0098] Returning to FIG. 3A , processing continues at step 316, where engine 200 may determine user performance based on the analysis from step 314. According to some embodiments, engine 200 may determine a user performance value, performance metric, or performance measurement based on the captured user movements (from step 312), including, but not limited to, current fatigue, intensity, energy / activity level, mood, type of activity, body position, instantaneous speed, angle of movement, movement trajectory, immobility, etc., or some combination thereof. According to some embodiments, user performance may correspond to, but is not limited to, user activity, compliance with certain (or applicable) laws and regulations, safety / security standards, etc. For example, as described herein, performance may indicate that the user is performing a job / task at a level or in a manner that violates certain employee regulations (e.g., not wearing a helmet in a certain zone area of ​​the location).

[0099] According to some embodiments, step 316 may include utilizing the output from step 314 as input to an AI / ML model to determine the user's performance, including, but not limited to, logistic regression, linear regression, stepwise regression, multivariate adaptive regression spline (MARS), least squares regression (LSR), neural networks, random forests, etc. Thus, based on such analysis, step 316 enables engine 200 to determine a performance metric for the user. For example, a performance value may be determined for the user according to a scale (e.g., 1-10, with 10 being the best performance). In some embodiments, the scale may be adjusted and / or dynamically modified (e.g., increasing the value from 1-20) depending on the difficulty of the task or the type of task. In some embodiments, further description of the determined performance information is provided below in connection with steps 402-404 of process 400 of FIG. 4.

[0100] In step 318, engine 200 may store data related to the identified performance in storage. This data may be stored in association with a user ID, a task ID, and / or a location ID, as described above.

[0101] In step 320, engine 200 may utilize the identified performance information from step 316 to further train the AI / ML algorithms applied / executed by engine 200.

[0102] At step 322, the engine 200 may generate an output based on the determined performance. This output may be output to a user or a set of users, as described in more detail below with reference to at least FIG. 4. For example, an administrator may receive the generated output as an electronic message that includes content corresponding to the user's determined performance. In another non-limiting example, as described below, the user may receive an alert at the UE 102, which may inform the user of the user's current fitness state.

[0103] According to some embodiments, engine 200 may perform an operation to determine whether the user has other tasks to perform (e.g., from the user's schedule). This is indicated in Figure 3 by the dashed line from step 322 to step 308; if there are other tasks to perform and the user is authorized / assigned to perform them, engine 200 may continue processing process 300 recursively.

[0104] Referring to FIG. 4, a process 400 is provided that details a non-limiting exemplary embodiment for automatically communicating alerts related to a user's identified performance (e.g., via process 300 described above).

[0105] According to some embodiments, process 400 can occur in real time (or substantially real time), where data related to a user's performance of a task is captured, and once performance identification is made (e.g., via step 316), process 400 can be executed to provide real-time feedback to the local or associated user and / or other users. In some embodiments, process 400 can operate by retrieving stored performance data about a user and performing the analyses described herein (e.g., for performance review and / or to further train algorithms implemented by engine 200).

[0106] According to some embodiments, step 402 of process 400 may be performed by the analysis module 204 of the operations engine 200. Step 404 may be performed by a specific module. Steps 406-414 may be performed by the output module 208.

[0107] According to some embodiments, process 400 begins at step 402, in which engine 200 may analyze a user's determined performance for a particular task. In some embodiments, for example, the determined performance may correspond to the performance determined by process 300 described above. In some embodiments, analysis of the determined performance may be performed by an AI / ML model for determining the user's performance, in a manner similar to that described above in connection with step 316, including, but not limited to, logistic regression, linear regression, stepwise regression, MARS, LSR, neural networks, random forests, etc.

[0108] In some embodiments, the analysis performed in step 402 may be related to performance thresholds, which may be, but are not limited to, a user, a type of user, a level of user, a user's experience, a type of task, a task length, a task difficulty, a law / regulation related to the task, an industry and / or work site, an environmental condition (e.g., the temperature of the location, the climate of the location, etc.), a time of day, a year and a month, etc., or some combination thereof.

[0109] In step 404, a value associated with performance may be identified (determined) by engine 200 based on the analysis of step 402. This may be performed in a similar manner as described above. For example, the performance value may be 5 / 10 and the performance threshold for the task may be 6 / 10, which may indicate that the user is not performing up to industry standards / efficiency / safety.

[0110] In step 406, engine 200 may generate an alert based on the identified value. In some embodiments, an alert may be generated if the identified performance value (or indicator) is at or below a performance threshold. In some embodiments, the performance value, and in some embodiments, the range up to the performance threshold, may be used as a basis for engine 200 to determine the type of alert and / or the type of user to send the alert to.

[0111] For example, if an alert indicates that a user is operating a machine at an unsafe level for a task, a supervisor can be notified by an SMS message. In another example, the same user can also or instead receive a haptic message sent to their UE, sensor, or peripheral device, warning them to stop working. Similarly, an audio alert may be sent instructing the user to "Stop." In some embodiments, engine 200 may utilize natural language processing (NLP) algorithms to equate levels of performance to audible messages. In some embodiments, a collection of message types, including audio, video, text, and / or images, is stored in a database and may be retrieved by engine 200 as part of the message generation process.

[0112] Thus, in some embodiments, once an alert is generated, engine 200 may send the alert to the user, as shown in step 408. In some embodiments, as described above, the alert may be any type of electronic message and may include any type of renderable digital content. In some embodiments, the alert may be sent by an application running on the user's device that corresponds to the functionality of engine 200.

[0113] In some embodiments, for example, the alert of step 408 may notify the user of another task or next assigned task having a difficulty level that better matches the user's current performance level. Such a determination may be made by engine 200 by matching the identified performance level, at least to a threshold level, with levels associated with other identified tasks scheduled for the location.

[0114] In some embodiments, for example, the alert may determine a dangerous condition (e.g., a fire) associated with the location where the user is currently performing a task. Thus, the alert may instruct the user to leave the location and report to a designated safe location. In this manner, the alert may, for example, reroute the user to a different location, a different task, or stop work entirely.

[0115] In some embodiments, for example, but not limited to, alerts may notify a user of a user's performance value (e.g., related to a performance threshold for a task the user is performing), a dangerous condition, an incorrect technique, and / or other undesirable behavior or location.

[0116] In some embodiments, once an alert is generated, engine 200 may transmit the alert to at least one other identified user associated with the location, as shown in step 410. In some embodiments, step 410 may include identifying such other user. In some embodiments, as described above, the alert may be any type of electronic message and may include any type of renderable digital content. In some embodiments, the alert may be transmitted by an application running on the identified other user's device that corresponds to the functionality of engine 200. In some embodiments, the alert may be broadcast by a speaker at the location so that all users can audibly receive it. This may occur if a user's performance corresponds to a dangerous activity level or task. In some embodiments, the alert may also be transmitted to a third party (e.g., a first responder, such as a fire department) if performance information may indicate injury to a worker user.

[0117] According to some embodiments, the alerts communicated by steps 408 and 410 may be one-way, two-way, or multi-way communications using text, voice, voice recognition, etc., or some combination thereof.

[0118] At step 412, information / data related to the communicated alert may be stored in a database. In some embodiments, as described above, such information may correspond to a user ID, a task ID, and / or a location ID. In some embodiments, the stored information may indicate performance values ​​and related information for the generated alert (e.g., type of alert, type of content, date and time sent, destination, etc.).

[0119] Then, in step 414, the stored information (or at least the information analyzed, identified, and / or generated during processing of process 400) can be utilized to further train the AI / ML algorithms executed by engine 200. This may enable more refined, efficient, and accurate identification of work user performance levels and / or safety backstops, as described herein.

[0120] 5, process 500 provides a non-limiting exemplary embodiment that utilizes stored user activity data and / or identified user performance (from processes 300-400 above) to automate the performance of a task by a computer-operated machine (or asset)—referred to as a robot for purposes of illustration only. As described herein, the activity data can provide the kinematics of the operating user and can be transferred to the robotic operator, thereby enabling the robotic operator to automatically operate and perform a specific task.

[0121] As a non-limiting example, a robot (or robotic worker, used interchangeably), for purposes of explanation, may be any type of real-world or controlled asset in a location that can perform a real-world or digital task. In some embodiments, a robot may be fully computer-operated or at least partially computer-operated. In some embodiments, a robot may be and / or integrated with, for example, an assistive mechanism, an external machine, and / or an exoskeleton.

[0122] According to some embodiments, steps 502 and 506 of process 500 may be performed by the identification module 202 of the operations engine 200. Steps 504, 508, and 510 may be performed by the identification module 206, and steps 512-514 may be performed by the output module 208.

[0123] According to some embodiments, process 500 begins at step 502, in which a task is identified by engine 200. According to some embodiments, the identification of the task may be performed in a manner similar to that described above in connection with at least step 308 of process 300.

[0124] In step 502, engine 200 may analyze the task and identify types of robots capable of and / or configured to perform the task. In some embodiments, such analysis may identify subparts, subroutines, and / or specific action sequences for the task. According to some embodiments, engine 200 may utilize any type of AI / ML algorithm or technique, now or hereafter known, to analyze the data file associated with the task and identify specific actions for the task (e.g., the neural networks described above).

[0125] In some embodiments, specific actions and / or subparts of a task may be compiled using stored modeling data of user actions for the task, as described above in connection with at least step 314. In some embodiments, engine 200 may analyze identified data, mapping data, or modeling data of users who previously performed the task above a predetermined threshold and identify steps of the task accordingly.

[0126] In step 506, a robot for performing (or using for) a task may be identified based on the type of robot (and in some embodiments, the type of task).

[0127] In step 508, modeling data for the execution of the task may be identified by engine 200. According to some embodiments, as described above, the modeling data may correspond to the 3D mapping identified by step 314. In some embodiments, specific performance values ​​or desired / requested types or values ​​of operating user kinematics may be utilized as search criteria to identify the modeling data (e.g., stored modeling data identified in at least 8 / 10 of the databases).

[0128] In step 510, engine 200 can compile a set of instructions for the robot to execute. The set of instructions can include specific actions for the robot to sequentially execute a task to completion accurately (and in some embodiments efficiently, e.g., within a certain period of time). In some embodiments, engine 200 can analyze the modeling data, extract information related to the specific steps indicated therein, and generate an executable, machine-readable data structure or file that includes the processing steps of the task in the order for accurately executing the task.

[0129] In some embodiments, such compiled instructions may be stored in storage (e.g., a database) in association with the ID of the task, location, robot, and / or user from which the modeling data originated, as described above.

[0130] In step 512, the engine 200 can cause the robot to communicate and / or load the instructions to the robot. Execution of the provided instructions can cause the robot to perform the instructions, as shown in step 514. Thus, the robot can automatically perform tasks according to the provided instructions.

[0131] In some embodiments, the robot may be configured with mounted and / or embedded / attachable sensors at specific points on or around the robot, and specific commands cause the robot to be manipulated by such attached sensors.

[0132] 8 is a schematic diagram of a client device illustrating an example embodiment of a client device usable in the present disclosure. Client device 800 may include more or fewer components than those shown in FIG. 8. However, the components shown are sufficient to disclose an example embodiment for implementing the present disclosure. Client device 800 may represent, for example, at least UE 102 described above in connection with FIG. 1.

[0133] As shown in the figure, in some embodiments, client device 800 includes a processing unit (CPU) 822 in communication with mass memory 830 via a bus 824. Client device 800 also includes a power supply 826, one or more network interfaces 850, an audio interface 852, a display 854, a keypad 856, an illuminator 858, an input / output interface 860, a tactile interface 862, an optional global positioning system (GPS) receiver 864, and a camera or other optical, thermal, or electromagnetic sensor 866. Device 800 may include one camera / sensor 866 or multiple cameras / sensors 866, as will be understood by those skilled in the art. Power supply 826 provides power to client device 800.

[0134] The client device 800 may optionally communicate with a base station (not shown) or directly with another computing device. In some embodiments, the network interface 850 may be known as a transceiver, a transceiver, or a network interface card (NIC).

[0135] Audio interface 852, in some embodiments, is configured to generate and receive audio signals, such as a human voice. Display 854 may be a liquid crystal display (LCD), a gas plasma display, a light emitting diode (LED), or other type of display used in computing devices. Display 854 may also include a touch-sensitive screen configured to receive input from an object, such as a stylus or the finger of a human hand.

[0136] Keypad 856 may include any input device configured to receive input from a user. Illuminator 858 may provide status indication and / or light.

[0137] Client device 800 also includes input / output interface 860 for external communication, which in some embodiments may utilize one or more communication technologies such as USB, infrared, Bluetooth, etc. Haptic interface 862 is configured to provide haptic feedback to a user of client device 800.

[0138] The optional GPS transceiver 864 can determine the physical coordinates of the client device 800 on the surface of the Earth, typically outputting a location as latitude and longitude values. The GPS transceiver 864 can also employ other geolocation mechanisms, including but not limited to triangulation, assisted GPS (AGPS), E-OTD, CI, SAI, ETA, BSS, etc., to further determine the physical location of the client device 800 on the surface of the Earth. However, in one embodiment, the client device, through other components, can provide other information that can be used to determine the device's physical location, including a MAC address, an Internet Protocol (IP) address, etc.

[0139] Mass memory 830 includes RAM 832, ROM 834, and other storage means. Mass memory 830 represents another example of a computer storage medium for storage of information such as computer-readable instructions, data structures, program modules, or other data. Mass memory 830 stores a basic input / output system ("BIOS") for controlling low-level operation of client device 800. Mass memory 830 also stores an operating system (OS) 841 for controlling the operation of client device 800.

[0140] Memory 830 further includes one or more data stores that may be utilized by client device 800 to store, among other things, applications 842 and / or other information and data. For example, a data store may be used to store information describing various capabilities of client device 800. This information may be provided to another device based on any of a variety of events, including being sent as part of a header during a communication (e.g., an index file for an HLS stream), being sent upon request, etc. At least a portion of the capability information may also be stored on a disk drive or other storage medium (not shown) within client device 800.

[0141] Applications 842 may include computer-executable instructions that, when executed by client device 800, send, receive, and / or otherwise process audio, video, images, and enable communication with a server and / or other users of other client devices. Applications 842 may further include clients configured to send, receive, and / or otherwise process games, goods / services, and / or other types of data, messages, and content hosted and offered by platforms associated with engine 200 and its affiliates.

[0142] As used herein, the terms "computer engine" and "engine" identify at least one software component and / or a combination of at least one software component and at least one hardware component that is designed / programmed / configured to manage / control other software components and / or hardware components (libraries, software development kits (SDKs), objects, etc.).

[0143] Examples of hardware elements may include a processor, a microprocessor, a circuit, a circuit element (e.g., a transistor, a resistor, a capacitor, an inductor, etc.), an integrated circuit, an application specific integrated circuit (ASIC), a programmable logic device (PLD), a digital signal processor (DSP), a field programmable gate array (FPGA), a logic gate, a register, a semiconductor device, a chip, a microchip, a chipset, etc. In some embodiments, one or more processors may be implemented as a complex instruction set computer (CICS) processor or a reduced instruction set computer (RISC) processor, an x86 instruction set compatible processor, a multi-core processor, or other microprocessor or central processing unit (CPU). In various implementations, one or more processors may be a dual-core processor, a dual-core mobile processor, etc.

[0144] As used herein, computer-related system, computer system, and system include any combination of hardware and software. Examples of software may include software components, programs, applications, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, application program interfaces (APIs), instruction sets, computer code, computer code segments, words, values, symbols, or any combination thereof. The decision whether an embodiment is implemented using hardware and / or software elements may depend on a variety of factors, such as desired computational speed, power levels, thermal budget, processing cycle budget, input data rate, output data rate, memory sources, data bus speeds, and other design or performance constraints.

[0145] For purposes of this disclosure, a module is a software, hardware, or firmware (or combination thereof) system, process, function, or component thereof that performs or facilitates the processes, features, and / or functions described herein (with or without human interaction or augmentation). A module may include sub-modules. The software components of a module may be stored on a computer-readable medium for execution by a processor. A module may be integrated into one or more servers or loaded and executed by one or more servers. One or more modules may be grouped into an engine or application.

[0146] One or more aspects of at least one embodiment may be implemented by representative instructions stored on a machine-readable medium that represent various logic within a processor, which, when read by a machine, cause the machine to manufacture logic for performing the techniques described herein. Such representations, known as "IP cores," may be stored on tangible machine-readable media and supplied to various customers or manufacturing facilities for loading into manufacturing machines that produce the logic or processors. Of course, the various embodiments described herein may be implemented using any suitable hardware and / or computing software language (e.g., C++, Objective-C, Swift, Java, JavaScript, Python, Perl, QT, etc.).

[0147] For example, exemplary software specially programmed in accordance with one or more principles of the present disclosure may be downloadable from a network, e.g., a website, as a standalone product or as an add-in package for installation into an existing software application. For example, exemplary software specially programmed in accordance with one or more principles of the present disclosure may also be available as a client-server software application or as a web-enabled software application. For example, exemplary software specially programmed in accordance with one or more principles of the present disclosure may also be embodied as a software package installed on a hardware device.

[0148] For purposes of this disclosure, the terms "user," "subscriber," "consumer," or "customer" should be understood to refer to a user of the applications described herein and / or a consumer of data provided by a data provider. By way of example, and not limitation, the terms "user" or "subscriber" can refer to a person receiving data provided from a data provider or service provider over the Internet in a browser session, or to an automated software application that receives and stores or processes the data. Those skilled in the art will appreciate that the methods and systems of the present disclosure can be implemented in a variety of ways. Accordingly, the methods and systems of the present disclosure are not limited by the exemplary embodiments and examples described above. In other words, functional elements performed by single or multiple components, and individual functions may be distributed across multiple software applications at the client level, server level, or both, in various combinations of hardware and software or firmware. In this regard, any number of features of the various embodiments described herein may be combined in a single or multiple embodiments, and alternative embodiments having fewer or more than all of the features described herein are possible.

[0149] Additionally, functionality may be distributed, in whole or in part, across multiple components in ways now known or that will become known in the future. Accordingly, many different combinations of software, hardware, and firmware are possible to achieve the functions, features, interfaces, and configurations described herein. Furthermore, the scope of the present disclosure covers conventionally known methods for implementing the features, functions, and interfaces described herein, as well as changes and modifications made to the hardware, software, or firmware components described herein, both now and in the future, as understood by those skilled in the art.

[0150] Furthermore, the method embodiments presented and described as flowcharts in this disclosure are provided as examples to provide a more complete understanding of the technology. The disclosed methods are not limited to the operations and logical flow presented herein. Alternative embodiments are contemplated in which the order of various operations is changed, or sub-operations described as part of a larger operation are performed independently.

[0151] While various embodiments have been described for purposes of this disclosure, such embodiments should not be construed as limiting the teachings of this disclosure to those embodiments. Various changes and modifications can be made to the elements and operations described above to achieve results within the scope of the systems and processes described in this disclosure.

Claims

1. Identifying, by a device, a task to be performed by a user at a location; identifying, by the device, a monitoring device at the location; monitoring, by the device via the monitoring device, the user's activity related to the performance of the task; capturing, by the device, data relating to the user's activities associated with performing the task; analyzing the data captured by the device; and determining, by the device, performance information of the user; generating, by the device, an alert based on the performance information; A method comprising:

2. analyzing the performance information to identify a performance value for the task by the user; determining a type of the alert based on the analysis of the performance information, the determination further comprising determining an identity of a destination to send the alert to; The method of claim 1 further comprising:

3. The method of claim 2 , wherein the alert is sent to a device associated with the user.

4. The method of claim 2 , wherein the alert is sent to at least one device of another user.

5. Analyzing the captured data includes: running a classifier to identify a type of activity of the user; executing a kinematics algorithm to identify motion information associated with the activity; executing a geographic information system (GIS) algorithm based on at least one of the captured data; generating a three-dimensional (3D) mapping of the user's activity relative to the location based on execution of the classifier, the kinematics algorithm, and the GIS algorithm; storing said 3D mapping in a database; The method of claim 1 further comprising:

6. analyzing the task and identifying a sequential procedure for performing the task based on the analysis of the task; Identifying a type of robot based on an analysis of the task; analyzing the 3D mapping; and identifying an instruction set for the robot to perform the task based on at least one of the analysis of the 3D mapping, an automatic learning algorithm for automatically analyzing and learning the task, and the sequential procedure for the task; compiling said instruction set into a machine-readable data structure; communicating the data structure to the robot via a network, whereby the robot automatically executes the sequential procedures according to performance values ​​provided by the 3D mapping; The method of claim 5 further comprising:

7. The method of claim 1 , wherein the monitoring device includes at least one of a sensor associated with the location, a sensor associated with the user, a camera associated with the location, and a camera associated with the user.

8. The method of claim 1 , wherein the task comprises at least one of a real-world manipulation and a digital manipulation.

9. at least one processor, the at least one processor Identifying a task to be performed by a user at a location; identifying a monitoring device at the location; monitoring, via the monitoring device, the user's activity related to the performance of the task; capturing data relating to the user's activities associated with performing the task; Analyzing the captured data; and identifying performance information for the user; generating an alert based on the performance information; A device that is configured to:

10. The at least one processor analyzing the performance information to identify a performance value for the task by the user; determining a type of the alert based on the analysis of the performance information, the determination further comprising determining an identity of a destination to send the alert to; The device of claim 9 , further configured to:

11. The device of claim 10 , wherein the alert is sent to a device associated with the user.

12. The device of claim 10 , wherein the alert is sent to at least one device of another user.

13. The at least one processor running a classifier to identify a type of activity of the user; executing a kinematics algorithm to identify motion information associated with the activity; executing a geographic information system (GIS) algorithm based on at least one of the captured data; generating a three-dimensional (3D) mapping of the user's activity relative to the location based on execution of the classifier, the kinematics algorithm, and the GIS algorithm; storing said 3D mapping in a database; The device of claim 9 , further configured to:

14. The at least one processor analyzing the task and identifying a sequential procedure for performing the task based on the analysis of the task; Identifying a type of robot based on an analysis of the task; analyzing the 3D mapping; and identifying an instruction set for the robot to perform the task based on at least one of the analysis of the 3D mapping, an automatic learning algorithm for automatically analyzing and learning the task, and the sequential procedure for the task; compiling said instruction set into a machine-readable data structure; communicating the data structure to the robot via a network, whereby the robot automatically executes the sequential procedures according to performance values ​​provided by the 3D mapping; The device of claim 13 further configured to:

15. A non-transitory computer-readable storage medium encoded with computer-executable instructions that, when executed by a device, perform a method, the method comprising: Identifying a task to be performed by a user at a location with the device; identifying, by the device, a monitoring device at the location; monitoring, by the device via the monitoring device, the user's activity related to the performance of the task; capturing, by the device, data relating to the user's activities associated with performing the task; analyzing the data captured by the device; and determining, by the device, performance information of the user; generating, by the device, an alert based on the performance information; 1. A non-transitory computer-readable storage medium comprising:

16. The method comprises: analyzing the performance information to identify a performance value for the task by the user; determining a type of the alert based on analysis of the performance information, the determining further comprising determining an identity of a destination to send the alert to; 16. The non-transitory computer-readable storage medium of claim 15, further comprising:

17. The non-transitory computer-readable storage medium of claim 16 , wherein the alert is transmitted to a device associated with the user.

18. The non-transitory computer-readable storage medium of claim 16 , wherein the alert is transmitted to at least one device of another user.

19. Analyzing the captured data includes: running a classifier to identify a type of activity of the user; executing a kinematics algorithm to identify motion information associated with the activity; executing a geographic information system (GIS) algorithm based on at least one of the captured data; generating a three-dimensional (3D) mapping of the user's activity relative to the location based on execution of the classifier, the kinematics algorithm, and the GIS algorithm; storing said 3D mapping in a database; 16. The non-transitory computer-readable storage medium of claim 15, further comprising:

20. The method comprises: analyzing the task and identifying a sequential procedure for performing the task based on the analysis of the task; Identifying a type of robot based on an analysis of the task; analyzing the 3D mapping; and identifying an instruction set for the robot to perform the task based on at least one of the analysis of the 3D mapping, an automatic learning algorithm for automatically analyzing and learning the task, and the sequential procedure for the task; compiling said instruction set into a machine-readable data structure; communicating the data structure to the robot via a network, whereby the robot automatically executes the successive steps according to performance values ​​provided by the 3D mapping; 20. The non-transitory computer-readable storage medium of claim 19, further comprising:

Images