Method for offline operation of intelligent multi-function robots

Abstract

A method of using an intelligent multi-functional robot having and using an internal artificial intelligence and functioning autonomously when not connected to an external artificial intelligence, the method comprising the steps of: downloading functions and data required for the robot to perform a specific task from a server when the robot is online during a pre-initialization process or initialization process prior to the autonomous operation of the robot; during the autonomous operation of the robot, the robot does not use the external server or the external artificial intelligence; natively launching software on the robot itself, where the hardware for processing the software is on the robot itself and the data stored on the robot includes only data required to perform pre-set functions; limiting the on-board hardware processing power to only include power required to perform the pre-set functions; and adapting the robot to move autonomously within a defined facility.

Description

[Technical field]

[0001] The present invention relates generally to the field of robotics, and more particularly to robots that operate offline.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS The applicant claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 304621, filed January 30, 2022, U.S. Provisional Patent Application No. 63 / 310120, filed February 15, 2022, U.S. Provisional Patent Application No. 63 / 411156, filed September 29, 2022, and U.S. Provisional Patent Application No. 63 / 478170, filed January 2, 2023, all of which are incorporated by reference herein. [Background technology]

[0003] To fully appreciate the present invention, it is useful to understand the terminology and basic elements that facilitate robotic operation.

[0004] Modern robots are generally divided into two categories: the first category are non-intelligent robots with very specific functions, usually found on an assembly line, and there can also be intelligent robots with general functions, sometimes mobile robots, that can use intelligence gained both from artificial intelligence (such as cloud artificial intelligence) or from processing behavior algorithms such as face and object recognition, audio recognition, etc.

[0005] Intelligent robots are becoming increasingly part of everyday life: we use such robots for medical support, as restaurant waiters, takeaway deliveries, etc.

[0006] These robots are usually controlled by a server that stores their data and functions. The processing power required to operate such a robot usually requires large and expensive processors. To be controlled by the server / to access data, resources and services, the robot needs to have some form of communication with the server, such as Bluetooth, Internet, etc.

[0007] An intelligent online robot typically has access to a very large amount of data. This data can be used to ensure that it can pull operational data or any other data as needed, ensuring that the data is always up to date. Without this capability, it becomes very difficult (or even impossible) for a robot with complex functionality to operate offline. It can be appreciated that a robot may not necessarily communicate with its server during its use. For example, a power outage may cause the Wi-Fi connection to be cut off, causing the robot to go offline. Alternatively, the robot may be out of range of communication with its server. Thus, for a robot to operate autonomously offline, it would therefore need all functionality provided by a server locally on the robot, and large and expensive processor and storage capabilities are required. This full functionality may also include processing power to run large amounts of operational algorithms to support the intelligence it needs to decide to operate independently.

[0008] Furthermore, it is neither practical nor cost effective to have an entire server processing unit on each robot to have full functionality available when offline.

[0009] To effect the invention herein, the processors are generally larger and more expensive than the average processors of similar robots that operate using a server, since the server can offload much of the heavy processing that would normally fall on the robot's processor. The robots disclosed herein operate offline and cannot do so. Thus, the processors of the robots of the present invention are much smaller than the average server processor, since it is not possible to put an entire server processing unit on each robot for any reasonable size or cost-effective considerations. Thus, the processors are larger than the average online robot and smaller than the average server processor.

[0010] Thus, there is a need in the industry and field for a device or system that can efficiently and effectively enable a robot to operate autonomously offline without utilizing processors, data, information, and ASI available on an external server. [Prior art documents] [Patent documents]

[0011] [Patent Document 1] U.S. Provisional Patent Application No. 63 / 304621 [Patent Document 2] U.S. Provisional Patent Application No. 63 / 310120 [Patent Document 3] U.S. Provisional Patent Application No. 63 / 411,156 [Patent Document 4] U.S. Provisional Patent Application No. 63 / 478170 Summary of the Invention [Problem to be solved by the invention]

[0012] To achieve these and other objectives, the devices herein enable robots to operate autonomously offline without utilizing processors, data, information, and ASI available on external servers. [Means for solving the problem]

[0013] Therefore, to achieve these and other objectives, the invention disclosed herein is a method of using an intelligent multi-functional robot having and using an internal artificial intelligence to function autonomously when not connected to an external artificial intelligence, the method comprising the steps of: downloading functions and data required for the robot to perform a specific task from a server when the robot is online during a pre-initialization process or initialization process prior to the autonomous operation of the robot; during the autonomous operation of the robot, the robot does not use an external server or external artificial intelligence; launching software natively on the robot itself, where the hardware for processing the software is on the robot itself and the data stored on the robot includes only the data required to perform pre-set functions; limiting the on-board hardware processing power to include only the power required to perform the pre-set functions; and adapting the robot to move autonomously within a defined facility.

[0014] In a preferred embodiment, the robot is initialized to perform only the task or tasks for which it is designed. Preferably, the pre-configured functionality is selected from the group consisting of artificial intelligence (AI), communication with the user, navigation, tracking the user, recognition of individual users through voice and audio recognition, and map generation.

[0015] According to another aspect of the present invention, the present invention provides an intelligent multi-functional robot capable of autonomous movement and including multiple functional modules, where the brain, router, and functional features of each module operate within their own isolated virtual environment container. Further, each module runs standalone scripts packaged and deployed within their own isolated virtual environment container. The functional features include navigation, image processing, audio processing, or video processing.

[0016] According to another aspect of the present invention, the present invention provides a server for managing a plurality of robots, each robot performing a set of designated tasks, the server comprising, for each of the plurality of robots, a robot management table listing each robot's tasks and associated scripts according to a unique robot ID, an operation database storing operation data required to invoke the scripts, a preprocessor for invoking at least one algorithm to obtain and update the operation data, and an initializer for initializing a particular robot according to the unique robot ID with the associated script when the server is in network communication with that particular robot.

[0017] Another aspect of the invention constitutes a method for programming an intelligent multitasking robot that uses internal intelligence to function autonomously when not connected to an external server, the method comprising the steps of: during a pre-initialization process or initialization process prior to autonomous operation of the robot, dividing processing into low resource processing tasks for the robot and high resource processing tasks for the external server; initializing the robot with the low resource processing tasks, which provide the internal intelligence; and during autonomous operation of the robot, the robot uses the internal intelligence and does not utilize a server for external intelligence, the data for the internal intelligence stored in the robot including only data necessary to perform pre-configured functions.

[0018] Preferably, the on-board hardware processing power includes only the power required to provide the internal intelligence. Furthermore, the robot is adapted to move autonomously within a defined facility. The low-resource processing tasks include functional features, each of which operates within its own isolated virtual environment container. One of the low-resource processing tasks launches a standalone script to operate the functional feature. The functional features include navigation, image processing, audio processing, or speech processing.

[0019] According to another aspect of the present invention, the present invention provides a robot for performing a set of predetermined tasks, each task having a plurality of feature elements, the robot comprising: a scripter that downloads a script and a plurality of virtual containers associated with a unique ID number of the robot from a server communicating with the robot according to a unique ID number of the robot, one of the plurality of virtual containers being a router that communicates with the server and downloads a subset of the operational data and supports the plurality of virtual containers, a second virtual container of the plurality of virtual containers being a controller that controls the robot according to the script, each of the remaining virtual containers performing one of the plurality of feature elements; and a robot database that stores the subset of the operational data, the scripter operating when the robot is online with the server and the controller operating when the robot is offline from the server.

[0020] The individual processing power required for the robot to function, and the memory required for data storage, can be practically reduced by using an initialization process that downloads the functions and data required for the robot to perform a specific task when the robot is online with a dedicated server. The robot can be initialized to perform only the task or tasks for which it is designed. Thus, only the data and instructions required for the task are downloaded, requiring less processing power and therefore cheaper processors and memory units. Continuous operation between the robot and the server can allow two-way updates between the server and the robot when the robot is determined to be online. Thus, the robot can be designed from a data storage perspective and from a software side by redesigning the software to accommodate function optimization that can be launched with limited processing power.

[0021] This method of operation may also increase data security, reliability, and operability, especially in the field of care, especially when working with elderly people, who usually have problems with proper adoption and usage. Elderly people who lack technological knowledge may need stable and reliable usage capabilities that encourage building a trusting relationship with the robot.

[0022] The subject matter which is regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of this specification. The invention, however, together with its objects, features, and advantages, both as to organization and method of operation, may be better understood by reference to the following detailed description when read in conjunction with the accompanying drawings. [Brief description of the drawings]

[0023] [Figure 1] The basic operating software architecture is shown. [Diagram 2] The server side structure is shown below. [Diagram 3] 2 shows a flow chart for initialization. [Figure 4]1 shows a flowchart of updating data from a server. [Diagram 5] 13 is a flowchart showing updating of data from a robot. [Figure 6] FIG. 1 is a schematic diagram of a system for initializing a robot to function offline, constructed and operative in accordance with a preferred embodiment of the present invention; [Figure 7A] FIG. 7 is a schematic diagram of elements of a processor of the robot of FIG. 6 at initialization, constructed and operative in accordance with a preferred embodiment of the present invention. [Figure 7B] FIG. 7 is a schematic diagram of elements of a processor of the robot of FIG. 6 at run time, constructed and operative in accordance with a preferred embodiment of the present invention. [Figure 8] FIG. 7 is a schematic diagram of elements of the pre-processor of FIG. 6, constructed and operative in accordance with a preferred embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] It will be appreciated that for simplicity of illustration and clarity, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated in common among several figures to indicate like or analogous elements.

[0025] In the following detailed description, some specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention.

[0026] In a basic embodiment, the present invention constitutes a method of using an intelligent multifunctional robot having and using an internal artificial intelligence and functioning autonomously when not connected to an external artificial intelligence, the method comprising the steps of: downloading functions and data required for the robot to perform a specific task from a server when the robot is online during a pre-initialization process or initialization process prior to the autonomous operation of the robot; during the autonomous operation of the robot, the robot does not use an external server or external artificial intelligence; launching software natively on the robot itself, where the hardware for processing the software is on the robot itself and the data stored on the robot includes only the data required to perform pre-set functions; limiting the on-board hardware processing power to include only the power required to perform the pre-set functions; and adapting the robot to move autonomously within a defined facility.

[0027] In a preferred embodiment, the robot is initialized to perform only the task or tasks for which it is designed. Preferably, the pre-configured functionality is selected from the group consisting of artificial intelligence (AI), communication with the user, navigation, tracking the user, recognition of individual users through voice and audio recognition, and map generation.

[0028] According to another aspect of the present invention, the present invention provides an intelligent multi-functional robot capable of autonomous movement and including multiple functional modules, where the brain, router, and functional features of each module operate within their own isolated virtual environment container. Further, each module runs standalone scripts packaged and deployed within their own isolated virtual environment container. The functional features include navigation, image processing, audio processing, or video processing.

[0029] According to another aspect of the present invention, the present invention provides a server for managing a plurality of robots, each robot performing a set of designated tasks, the server comprising, for each of the plurality of robots, a robot management table listing each robot's tasks and associated scripts according to a unique robot ID, an operation database storing operation data required to invoke the scripts, a preprocessor for invoking at least one algorithm to obtain and update the operation data, and an initializer for initializing a particular robot according to the unique robot ID with the associated script when the server is in network communication with that particular robot.

[0030] Another aspect of the invention constitutes a method for programming an intelligent multitasking robot that uses internal intelligence to function autonomously when not connected to an external server, the method comprising the steps of: during a pre-initialization process or initialization process prior to autonomous operation of the robot, dividing processing into low resource processing tasks for the robot and high resource processing tasks for the external server; initializing the robot with the low resource processing tasks, which provide the internal intelligence; and during autonomous operation of the robot, the robot uses the internal intelligence and does not utilize a server for external intelligence, the data for the internal intelligence stored in the robot including only data necessary to perform pre-configured functions.

[0031] Preferably, the on-board hardware processing power includes only the power required to provide the internal intelligence. Furthermore, the robot is adapted to move autonomously within a defined facility. The low-resource processing tasks include functional features, each of which operates within its own isolated virtual environment container. One of the low-resource processing tasks launches a standalone script to operate the functional feature. The functional features include navigation, image processing, audio processing, or video processing.

[0032] According to another aspect of the present invention, the present invention provides a robot for performing a set of predetermined tasks, each task having a plurality of feature elements, the robot comprising: a scripter that downloads a script and a plurality of virtual containers associated with a unique ID number of the robot from a server communicating with the robot according to a unique ID number of the robot, one of the plurality of virtual containers being a router that communicates with the server and downloads a subset of the operational data and supports the plurality of virtual containers, a second virtual container of the plurality of virtual containers being a controller that controls the robot according to the script, each of the remaining virtual containers performing one of the plurality of feature elements; and a robot database that stores the subset of the operational data, the scripter operating when the robot is online with the server and the controller operating when the robot is offline from the server.

[0033] Figure 1 shows the basic operational software architecture.

[0034] Robot Processor(s): The processor manages instructions such as arithmetic, logical, input / output (I / O), and other basic instructions. The processors used herein in the preferred embodiment are, by way of non-limiting example, 1Xavier and 1Nano, although alternative configurations and components including one processor or many processors also work. Processors available from any company or custom made processors work.

[0035] To effect the invention herein, the processors are generally larger and more expensive than the average processors of similar robots that operate using a server, since the server can offload much of the heavy processing that would normally fall on the robot's processor. The robots disclosed herein operate offline and cannot do so. Thus, the processors of the robots of the present invention are much smaller than the average server processor, since it is not possible to put an entire server processing unit on each robot for any reasonable size or cost-effective considerations. Thus, the processors are larger than the average online robot and smaller than the average server processor.

[0036] Additionally, it offloads much of the heavy processing that would normally fall on the robot's processor. Thus, it does not use the server during operation, but uses the disclosed system of pre-initialization and initialization to use the server prior to autonomous operation. By using the pre-initialization and initialization process, it offloads processing at an earlier point in time that may provide benefits during operation.

[0037] Router: A router is a device that connects two or more packet-switched networks or packet-switched subnetworks. It serves two main functions: managing traffic between these networks by forwarding data packets to their target IP addresses, and allowing multiple devices to use the same Internet connection. The router needs to pull down data that the server prepares. Once the router has the data, it sends it to the brain, which does the rest of the processing. As explained below, the router is also responsible for sending IAmAlivePING signals. Any reference to a decision made by the router or an action taken by the router may refer to the router's own intelligence or decision-making capabilities, or alternatively, it may refer to the router taking actionable commands that originate from another docker, such as the brain or scripter.

[0038] Brain: The robot is made up of senses and abilities. These can be, but are not limited to: Language Recognition Natural Language Processing Vision Mobility Conversational ability The user interface on top of the robot screen / face.

[0039] The role of the brain is to manage these senses and capabilities, providing the "real world" that enables the robot to function. The brain satisfies the functional specification of any given robot.

[0040] As an example, in a clinic assistant robot, the role of the robot is to register the patient, take his vital signs, escort him to the examination room and then assist the doctor during the examination. The brain in such an example manages all of the components described above so that the robot can fulfill this functional specification. The brain creates a closed, defined real world in which the robot exists.

[0041] Local Database: A database is an organized collection of data that is stored and accessed electronically. A local database is simply a database that the robot can access on the robot itself. Examples include memory modules, hard disk drives (HDD), solid state drives (SSD), etc. Prior to initialization, any existing data or software on the robot is stored in a local database. During the robot's initialization phase, the robot connects to a server, downloads a subset of software and data from the server database, and stores that subset locally in a local database. The local database holds data along with information about features required for operation and can be directly accessed by the features. Any data or software that the robot processes at any point in time can be stored in the local database.

[0042] MQTT Broker (Local): As known in the art (see https: / / mqtt.org / ), an MQTT Broker allows MQTT clients to communicate, and in this case specifically refers to a local MQTT Broker. This can be a self-hosted MQTT Broker or a managed MQTT Broker.

[0043] Features: A feature set is a set of functions of a robot that need to realize a behavior. This can include: Navigation: currently running in ROS. The ability to move autonomously around an environment with intent and intelligence. Image processing: plays a role in object recognition, face recognition, OCR (optical character recognition), etc. Audio processing: Audio recognition, voice matching, NLP, STT, TTS, etc. Face-up: Responsible for UX / UI, displays, touch screens, responding to user input, etc.

[0044] The feature retrieves the data it needs to operate from a local database.,Examples of retrieved features include:,navigation;,map,place names;,image processing;,recognized people, objects;,audio processing;,recognized voices and words;,and FaceApp,screen.

[0045] Scripter: The scripter is responsible for managing the download and execution of the virtual container that contains the robot's features.

[0046] Each of the brains, routers, and features runs in a virtual environment container called a Docker. Instead of each module running as a standalone script on the robot's machine, those modules and all their dependencies (similar to external libraries) are packaged and deployed in their own isolated container. This allows the scripter to generate a single script that runs directly on the robot's machine and manages the deployment and version control of all the robot's modules. Using Docker commands, for example, it can shut down or power on modules as needed, or check if the current version of the virtual container is up to date and update it if needed. When an update is available from the server, the server can ping the router, which then sends a message to the scripter with the name of the Docker to update. The scripter then uses an API call to download the updated Docker from the server (see https: / / aws.amazon.com / what-is / api / for the API). Once the docker is downloaded, the scripter communicates with the brain, determines the best time, and restarts the updated docker.

[0047] Robot Server: A server is a computer or system that provides resources, data, services, or programs over a network to other computers, known as clients.

[0048] The server is also an administration portal that facility administrators can use to manage their data via a user interface (for the clinic example, via the portal the administrator can add users, user information, and user status, e.g., doctors and nurses. Doctors can then be assigned to rooms when their shifts begin).

[0049] Another example is a holistic facial recognition algorithm. All the heavy lifting is done by a server administration portal. Users upload their pictures. The server then processes those pictures and converts them into vectors that can be used for facial recognition by the robots. After this is done, these vectors are downloaded to each robot during their initialization or update state.

[0050] MQTT Broker (Server): In the preferred embodiment, we utilize the AWS IOT MQTT (Amazon Web Services, Internet of Things) broker, but as an alternative, other MQTT brokers may be used.

[0051] Backend: Simply put, the backend is where the technical processes happen. Backend development refers to the development of the server-side logic that runs websites and apps behind the scenes. It includes the database, the server, and all the code required to build the application. This is where pre-initialization and update processing can happen.

[0052] Front-end: Simply put, the front-end is where user interaction happens, where pre-initialization and input of updated information happens via the user interface.

[0053] Server Database: An organized collection of electronic data. The stored data may store, but is not limited to, faces, people, authority status, objects, maps, place names, recognition terms and phrases, customer data, or any other robot operational data necessary or helpful for the proper operation of the robot and its functions.

[0054] User Interface: Using a web page or phone app portal, or any suitable means of user interface, the user inputs any personalized data that the robot may need to operate appropriately, as they see fit.

[0055] Data Setup and Pre-Initialization: During the initialization phase, the data required / wanted / needed for the robot and the data required / wanted / needed from the customer is uploaded from the server to the robot. This data needs to be pre-installed on the server to prepare for the initialization process. During this data setup or pre-installation phase, any robot standard features or specific robot features required by the customer may be added, such as adding customer requirements such as the robot environment, staff, and other known robot users, recognizing maps, place names, limitations of the robot environment (e.g., the robot is not allowed to enter stairs, storage closets, etc.), etc. This may not be in any particular order. In analogy, the pre-installation phase is like a conversation between the robot installer and the customer, where they communicate about what information is needed for the robot to effectively operate according to the customer's wants and needs, in addition to any basic data that may be downloaded during the initialization process. Once the data is selected and collected (e.g., if facial recognition images of recognized users need to be obtained), it is uploaded by the server front-end via the user interface. Any data that needs to be processed is processed in the server back-end, and the resulting data is stored in the server database.

[0056] Each robot is preferably given a unique robot ID. The location of the data is then linked to the unique robot ID (there are standard ways known in the art, e.g., a list of pointers or structures, but no specific ways so far) and is ready and waiting for initialization to occur. The robot ID can be unique by using something as simple as an iterative number, or it can use more complex means of generating unique numbers. Note that the data downloaded to the robot during the initialization phase may not be unique, unlike the robot ID, which is always unique. For example, there may be a customer who has two robots with two different unique robot IDs, and the robots may be used in the same environment, so downloading similar or identical data.

[0057] Robot Server Communication: The server and the robot need to communicate under three circumstances: initialization, update, and IAmAlive PING. To communicate with each other, the robots are equipped with a server communication device and a router.

[0058] IAmAlive PING: Used to check the reachability of a host using echo request and echo replay messages. The ping tool is used to check the connection between the sender and the host. The server requests the robots to send an IAmAlive ping to know which robots are connected to the server. This IAmAlive is preferably sent every 5 minutes, but many other time frames are possible: every hour, every 10 minutes, every minute, every millisecond, etc. This allows the server to know all the robots connected to it for communication of updates, etc.

[0059] Initialization: The process starts with the robot brain checking the local database. There is no key information and no data in the local DB since the robot has not yet been initialized. This is the trigger for the brain to start the initialization process. The robot connects to the server via the router using its unique robot ID. The router starts making API requests using the provided robot ID to get data from the server. Part of this data includes the company ID, which is stored on the server and is used to distinguish which company the robot belongs to and which dataset the robots in that fleet should be linked to. The router then makes subsequent API calls using the robot ID and company ID to complete the data initialization process. The server already knows the exact data associated with the robot ID from the data setup process and sends the requested data to the robot via the robot router.

[0060] The robot downloads the data and stores it in its local database. Once the robot has this data, it can access it locally on the local database and does not need a server for action bar updates and IAmAlive PING.

[0061] update:

[0062] Server side: The update process occurs when the information described in the dataset state or any further data needs to be updated. If the update data is customer data, the update process first requires the customer to add or remove any data to the system via the user interface (e.g., if a new staff member joins, a facial recognition photo is required; if a user leaves, facial recognition is not needed anymore and facial recognition photo is requested to be removed; if renovation occurs, maps and place names need to be changed, etc.). This is similar to the pre-initialization phase of data setup. Next, the server needs to notify all robots with robot IDs that are in the range of the information update, in whole or in part, that an update is needed. By pinging IAmAlive, the robots know, in whole or in part, which robots are currently connected to the server and which robots have robot IDs associated with the data update. Then the server sends an update message to all robots currently connected to the server that have robot IDs associated with the data and / or software update (finally reaching all robots not yet connected to the server when they connect to the server). The server administrator can decide when an update occurs, or the robot's internal AI downloads the update when the robot is connected to the server and has downtime between tasks, or some combination of both. An update causes the server data on the database to be sent through the router and stored locally on the local database, similar to the initialization process.

[0063] Robot side: An AI or state machine system may determine if the robot may need or be assisted by a set of code that is not currently installed. This trigger may be, for example, by the robot's performance, overall or on average, falling below a threshold, either constantly or at a set number of times. A trigger may also occur if the performance of a task is suboptimal or incomplete or fails. Alternatively, a trigger may also include a situation where a long time has passed since the previous update. Examples of triggers include the robot encountering an obstruction during navigation, the robot being unable to reach its desired destination, or repeated failures in speech or voice recognition. The robot may determine on its own side if there may be additional scripts or dockers or necessary general information on the robot's side that may be available on the server side. When the robot connects to the Internet or the next time it connects to the Internet, it may initiate a request to the server to receive updates from the server. The scripter or router may download the code and / or data in a process similar to that described for server-side updates. However, the API call to download from the server may contain this new code information in addition to the updated version of the pre-installed code. The scripter and the brain may then communicate to determine the best time to reboot the robot.

[0064] Component Connection List: (No Duplicates)

[0065] Router-Local DB: The Router sets up and manages the local DB. It can get the robot ID from the local DB during the initiation of API calls. For its MQTT communication, it gets the robot ID and company ID from the server DB. The Router is the middleman between the server and the robot processor, and the data is uploaded to the local DB where it is stored for access by the Router, Brain, and Features.

[0066] Router-MQTT Broker (local): The Router communicates with the Brain and Scripter using a local MQTT Broker. The Router tells the Brain when the local DB is updated, sends the patient registration form to the Brain, sends a reboot command (which comes from the server and the Brain executes the reboot), and runs "I am alive" on the Brain. It also tells the Scripter to update the Docker software. When the Brain sees that the local DB is not yet set up, it sends an initialize command to the Router.

[0067] Scripter - MQTT Broker (local): The scripter uses the MQTT broker to communicate with the router and the brain to know when to download new dockers and reboot the robot. The router tells the scripter when new dockers are available from the server, and the brain tells the scripter when the robot is ready to reboot.

[0068] Scripter-Local DB: When the robot boots, the scripter accesses the local DB to check if Docker is already initialized. If not, the scripter starts the initialization. During the initialization, the scripter accesses the local DB to get the robot ID needed to make API calls. The scripter can remember the downloaded software in the local DB.

[0069] Scripter-Server Backend: Whenever the scripter wants to download new software for the robot, it contacts the server backend via an API call. During initialization, the scripter presents the robot ID to the server, which has already associated a feature set with the robot ID, and the server then commands the scripter to be downloaded.

[0070] Router-MQTT Broker (Server): The router uses the server's MQTT broker to receive update notifications and data and pass them to the brain. Update notifications can be for local DB, software, and data can be for registration form. Reboot command is also sent through the server MQTT.

[0071] Router-Server Backend: An API call is made to "I am alive" to read the data that the router has stored in the robot's local DB. The router can also update the server with new data via the server backend.

[0072] Local DB-Feature: A feature accesses data it needs to operate from a local DB, e.g. face vectors for face recognition, maps / locations for navigation, and user data for terre-presence.

[0073] Local DB-Brain: The brain accesses the local DB of data needed for the command, such as the map location to send as a navigation destination.

[0074] Brain - MQTT Broker (local): The brain communicates with all features via MQTT and does the "I am alive" thing. It sends commands to all features and receives status updates from all features.

[0075] Brain-Server Backend: (Connection not shown.) This is an optional connection to increase the speed of downloading and uploading certain data.

[0076] MQTT Broker (local) - Feature: The feature uses MQTT to receive commands from the brain and give feedback to the brain, for example navigation destinations, terms to pronounce for speech-to-text, and screen transitions for faceapp (UI).

[0077] MQTT Broker (Server) - Server backend: Passes update notifications and selected data from the server to the router.

[0078] Server Backend - Server Frontend: The server frontend forwards the data received by the user interface and uploaded by the developer to the backend where it can be forwarded to the router via MQTT and API calls or stored in the server DB.

[0079] Server Backend - Server DB: The Server DB stores data that can be accessed, processed, added, or deleted by the Server Backend.

[0080] Server Front End - User Interface: Developer or customer inputs data such as map data, face vectors, and robot id into the server.

[0081] Figure 2 shows the server side structure.

[0082] The server structure contains a systematic way to store and access data and software for all robots of all companies or facilities. This server structure can be a single server or multiple servers in communication. At the highest level, there is full server access. This can be held, for example, by administrators and technical staff who operate all the robots. This level can have general data shared by all robots. Below that level, there can be a company level. Alternatively, this can be a company / facility level if the company / facility levels are exactly the same. Each company has a unique company ID, which points to a collection of database data that is common to all robots of that company (see local database), and optionally a set of facilities, and a set of robot IDs, each of which points to a specific robot of the company. Examples of company data can be a company logo to display in FaceApp, company applications, company policies and standards, company staff, etc. At a lower level, there is a robot level. This can include a unique robot ID and robot level data. Examples of robot level data can be, for example, the robot name, and diagnostic or performance data. Each robot ID will point to a set of features. Each robot may contain a set of different, similar, or identical features. Each feature has a set of feature requirements, which are the dockers required to make the feature function. Currently, each feature only requires its own docker, but there may be features that require multiple dockers to operate, or multiple docking to operate more efficiently or effectively. For example, navigation may rely on image processing to avoid obstacles. These features and their level of dependency on the dockers are stored in a feature requirement list. This allows the robot initialization to know exactly where its features are located and what data to download during the initialization process. It also allows the appropriate allocation of dockers to each robot depending on the robot's intended functionality.

[0083] This can have applications for improving performance and cost. For example, downloading and running every docker on every robot can overload and even damage the CPU, or require a much larger and more expensive CPU to function. It is much more efficient and cost effective for each robot to run only the dockers it needs for its specific function, rather than all of the software as a packaged item.

[0084] FIG. 3 shows a flow chart for initialization.

[0085] Creating data and ID (step 101):

[0086] The robot is given the robot ID, API URL, and MQTT broker information for the local broker and the server broker. The robot can also be given the MQTT broker information for the local broker and the server broker, or alternatively this is provided by the scripter which pulls this information in step 6.

[0087] A customer uses a user interface to upload a data set to the server database, which is done via a server front-end and a server back-end route.

[0088] This data set can be associated with a company ID.

[0089] The robot ID (step 102) is associated with a subset of features and / or data selected from the dataset created in the previous step.

[0090] The robot starts / boots (step 103). In addition, at any point up to this step the robot can be assembled and its smart connections initialized.

[0091] The scripter checks the local DB, certain parts of the local DB are empty and fields / files / folders should be non-zero if the robot is already initialized. Empty / non-existent fields indicate that the robot is not yet initialized.

[0092] Next, trigger the scripter to start the initialization.

[0093] The scripter already has the robot ID and URL from the robot's local storage and uses them to make an API request to the server to download the required feature set, which may include a router, a brain, and an MQTT broker (local).

[0094] All processor modules that need to connect to a local MQTT broker (step 104) will therefore do so using the information loaded in step 1.

[0095] The brain checks the local DB (step 105).

[0096] Certain parts of the local DB are empty and fields / files / folders should be non-zero if the robot has already been initialized. Empty / non-existent fields indicate that the robot has not yet been initialized.

[0097] Next, the brain is triggered to begin initialization (step 106).

[0098] The brain sends an MQTT command to the router to start the initialization process.

[0099] The router begins an initialization process with the server (step 107).

[0100] The router already has the robot ID and URL from the robot's local storage and uses them to make an API request to the server.

[0101] The router retrieves the company ID from the server as part of the first data set pulled from the server DB via an API request.

[0102] A subsequent API request (step 108) using the robot ID, company ID, and URL is made to assemble a local database of robots (this step can occur before, during, or after step 9).

[0103] The router downloads robot-specific user information, including but not limited to face vectors, map information, etc., to a local database.

[0104] Data received from the server includes a timestamp of when it was uploaded to the server, and the router stores the timestamp on the robot's local database for reference during future updates.

[0105] The router pings the brain via MQTT, where a local database is set up.

[0106] The router uses the robot ID and company ID to subscribe to MQTT topics coming from the server (step 109) and an IAMALive connection with the server begins (this step can happen before, during, or after step 8).

[0107] Update notifications and update data are sent (in a queue) over MQTT to the router, at which point it receives any queued update notifications.

[0108] The router initializes IAmAlive with the server using an API call.

[0109] The feature set pulls the required data directly from the local DB (step 110) (this step must occur after step 8, but can occur before step 9).

[0110] I am AlivePING with the server

[0111] The robot must first undergo the initialization process described above, starting from step 7 onwards.

[0112] The router starts an internal clock that pings the server via an API call at a predefined interval (e.g. once every 2 minutes or 5 minutes).

[0113] At each interval, the router may make an API call to the server using the robot's robot ID and the provided URL to inform the server that the robot is online.

[0114] The clock runs in the background and does not affect the normal functioning of the robot.

[0115] Data updates from the server

[0116] Figure 4 shows a flow chart of updating data from the server.

[0117] The robot must first undergo the initialization process described above, starting from step 9 onwards.

[0118] New data and / or software for the robot is uploaded to the server (step 201) by the development team or a user (this can happen before or after step 1).

[0119] The server pings the router via the server MQTT broker that an update is available (step 202). Note that pings for software and data updates can be on different MQTT topics, so the router automatically knows what type of update is occurring.

[0120] The router checks whether the update is new or whether it already has the data (step 203).

[0121] The update notification contains a timestamp of when the data was uploaded to the server. The robot checks this timestamp (step 204) and compares it to the timestamp stored in the local database. If the timestamp in the notification is more recent than the timestamp in the database, the update is new and the robot proceeds with the update process, going to step 5. If not, the update is canceled.

[0122] If the update is a software update, the router pings the scripter to do the update. After the API call, the scripter pings the router, which then pings the server.

[0123] If the update is a data update, the router makes the same API calls described in the initialization process (step 205) to write the new data to the local database.

[0124] Routers can be optimized to only download new data, but not re-download old data that they already have.

[0125] The router obtains a new timestamp from the data update and writes it to its local database for reference during future updates.

[0126] The router sends an MQTT message to the server and / or brain that the update is complete (step 206).

[0127] Data update from robot

[0128] FIG. 5 is a flow chart showing updating of data from a robot.

[0129] The robot is triggered to check for updates (step 301). See "Server-Robot Communication" above.

[0130] The robot sends feedback to the server (step 302) along with information regarding the trigger. This can be, for example, the nature of the task (navigation, image processing, etc.), the time, and the location of the fault. Alternatively, the robot can analyze its own performance and request specific software or categories of software from the server.

[0131] Once the failure data is collected by the server, the server side can use this data to determine if any new software is appropriate for the robot to download (step 303). This decision can be handled either by the work of a human operator on the server, or by AI software on the server itself. If the robot requests software that has already been determined to be needed, the human or AI operating the server will check if such software is available.

[0132] The server has a repository of all software available to the robot. A server-side human or AI cancels the update if the robot cannot identify any suitable new software to download from within this repository. The search criteria are determined either by the robot's request or by server-side analysis of the robot's triggers. The scripter can download the code in a similar process to that described for software updates, except that the API call to download from the server can include information about this new code in addition to the updated version of the pre-installed code. The scripter and the brain can then communicate to determine the best time to reboot the robot.

[0133] The robot checks whether the update is software or data.

[0134] If new software is available, the server pings the router via the MQTT broker (server) with information about the new software (step 304).

[0135] The scripter uses the information received in step 4 to make an API call to download the new software (step 305).

[0136] The router now pings the server and / or brain to inform them that the robot is downloading the new software (step 306).

[0137] Docker is an open source platform for building, deploying, and managing containerized applications. As defined by IBM, Docker is an open source platform that allows developers to build, deploy, run, update, and manage containers. That is, a container is a standardized executable component that combines application source code with the operating system (OS) libraries and dependencies required to run that code in any environment. Containers simplify the development and distribution of distributed applications. Containers are becoming increasingly popular as organizations shift to cloud-native development and hybrid multi-cloud environments. Developers can create containers without Docker by directly leveraging capabilities built into Linux and other operating systems. But Docker makes containerization faster, easier, and more secure.

[0138] The Docker architecture resides in technology available from Docker Inc. Docker provides the ability to package and run applications in loosely isolated environments called containers. The isolation and security allows many containers to run simultaneously on a given host. Containers are lightweight and contain everything needed to run an application, without having to rely on what is currently installed on the host.

[0139] Containers are made possible by process isolation and virtualization capabilities built into the Linux kernel. These capabilities, such as control groups (C-groups) for allocating resources among processes and namespaces for restricting process access or visibility to other resources or areas of the system, allow multiple application components to share the resources of a single instance of a host operating system, much in the same way that a hypervisor allows multiple virtual machines (VMs) to share the CPU, memory, and other resources of a single hardware server. As a result, container technology provides all of the functionality and benefits of VMs, including application isolation, cost-effective scalability, and disposability, in addition to important additional benefits such as lighter weight, improved developer productivity, and greater resource efficiency.

[0140] Robot initialization (i.e., the software and data required to operate the robot) may be performed using virtual containers such as those provided by the Docker architecture mentioned above. A virtual container may be considered a subset of a virtual machine, and the initialization process may launch the robot using a given virtual container. The use of virtual containers provides a modular approach to task elements to provide the required functionality.

[0141] Dockers are a kind of "virtual container", a small simulated operating system that contains all the dependencies to run a specific piece of code. Dockers do not run directly on the robot's hardware, but instead each runs its own virtual operating system. Therefore, Dockers need to be initialized by another piece of code that actually runs on the robot's hardware (in this case, Scripter). Each feature (face recognition, NLP, OCR, etc.) has its own Docker.

[0142] The brain is the decision maker and is multifunctional / adaptive (capable of specialization) in both performance or task function.

[0143] For purposes of describing the invention disclosed herein, a script is a piece of code that is launched by a particular scripter and tells the scripter which set of software packages (dockers) to load from a server upon initialization. The scripter is similar to firmware in that it is the only code that runs directly on the robot's hardware. All other code runs in virtual containers (dockers), which can be compared to an operating system (OS), such as windows, macOS, Linux, etc.

[0144] It will be appreciated that a script may include code / software that defines a task. For example, a robot may be specified as having a job as a clinic assistant robot. The robot's role may be to register patients, take vital signs, escort patients to the examination room, and then assist the doctor during the examination.

[0145] Thus, the script may define the task of a "clinic assistant robot." It is further recognized that a single task may require specific features. For example, to accompany a patient to a consultation room, the script may control the movement of the robot, but the robot may need to have the correct navigation map of the area downloaded to navigate accurately within the defined facility, and may also need some form of object recognition capability to prevent it from colliding with objects. In this scenario, the ability to navigate and object recognition may be considered as features required for the task.

[0146] Currently, the scripts are hard-coded on the scripter, and the scripter is preloaded with what set of dockers to pull from the server during initialization. However, in the future, this may be soft-coded, meaning that a robot can start without knowing the set of features it is supposed to download. At initialization, the scripter can present its robot ID to an "initializer" on the server, which will know what scripts the robot is supposed to have (according to the robot ID). Once the scripter receives the script, it can start downloading and launching the set of dockers detailed in the script. Once the dockers are launched, the router (one of the dockers) will complete the initialization process by downloading the data required for the robot's functionality. The soft-coded options may also affect the router initialization, since it may only need to download data related to the robot's specific feature set (e.g. maps need to be downloaded for a robot that does not have navigation). Once the scripter receives the script with the set of dockers it needs to download, it may need to pass that list to the router, so that the router knows exactly what data it is supposed to pull from the server. Alternatively, the selection of data downloaded by the router can be handled by an "initializer" on the server, similar to how it is done by a scripter: the router can present its robot ID to the initializer, which can then tell it what data to download.

[0147] Soft-coded scripts may allow developers to remotely (via a server) choose the feature sets that a robot should have, without having to hard-code the feature sets onto the robot during manufacturing.

[0148] The scripter communicates with the brain about when to reboot. When the scripter downloads a software update, it needs to reboot the robot (just like rebooting a phone or computer after downloading an OS update) before the new software can actually start on the robot. Before it does this, it communicates with the brain to find the ideal moment to reboot, ensuring the robot doesn't stop in the middle of an important task.

[0149] Only rebooting the power on and off will not affect data.

[0150] Robots can work together when connected to a server, for example if several robots in a facility have several tasks to do, the robots can upload their tasks to the server, for example to become a facility-level or floor-level task queue, and upon receiving the tasks, the server can order and categorize the tasks and send them back to the robots to better optimize efficiency.

[0151] Servers have much more processing power than the hardware on a robot: a typical server processor might have 0.5-6TB of RAM, while a robot's CPU can be expected to have 1-64GB.

[0152] At least one time during the initialization process, the robot needs to connect to the server, and at least one time during the update process, the robot also needs to connect to the server. The robot may optionally connect to the server at any time, and if so, the server may learn of this connection by doing an IamAlivePING. Outside of these times, the robot does not need to connect to the server in order to operate according to its functional requirements.

[0153] The level of detail hidden within the Robot ID, i.e., Company ID, Facility ID, Robot ID, Task ID, Category ID, may be present in one ID or multiple IDs, or may all be grouped and associated with one ID or multiple IDs.

[0154] A robot ID may hold more information than a simple 001, 002, 003.... A robot ID may be a more complex number or ID that contains information about the use of the robot so that a server or human can match a specific function or feature to a robot by parsing the robot ID. If implemented, this may improve the performance of the initialization process. This parsed robot ID may include levels of detail such as company ID, facility ID, floor ID, task ID, category ID, and may be referred to as being present in one ID or multiple IDs, or all grouped and associated with one ID or multiple IDs.

[0155] For example, if the robot ID is a 64-bit code or a set of multiple codes, The first set of numbers (or the first code) may represent a company that the robot targets; The second set of numbers (or second code) may represent a facility that the robot is intended to target; A third set of numbers (or a third code) can represent the floor the robot is targeting, A fourth set of numbers (or a fourth code) can represent the category of tasks the robot is intended for; A fifth set of numbers (or a fifth code) can represent the task the robot is intended to accomplish. Possible reasons include:

[0156] update:

[0157] Server-side vs. robot updates

[0158] The server needs to know which updates are relevant for which robots by robot ID and also by company ID if customer data is being updated. If more specific robot IDs are used as explained above, the server has a general means of analyzing which robots are affected by which updates. See the example of distribution at the company ID, facility ID, robot ID, task ID, category ID levels given above. For example, if there is a change in the company logo, it will affect all robots with the company ID. If the office is renovated, it will affect all robots with the facility ID. For example, if there is a change in the company logo, it will affect all robots with the associated IDs, etc.

[0159] Alternatively or additionally, the robot may know by its own analysis what type of updates to request from the server, for example, the robot knows what dockers and data it needs and accordingly receives only the appropriate updates.

[0160] New description of the update.

[0161] Whenever a data or software update is available, the server sends a message to the router containing information about the date / time the update occurred and the nature of the update (type of data if it is data, docker to be updated if it is software / docker). If software / docker is updated, the router passes the information received from the server to the scripter, which downloads the new docker accordingly. If there is a data update, the router itself handles the update: the router is for data and the scripter is for software.

[0162] The robot can update the server with new data. Using API calls, the robot can update the database with new data gathered during operation (facial recognition data, map data, user profile information, etc.). This data can be stored in a location on the database that can be accessed by other robots at that company or facility, as detailed by company ID, facility ID, robot ID, etc. Different types of data can be assigned different access levels that can be determined by the customer or development team. For example, a company may want to allow access to new facial recognition data only to robots in a specific facility, or to all robots across all facilities of that company.

[0163] The relationship between features and tasks.

[0164] Breaking down tasks into features. For example, a "receptionist" task is a high-level concept that may require features for speech-to-text, OSR, image recognition, and database interaction. In development, we focus on completing the set of features that we believe are essential for the robot's functionality. Any higher-level task that an end customer might think of wanting a robot to perform in their facility is broken down into these smaller features.

[0165] During the initialization process, the robot downloads a set of dockers, each of which enables certain features. During actual operation, the robot uses these features to perform the desired task.

[0166] Task level description helps customers to select the functionality they want for their robot. Features are the actual docker / code. When a customer orders a robot, they are presented with options regarding the tasks they would like the robot to perform.

[0167] Robot processor before and after initialization:

[0168] Pre-initialization only has a scripter and a local DB and a robot ID to minimize the amount of software that is loaded onto the robot before initialization. This allows for maximum flexibility and efficiency of distribution since the set of features for the robot can be determined remotely, rather than being hard-coded into the robot. In theory, three identical robots that are not initialized can end up initializing with three totally different features. All that is given to a robot before initialization (hard-coded) is its robot ID, a scripter that downloads a selected set of features onto the robot, and a local DB that stores all of the downloaded software and data locally on the robot.

[0169] After initialization, you have the actual dockers for full functionality. Once initialization is complete, the robot has a full feature set. The scripter has downloaded all the dockers required for that feature set, and the dockers have downloaded all the data they need to function.

[0170] Reference is now made to FIG. 6, which illustrates a system 100 for initializing and updating an offline robot, according to an embodiment of the present invention. The system 100 may include a server 50 capable of communicating over a network connection 5 (such as the Internet) with multiple robots 20. Although two robots 20A and 20B are shown with different sets of tasks assigned to them, in some embodiments, they may perform the same tasks. The robot 20A may be a clinic assistance robot, tasked with tasks such as registering patients, taking vital signs, escorting patients to examination rooms, and assisting doctors. The robot 20B may be a service industry robot, tasked with taking customer orders, answering questions about the menu, and serving food. It is recognized that such robots may also be equipped with accessories such as microphones, speakers, cameras, and sensors for communication and taking readings, and other hardware accessories necessary to perform their tasks, such as arms for dispensing medicines or serving food.

[0171] The example of a clinic assistant robot is used as a representative robot, the setup of which may be provided by a clinic administrator and in which the robot 20 may interact with a patient. In other embodiments, other types of robots and users may also be used.

[0172] The system 100 may have a systematic way to store and access data and software for all robots 20 of all companies or facilities. At the highest level, there is full server access. This may be held, for example, by administrators and technical staff who operate all the robots. This level may have general data shared by all robots. Below that level, there may be a corporate level. Alternatively, this may be the corporate / facility level if the corporate / facility levels are identical. Each company may have a unique corporate ID, which refers to a collection of database data that is common to all robots of that company, and optionally a set of facilities, and a set of robot IDs, each of which indicates a particular robot of the company. Examples of corporate data may be a corporate logo to display on the robot interface, corporate applications, corporate policies and standards, corporate staff, etc. At a lower level, there is a robot level. This may include a unique robot ID and robot level data. Examples of robot level data may be, for example, the robot name, and diagnostic or performance data.

[0173] The server 50 may further include a server database 51, a preprocessor 52, an initializer 54, a server updater 55, a ping recognizer 56, and a portal 57 that receives manually defined commands and inputs for the preprocessor 52 and the server updater 55. The functions of these elements are described in more detail herein below.

[0174] Each robot 20 may include a robot processor 30 that communicates with a server 50 during an initialization process to control and manage the run-time functioning of the robot 20, and may also provide and receive updates to and from the server 50.

[0175] Reference is now made to Figure 7A, which illustrates the elements of the robot processor 30 prior to initializing the robot 20. The robot processor may be manufactured to include only a scriptor 31, a local database 36, and an assigned robot ID. As described in more detail herein below, the scriptor 31 may be responsible for managing the download and execution of a virtual container that contains the features and functionality of the robot. It is recognized that until the robot is initialized, the local database 36 may be empty.

[0176] 7B shows the elements of the robot processor 30 at run-time after the robot 20 has been initialized. At run-time, the robot processor 30 may include a scripter 31 and a local database 36, along with multiple feature elements virtual container 32, a controller 33, and a router 34. The controller 33 may include a robot updater 331. The functions of these elements are described in more detail herein below.

[0177] The server 50 and the robot 20 may communicate through an MQTT (Message Queue Telemetry Transport) broker, which may be self-hosted or managed. MQTT is a lightweight messaging protocol used when clients require a small code footprint and are connected to unreliable networks or networks with limited bandwidth resources. The backend may be considered the server 50. The server 50 contains the data and software packages for all the robots according to the robot ID. Both the server 50 and the router 34 may be connected to an MQTT broker, which allows for fast communication of small messages. They may involve large downloads, unlike data or software updates made by API calls. The robot 20 may also use an MQTT broker locally to manage communication between all of its elements (i.e., the scripter 31, the local database 36, the controller 33, the router 34, and the multiple feature element virtual containers 32). This is especially important for the controller 33, which makes decisions for the robot 20. For example, when the robot 20 arrives at a specific destination, if it needs to change to a specific screen, communication is needed between the navigation virtual container, the controller 33, and the robot UI virtual container. The Navigation Virtual Container may send a message via MQTT to the Controller 33 telling it that the Robot 22 has arrived at its destination. The Controller 33, after determining what to do based on its internal logic, may then send an MQTT message to the Robot UI Virtual Container telling it which screen to display. The Robot UI Virtual Container may receive this message and then take whatever action is necessary to make the requested screen change.

[0178] The elements of the pre-processor 52 are shown in Figure 8. "Heavy processing" requiring large processor capacity may occur on the server 50, with the output stored in a server database 51. The server 50 and pre-processor 52 may provide high resource processing to provide support for intelligent operation of the robot 20. Once the robot 20 is initialized, only low resource processing tasks may be required to provide the internal intelligence of the robot 20.

[0179] The Pre-Processor 52 may include, but is not limited to, an Image Processor 521, an Audio Processor 522, and a Navigator 523. The Pre-Processor 52 elements may invoke scripts to provide the necessary motion data to activate the robot 20. All of the Pre-Processor 52 sub-elements may utilize standard algorithms known in the art.

[0180] The image processor 521 may provide object recognition, face recognition, OCR (optical character recognition), etc. The image processor may use a haar classifier for face recognition.

[0181] The audio processor 522 may be responsible for audio recognition, voice matching, NLP (natural language processing), speech to text, text to speech, etc.

[0182] The navigator 523 may continually process updated maps for the local facility using the ROS navigation stack. As described in more detail herein below, for initial navigation, the navigator 523 may operate in part from information received by the robots 20. A single robot 20 may be used to create an initial map, which may then be used by other robots 20 as a reference for their own navigation. During their own navigation, the other robots 20 may update their maps using inputs they receive from their sensors while the sensors are active. Thus, the map may be continually updated.

[0183] The server database 51 may store the output of the pre-processor 52 elements, and may store, without limitation, faces, people, authority status, objects, maps, place names, recognition terms and phrases, customer data, or any other robot operation data necessary or helpful for the robot 20 to function properly according to its assigned task. The server database 51 may also store a robot management table listing for each robot task and associated script, where each robot has a task to perform and a script and data to receive. The robot management table listing may also assign tasks to robots according to ID number and / or customer number.

[0184] The initializer 54 may be responsible, in coordination with the scripter 31, for retrieving the scripts for the functions and the necessary virtual containers from the server database 51, in order to download them to the robot 20 according to the ID of the robot 20.

[0185] As described in more detail herein below, the server updater 55 may coordinate server and data updates for both the server 50 and the robot processor 30.

[0186] As described in more detail herein below, the ping recognizer 56 may register when the robot 20 comes online in order to provide updates.

[0187] The portal 57 may be any suitable UI element and may be used by a facility administrator (for example) to manage that data. For example, a clinic administrator may use the portal 57 to add users, user information and status, doctors and nurses. Doctors may then be assigned to rooms when their shifts begin, etc.

[0188] Before initialization, the robot 20 is equipped with a scriptor 31, a local database 36, and may have an ID number.

[0189] The scripter 31 may manage the downloading and execution, via the initializer 54, of the robot's scripts, virtual containers including the controller 33 and router 34, and any virtual containers representing feature elements 32 required for the robot's functionality, according to the robot's ID number and management table list from the server database 51. Each virtual container may be considered as a packet of code that runs independently in its own virtual environment. The scripter 31 (using virtual container commands) may also, for example, shut down or power on the virtual container as needed, or check if the current version of the virtual container is up to date and perform updates as needed, as described herein below. When launched by the scripter 36, each virtual container may function independently.

[0190] The router 34 may handle downloading data required for the robot 20 features, such as maps of the feature element virtual container 32, place names, facial recognition data, etc. Once the router 34 receives the data, it may transmit the data to the controller 33. The router 34 may manage traffic between the server 50 and the robot 20 by forwarding data packets to their target IP addresses, allowing multiple devices to use the same Internet connection. As described in more detail herein below, the router 34 may also be responsible for sending I Am Alive ping signals.

[0191] The controller 33 may manage the intelligence and capabilities of the robot 20 using the feature element virtual container 32. The robot 20 may include senses and capabilities. These may be, but are not limited to, language recognition, natural language processing, vision, mobility, speech, and the ability to communicate with the patient through a user interface (such as a robot screen / face). As an example, in a clinic assistant robot, the role of the robot is to register the patient, take vital signs, escort the patient to the examination room, and then assist the doctor during the examination. The controller 33 may manage all of the components that enable the robot to fulfill this functional specification. The controller 33 may create a closed, defined real world in which the robot 20 exists.

[0192] Each robot 20 may have a unique ID that distinguishes it from other robots. Each set of data entered by the administrator (through the portal 57) as part of the setup process may also have a unique customer number. The data downloaded to the robot 20 during initialization may not be unique. An administrator using two robots 20 with two different IDs may have similar or identical data since both robots may operate in the same environment. Each robot ID may refer to a set of features. As shown in FIG. 6, each robot 20 may include a different, similar, or identical set of features. Each feature may have a set of feature requirements that are necessary to function. For example, navigation may rely on image processing to avoid obstacles. This allows the scripter 31 (following the associated script) to know exactly where its features are located and what data to download during the initialization process. It also allows the appropriate allocation of virtual environment containers to the robots 20 depending on the intended functionality of the robots.

[0193] Once the robot 20 has been initialized with its virtual environment container and associated data, it may be booted for operation.

[0194] As described herein above, the feature element virtual container 32 may provide functionality and intelligence to the features. Figure 7B illustrates virtual containers 32a-d representing features such as speech, vision, movement, and hearing. It is recognized that other features not shown may also be used.

[0195] The robot UI 35 may be used to communicate with and receive input from a patient or customer. As described herein above, the robot 20 may include accessories such as microphones, speakers, cameras, and sensors that allow communication to occur and can take readings (such as patient vitals), as well as appendages that perform specific tasks, such as a robotic arm.

[0196] The local database 36 may store a subset of the data downloaded from the server database 51 during the initialization process. The data includes data necessary for the feature element virtual container 32 to operate, such as maps, place names, recognized people, etc. Any data received from the server 50 may include a timestamp of when it was uploaded. The local database 36 may be a memory module, a hard disk drive (HDD), or a solid state drive (SSD).

[0197] There may be ongoing updates between the server 50 and the robot 20. The server 50 may have software and data updates for the robot 20, and the robot 20 may provide data requests to the server 50. Also, system data may be continually updated. For example, if the updated data is customer data, the update process may first require an administrator to add or remove any data from the system 100 via the portal 57. Further examples include adding a new staff member who requires a facial recognition photo, or when a patient is discharged, being able to remove their facial recognition photo. Other examples are changing maps, place names, etc.

[0198] When there is an update on the server side, the server updater 55 may ping the router 34 to inform all robots with robot IDs that are in the scope of the information update, in whole and / or in part, that an update is required. The router 34 may check if the update is new according to the data timestamp in the local database 36, and then send a message to the scripter 31 with the name of the virtual container to update. Any virtual container may be updated, including the element feature virtual container 32, the router 34, and the controller 33.

[0199] The controller 33 may determine if the robot 20 may need or be assisted by scripts or parts of scripts / sets of code that are not currently installed. The controller 33 may determine that the robot's performance, overall or on average, is below a threshold, either all the time or at a set number of times. It may also determine if the performance of a task is suboptimal or incomplete or failed. The controller 33 may also determine if there are additional scripts or virtual environment containers or general information required by the robot 20 that may not be found and may need to be downloaded from the server 50.

[0200] The controller 33 may further include a robot updater 331. The robot updater 331 may determine that a certain amount of time has passed since a previous update. Examples include the robot 20 encountering an obstacle during navigation, the robot 20 being unable to reach its desired destination, or repeated failures in speech or voice recognition.

[0201] The robot updater 331 may also determine if there are additional scripts or virtual containers or general information required by the robot 20 that may be available on the server 50. When the controller 33 connects to the Internet, or the next time it connects to the Internet, it may initiate a request to the initializer 54 to receive updates. The scripter 31 may download the additional scripts or virtual containers.

[0202] The server database 51 may store all software that is available to the robot 20. If the initializer 54 does not identify any new software appropriate for the robot 20 in the server database 51, it cancels the update. The condition for the search for an update may be determined either by a request of the robot or by server-side analysis of the results of any performance analysis performed by the controller 33.

[0203] The Ping recognizer 56 may, in whole or in part, know which robots 20 are currently connected to the server 50 and which robots 20 have robot IDs associated with a data update. The server updater 55 may then send update messages to all robots 20 currently connected to the server 50 that have robot IDs associated with the data update (eventually reaching all robots not yet connected to the server when they connect to the server). The server 50 administrator may decide when the update occurs, or the router 34 may download the update at a point when the robots 20 are connected to the server 50 and have downtime between tasks, or some combination of both. The update may cause server data to be sent from the server database 51 through the router 34 and stored locally in the local database 36. The router 34 may store this information on the local database 36 for reference during future updates.

[0204] When an update is available from the server 50, the server updater 55 may ping the router 34 and send a message to the scripter 31 with the name of the virtual container to update. The scripter 31 may use an API call to download the updated virtual container from the server 50. Once the updated virtual container is downloaded, the scripter 31 may communicate with the controller 33 to determine the best time to restart the robot 20 so as not to interrupt the robot 20 if it is in the middle of a critical task.

[0205] The robot 20 requires a network connection 5 with the server 50 to initialize the robot 20 for the task being controlled, and also for the robot 20 to receive any updates (e.g., navigation map updates) from the server 50 and vice versa.

[0206] The router 34 may send an "I Am Alive" ping signal. The ping signal may be received by the ping recognizer 56 if the robot is online with the server 50. The router 34 may contain an internal clock that is set to the ping server 50. The ping signal may be sent at predefined time intervals, such as every 5 minutes, every hour, every 10 minutes, every minute, every millisecond, etc. At each interval, the router 34 makes an API call to the ping recognizer 56 using the robot's ID and the provided URL to inform the server 50 that the robot 20 is online.

[0207] Thus, the robot 20 can offload much of the heavy processing that would normally be placed on its processor, allowing the installation of a smaller, cheaper processor instead. This can have applications for improving performance and cost. For example, downloading and running all the virtual containers of all the robots can overload and even damage the server's central processing unit (CPU) or require a much larger and more expensive server's CPU to function. It is much more efficient and cost effective for each robot to run only the virtual containers it needs for its specific function. Furthermore, when having a central server, serving multiple robots doing the same / different tasks can also be advantageous, since updates from one robot can benefit another robot.

[0208] Unless otherwise noted, and as is evident from the preceding discussion, throughout this specification discussion utilizing terms such as "processing," "computing," "calculating," "determining," and the like, refers to the actions and / or processes of any type of general purpose computer. It is recognized that general purpose computers include client / server systems, mobile computing devices, smart appliances, cloud computing units, and the like, or similar electronic computing devices that manipulate and / or transform data within registers and / or memory of a computing system to become other data within the memory, registers, or other such information storage, transmission, or display device of the computing system.

[0209] An embodiment of the present invention may include an apparatus for performing the operations herein. The apparatus may be specially constructed for a desired purpose, or the apparatus may generally include a computing device or computing system having at least one processor and at least one memory selectively activated or reconfigured by a computer program stored in the computer. The resulting apparatus when instructed by the software may transform a general-purpose computer into an inventive element as described herein. The instructions may define an inventive device that operates with a desired computer platform. Such a computer program may be stored in a computer-readable storage medium. Computer-readable storage media may include, but are not limited to, any type of disk, including optical disks, magneto-optical disks, read-only memory (ROM), volatile and non-volatile memory, random access memory (RAM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical cards, flash memory, disk-on-key, or any other type of medium suitable for storing electronic instructions and capable of being coupled to a computer system bus. The computer-readable storage medium may also be implemented in cloud storage.

[0210] Some general purpose computers may include at least one communication element that allows communication with a data network and / or a mobile communication network.

[0211] The processes and displays presented herein are not inherently related to any particular computer or other apparatus. Various general purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the desired methods. A variety of desired structures for these systems will become apparent from the description that follows. Further, embodiments of the present invention are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages ​​may be used to implement the teachings of the present invention as described herein.

[0212] While certain features of the invention have been illustrated and described herein, many modifications, substitutions, changes, and equivalents will now occur to those skilled in the art, and it is therefore to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit and scope of the invention.

Claims

1. An intelligent multi-function robot capable of autonomous movement, a scriptor that downloads to the robot, in accordance with the robot's unique ID number, a script detailing a plurality of functional modules associated with said unique ID number, each functional module running in a downloadable, isolated virtual environment container, said virtual environment container being downloaded from a server; one of the functional modules is a router that communicates with the server to further download a subset of operational data to support the functional module; a scripter, wherein a second one of the functional modules is a controller, and each of the remaining functional modules performs a functional feature of the robot, and the controller coordinates the remaining functional modules to control the robot; a robot database that stores a subset of the motion data; The scriptor operates when the robot is online with the server, and the controller operates at least when the robot is offline from the server.

2. The robot described in claim 1, wherein the plurality of functional features include at least one of vision, mobility, speech, and hearing.

3. The robot described in claim 1, wherein one of the functional modules is an interface between the robot and a recipient of a service provided by the robot.

4. A server for initializing a plurality of robots, each robot of said plurality of robots having a unique robot ID stored on said server and programmed to be initialized to have a particular set of characteristics for performing a set of tasks, said server comprising: a robot management table listing, for each robot, at least one feature of the particular set of features and at least one script according to the unique robot ID; an action database storing action data required to execute said at least one script; a preprocessor that invokes at least one algorithm to obtain and update said operational data; an initializer that initializes a particular robot of the plurality of robots according to the unique robot ID and an associated script when the particular robot is within a network communication distance of the particular robot; the initialization of the particular robot includes downloading, from the robot management table to the particular robot, the at least one feature of the particular set of features listed for the particular robot; A server, wherein upon initialization of the particular robot, the particular robot is enabled to operate the at least one feature independent of network communication with the server.

5. A server updater that coordinates updates of the feature sets and the operational data for the plurality of robots; a ping recognizer for identifying a ping signal transmitted from the specific robot when the specific robot is within the distance enabling network communication with the specific robot; a portal for receiving instructions and inputs for said preprocessor; The server of claim 4 further comprising:

6. The set of features is: an image processor that provides at least one of object recognition, facial recognition, and OCR (optical character recognition) capabilities to the plurality of robots; an audio processor that provides at least one of audio recognition, voice matching, NLP (natural language processing), speech-to-text, and text-to-speech capabilities to the plurality of robots; a navigator that processes a map of a local facility for the plurality of robots; The server of claim 4 , comprising:

7. A plurality of robots, each robot of the plurality of robots having a unique robot ID and programmable to have a particular set of characteristics for performing a set of tasks; and a plurality of servers for initializing the plurality of robots, wherein at least one server of the plurality of servers: a robot management table listing, for each robot, at least one characteristic and at least one script according to the unique robot ID; a server database for storing organized collections of electronic data, one of said organized collections of electronic data comprising an operations database storing operations data necessary to execute said at least one script; a preprocessor that invokes at least one algorithm to obtain and update said operational data; an initializer that initializes a particular robot of the plurality of robots according to the unique robot ID and an associated script when the particular robot is within a network communication distance of the particular robot; the initialization of the particular robot includes downloading, from the robot management table to the particular robot, the at least one feature of the particular set of features listed for the particular robot; Upon initialization of the particular robot, the particular robot is enabled to operate the at least one feature independent of network communication with the plurality of servers.

8. The system described in claim 7, wherein at least one of the plurality of servers includes an organized collection of data including a specific set of characteristics, and at least another of the plurality of servers includes an organized collection of data including a specific set of roles.

9. The at least one server further comprises: a server updater that coordinates updates of the feature sets and the operational data for the plurality of robots; a ping recognizer for identifying a ping signal transmitted from the specific robot when the specific robot is within the distance enabling network communication with the specific robot; a portal for receiving instructions and inputs for said preprocessor; The system of claim 8 , comprising:

10. The set of features comprises: an image processor that provides at least one of object recognition, facial recognition, and OCR (optical character recognition) capabilities to the plurality of robots; an audio processor that provides at least one of audio recognition, voice matching, NLP (natural language processing), speech-to-text, and text-to-speech capabilities to the plurality of robots; a navigator that processes a map of a local facility for the plurality of robots; The system of claim 8 , comprising:

Images