System and method for model training, knowledge management, and course content generation

Abstract

A system and method are provided for generating models to be used in automatically generating electronic content. The method includes obtaining content from existing sources, training a subject matter specific model using the content, and providing access to the subject matter specific model to leverage the subject matter knowledge an expertise trained into the model to generate a course.

Description

SYSTEM AND METHOD FOR MODEL TRAINING, KNOWLEDGE MANAGEMENT, AND COURSE CONTENT GENERATIONCROSS-REFERNCE TO RELATED APPLICATION(S)

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 706,278 filed on October 11 , 2024; and to U.S. Provisional Patent Application No. 63 / 874,230 filed on September 2, 2025; the contents of both being incorporated herein by reference in their entirety.TECHNICAL FIELD

[0002] The following generally relates to model training, knowledge management, and generating course content, including automated processes for same.BACKGROUND

[0003] Generating and publishing electronic content can be time consuming, particularly when the content is to be presented in a specific way in order to have a desired or otherwise beneficial effect. For example, content for electronic training (e-training) or electronic learning (e-learning) often required consideration of what type of content is included, in what order, and in what fashion. Typically, this is a manual process based on inputs from a client or customer, previous experience, and previous work.SUMMARY

[0004] In one aspect, there is provided a computer-implemented method of generating models to be used in automatically generating electronic content, comprising: obtaining content from existing sources; training a subject matter specific model using the content; and providing access to the subject matter specific model to leverage the subject matter knowledge an expertise trained into the model to generate a course.

[0005] In another aspect, there is provided a computer-implemented method of generating models to be used in automatically generating electronic content, comprising: enabling local content to be placed into a repository; having a training engine access the content in the repository to train a client specific model; storing the client specific model; and enabling the client specific model to be used internally by an organization to generate a course.

[0006] In an implementation, the method may further include enabling the client specific model to be licensed.1CPST Doc: 1410-2598-6841.1

[0007] In implementation, the method may further include providing access to the client specific model in a marketplace; receiving a request to license use of the client specific model for course generation; processing the request to determine agreed upon license terms; and providing a copy or access to the client specific model according to the license terms.

[0008] In another aspect, there is provided a computer-implemented method of generating learning paths, comprising: providing an initial skills assessment quiz or test; using one or more artificial intelligence techniques to analyze the results of the quiz or test to prepare a custom learning path that focuses on skills or knowledge determined to be lacking; and providing the custom learning path while reassessing the path as components of the learning path are provided to a user.

[0009] In another aspect, there is provided a computer-implemented method of scheduling course content, comprising: obtaining a schedule and preferences associated with a learner; comparing the schedule and preferences to a suggested learning path; dividing the learning path into a plurality of components; determining a schedule for the components and adding to a calendar; tracking appointments for the components as they are met or missed and adjust scheduling; and providing notifications according to the schedule and obtaining feedback to adjust the schedule prior to a scheduled component.

[0010] In another aspect, there is provided a computer-implemented method of generating a continuous learning pathway for electronic content, comprising: enabling content to be added to a knowledge bank from one or more sources; aggregating the content and separating the content into a plurality of levels; for each level, using an Al engine to generate a proficiency challenge; and enabling the proficiencies challenges to be used to assess knowledge of corresponding content.

[0011] In another aspect, there is provided a computer readable medium comprising computer-executable instructions for generating electronic content, comprising instructions that, when executed by a processor of a computing system, cause the computing system to perform any of the above methods.

[0012] In another aspect, there is provided a computing system for generating electronic content, the system comprising at least one processor and at least one memory, the memory storing computer executable instructions that, when executed by the at least one processor, cause the computing system to perform any of the above methods.2CPST Doc: 1410-2598-6841.1BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Embodiments will now be described with reference to the appended drawings wherein:

[0014] FIG. 1 is a block diagram of an example of a cloud-based course generator system for generating and publishing electronic content.

[0015] FIG. 2 is a block diagram of an example of a configuration for the course generator system.

[0016] FIG. 3 is a block diagram of an example of a client device.

[0017] FIG. 4 is a block diagram of an example configuration for an artificial intelligence(Al) module shown in FIG. 2.

[0018] FIG. 5a is a block diagram of an example of a learning management system (LMS).

[0019] FIG. 5b is a block diagram of an example of a knowledge bank that may be used with an LMS.

[0020] FIG. 6 is a schematic flow diagram illustrating the generation of a course using subject matter (SM)-specific models and client-specific models.

[0021] FIG. 7 is a block diagram of an example of a configuration for a skills assessment module shown in FIG. 2.

[0022] FIG. 8 is a flow chart illustrating operations performed in creating an SM-specific model.

[0023] FIG. 9 is a flow chart illustrating operations performed in generating a clientspecific model.

[0024] FIG. 10 is a flow chart illustrating operations performed in enabling a clientspecific model to be licensed.

[0025] FIG. 11 is a flow chart illustrating operations performed in generating a custom learning path based on an initial skills assessment.

[0026] FIG. 12 is a flow chart illustrating operations performed in scheduling a learning path according to a user’s schedule and preferences.

[0027] FIG. 13 is a flow chart illustrating operations performed in utilizing a large language model (LLM) to auto-generate at least some content for a course.3CPST Doc: 1410-2598-6841.1

[0028] FIG. 14 is a flow chart illustrating a processing flow for incorporating multiple types of media files into a published output and for wrapping data into sharable content objects (SCOs).

[0029] FIGS. 15a to 5g illustrate building a course using Al generation options.

[0030] FIG. 16 illustrates a knowledge bank dashboard.

[0031] FIGS. 17a to 17d illustrate an interactive content walkthrough.

[0032] FIGS. 18a and 18b illustrate skills matrices in a skill verification page, team level performance matrix, skills assessment matrix, and teams skills assessment matrix.

[0033] FIGS. 19a to 19d illustrate capabilities stemming from the knowledge management platform.

[0034] FIGS. 20a and 20b compare workflows for content generation in existing systems and the system described herein.DETAILED DESCRIPTION

[0035] The following describes an intelligent, iterative learning system that may include “knowledge banks”, enhanced by Agentic Al and multimodal content automation. The system is configured to integrate content creation, adaptive assessment, skill analysis, and real-time application feedback to create a continuously improving learning ecosystem. The system transforms static course libraries into dynamic, self-evolving knowledge repositories capable of understanding and organizing content automatically, generating adaptive exams and learning paths, integrating external sources such as YouTube, SharePoint, or Box; autocreating courses from video, document, and audio sources; building custom, Al-curated curricula aligned to business or project goals; and measuring both theoretical and practical application of learned skills through software plugins.

[0036] Knowledge banks as used herein may refer to collections of courses created in the a course generator authoring tool. Courses may be added by dragging and dropping them into a knowledge bank. Once populated, an intelligent agent can process the knowledge bank to read and comprehend course content (text, videos, PDFs); generate content summaries and reference mappings that identify where concepts are covered; segment content into levels (e.g., Fundamental, Intermediate, Advanced) based on customer-defined depth.

[0037] Learners can take Al-generated exams tailored to each level. These exams are dynamic and adaptive questions evolve based on prior responses to measure conceptual4CPST Doc: 1410-2598-6841.1understanding rather than memorized answers. The system produces a conceptual understanding score — the Al’s estimate of how well the learner understands the content, not merely how many answers they got correct. The Al may then generate a personalized learning path by selecting or recommending specific content modules within the knowledge bank.

[0038] As the learner progresses, the intelligent agent may periodically introduce “knowledge check” questions to validate retention. Based on performance, the learning path can adjust dynamically, learners can retake assessments to demonstrate improved understanding, each iteration refines the learner’s score and learning journey until a defined mastery threshold (“passing grade”) is achieved.

[0039] All learner performance data can be visualized through knowledge matrices, which use color coding to display understanding levels across users, groups, or knowledge banks; track readiness for projects, onboarding progress, or competency development; support filtering by individual, team, course, or organization-wide level.

[0040] These matrices can be queried by the Al agent to answer questions such as: “Which employees possess these skills?”, “Who requires minimal upskilling to meet this project’s requirements?”, etc. Knowledge banks may rely on courses built in the course generator system. The system may enables direct integration with external repositories, such as YouTube videos, SharePoint, Box.com, Dropbox, Egnyte, etc.

[0041] To maintain content accuracy and currency, the system can be configured to ensure data security and privacy when accessing external documents; maintain dynamic content synchronization, avoiding outdated or draft materials; and provide Al-generated summaries and structure external materials into tiered learning levels for assessment.

[0042] This feature enables live-linked knowledge banks that remain current with the latest document or video versions, ensuring learners are always trained on final, approved content. The system can further enable Al-driven course generation from: documents (PDFs, text, transcripts), videos (screen recordings, training clips), audio sources (podcasts, voice notes, or even virtual assistant transcripts such as Siri, Copilot, or ChatGPT dialogues), etc.

[0043] The system may enable video-to-course conversion wherein Al extracts screenshots as images, audio is transcribed to text and closed captions, and content is analyzed contextually to create text, audio, and visual learning components. The resulting course can include microlearning modules, assessments, and reporting features. The system may also enable document-to-course conversion wherein text and images are5CPST Doc: 1410-2598-6841.1extracted, understood, and automatically structured into instructional materials; and exams and activities are generated based on the extracted concepts. This allows entire courses to be automatically generated from unstructured content with minimal human intervention, drastically reducing production time. Agentic Al enables autonomous curriculum creation.

[0044] As such, instead of manually assigning courses, users can upload scope documents or specify desired learning goals; and the Al can analyzes required skills, compare them with available content (existing courses, videos, documents), and automatically generates a curriculum. The Al can choose to use existing courses, recombine course modules, and generate new content using linked sources.

[0045] For example, users can simply type, “Create a Knowledge Bank to achieve [goal],” and Agentic Al will construct the knowledge bank structure, the curriculum and learning paths, and multi-level assessments and reporting dashboards. This creates a fully automated, goal-driven training system. The learning and course creation process is iterative and continuous, designed to adapt to evolving job roles, technology updates, and organizational needs; prevent learner overwhelm by delivering content in manageable, progressive chunks; and ensure that all training content remains current, relevant, and appropriately scoped. The system can reanalyze learner data over time to refresh course materials and learning paths automatically. The system therefore extends beyond course delivery into practical skill application within software environments.

[0046] A plugin (overlay) may be integrated with third-party software to observe user actions in real time, compare workflows against defined best practices or company standards derived from course content, provide instant feedback or corrective prompts to reinforce correct behavior, and generate custom micro-courses on the spot to address detected knowledge gaps.

[0047] Benefits of the system includes that it can bridge the gap between theoretical knowledge and applied skills, enable performance-based assessment and adaptive retraining, and produce rich data on real-world competency for employers. The combined innovations achieve efficient content creation — automating conversion of existing media into structured courses; adaptive learning delivery — ensuring each learner receives only what they need to close skill gaps; organizational insight — identifying optimal team assignments and upskilling paths; and practical reinforcement — verifying learning through real-world application and feedback.6CPST Doc: 1410-2598-6841.1Course Generator System

[0048] Referring now to the figures, a course generator system 12 is shown in FIG. 1 , which provides a computing platform or computing environment 10, e.g., a cloud-based computing platform of one or more computing servers, for users to generate content, either individually or directly, or via access provided by a learning management system (LMS) connectable to the course generator system 14. In this example, the course generator system 12 is coupled to one or more networks 16 to enable an LMS 14 and / or client devices 18 to access and utilize the system 12. It can be appreciated that multiple LMSs 14 and client devices 18 can access the course generator system 12 via the network(s) 16 (as shown), and the particular configuration shown in FIG. 1 is illustrative only.

[0049] The course generator system 12 generates courses and may utilize templates for creating such courses, which may be stored in a courses and templates storage device 20. It can be appreciated that the storage device 20 is shown separately from the course generator system 12 for ease of illustration and to indicate that the storage device 20 may be directly or indirectly accessible to the LMSs 14 and client devices 18.

[0050] The course generator system 12 also includes or has access to a large language model (LLM) 22 or other available machine learning (ML)-based model that can be utilized to automate at least one process used in generating a course, template, content, etc. as described herein. That is, to address the technical challenges associated with authoring, arranging and compiling content, the LLM 22 may be leveraged by the system 12 to not only automate the authoring, compiling and publishing processes, but to overcome various technical challenges associated with obtaining, formatting, scaling, storing, streaming, editing and performing various other multi-media operations that would otherwise be slow and inefficient in a manual publishing environment. The LLM 22 and / or the course generator system 12 may access third party sources 24. For example, the LLM 22 may find or determine content by accessing publicly available content over the internet by utilizing communication and data transfer interfaces between entities shown herein.

[0051] As discussed further below, in additional to utilizing an internet-based tool such as the LLM 22, the system 12 may provide an ability to leverage Al to generate SM-specific as well as client-specific models based on information, data, files, past courses and any other data and information available to the system 12 or the LMS 14.

[0052] The network 16 shown in FIG. 1 is a network 16 such as a wired and / or a wireless communication system, for example, an Internet-based network accessible or7CPST Doc: 1410-2598-6841.1otherwise used by the system 12. The network 16 can include a communications network such as a telephone network, cellular, and / or data communication network to connect different types of communication devices. For example, the network 16 may include a private or public switched telephone network (PSTN), mobile network (e.g., code division multiple access (CDMA) network, global system for mobile communications (GSM) network, and / or any 3G, 4G, or 5G wireless carrier network, etc.), WiFi or other similar wireless network, and a private and / or public wide area network (e.g., the Internet).

[0053] FIG. 2 provides an example configuration for the course generator system 12. In certain embodiments, the system 12 may include one or more processors 30, and one or more communication interfaces 32, which may include interfaces to communicate with external networks, media, content, storage devices, computing devices, etc.Communication interfaces 32 enable the system 12 to communicate with one or more other components of the computing environment 10, such as client devices 18 (or one of its components) or LMSs 14 (or one of its components), via a bus or other communication network, such as the communication network 16. While not delineated in FIG. 2a, the system 12 includes at least one memory or memory device that can include a tangible and non- transitory computer-readable medium having stored therein computer programs, sets of instructions, code, or data to be executed by processor 30. FIG. 2 illustrates examples of modules, tools and engines stored in memory on the system 12 and operated by the processor 30. It can be appreciated that any of the modules, tools, and engines shown in FIG. 2 may also be hosted externally and be available to the system 12, e.g., via a communication interface 32. Similarly, the courses and templates data storage 20 may be hosted externally as shown, or internally (not shown).

[0054] In the example embodiment shown in FIG. 2, the system 12 includes a course generator server application 34 that can be accessed by client devices 18, e.g., via a course generator application 64 and / or web browser 66 (see FIG. 3 described below). The server application 34 in this example configuration includes a version control module 36 to control versioning of courses accessed by multiple entities, a compiler module 38 to compile and publish courses, and a template generator 40 to enable users or organizations to generate templates for course generation, e.g., as described in U.S. Patent No. 11 ,714,958, the contents of which are incorporated herein by reference in their entirety. The system 12 in this example also includes an artificial intelligence (Al) module 42 to allow content creators to leverage ML and other Al-related tools to more rapidly and efficiently generate, refine, update, and publish content for courses. For example, the Al module 42 may include a LLM8CPST Doc: 1410-2598-6841.1interface 44, such as an application programming interface (API) or other application-to- application gateway, to allow the Al module 42 to leverage the abilities of the LLM 22 in generating course content for the course generator server application 34, among other things. The LLM interface 44 may be configured to enable the system 12 to select from different LLMs 22 and / or different ML-based models where appropriate, to leverage specific information based on the type of course being generated. The system 12 also may include a translation module 46 to generate translated content for course materials, an insights module 48 to enable organizations to chart and evaluate the results of courses, quizzes, etc.; and an all in one system (AIOS) module 50 to allow for user management, course management, and data management pertaining to training.

[0055] A skills assessment module 43 is also shown in FIG. 2, which may be used to evaluate users based on initial assessments to generate custom learning paths to improve upon learning objectives and outcomes rather than align to strict linear learning paths. Further details of the skills assessment module 43 is described below.

[0056] FIG. 3 illustrates an example of a configuration for a client device 18. The client device 18 includes one or more communication interfaces 60 to enable the client device 18 to connect to the system 12 (and / or storage device 20) via the one or more networks 16. The client device 18 also includes a course generator application 64. The application 64 can include or have access to content generation tools, for example, a text editor or word processor, camera, video editing, etc. Also shown is a web browser 66 that may be used to access a web-based application similar to the application 64 or other content available via the internet, e.g., by accessing a secure website. As shown, content 68 can be generated and / or accessible from on the client device 18 or can be loaded into the client device 18, e.g., via a media interface 62. Also, an external content generator 70 (e.g., camera, other computer) can be used to generate such content 68. While not shown in FIG. 3 for ease of illustration, it can be appreciated that the client device 18 may be embodied using any suitable computing device such as a smartphone, tablet or laptop or desktop computer or customized computing device, which would include one or more processors and at least one memory or memory device that can include a tangible and non-transitory computer-readable medium having stored therein computer programs, sets of instructions, code, or data to be executed by such processor.

[0057] Referring now to FIG. 4, a configuration for the Al module 42 is shown. As was illustrated in FIG. 2, the Al module 42 includes an LLM interface 44 to enable the Al module 42 to leverage internet-based or other externally-based Al tools such as an LLM 22 to9CPST Doc: 1410-2598-6841.1generate course content. The Al module 42 is also configured to leverage internal data such as the courses and templates stored in storage device 20 by the system 12, to create subject matter (SM)-specific models 84 that can be used to leverage internal information to create new courses. That is, previously used templates and previously generated courses along with any files, data, and other information stored in the system 12 can be used to create SM-specific sources of intelligence or “brains” that can be leveraged by a subject matter (SM)-specific model training system 86. For example, the process shown in FIG. 4 can be configured to use pre-existing knowledge repositories and multiple collaborators to train an Al model for organization wide knowledge management. The training system 86 may be used to generate a multitude of different SM-specific models 84 that can be stored in a models database 82 for use by clients of the course generation system 12. The SM model training system 86 may be coupled to the data storage 20 to access information within the system 12. The SM model training system 86 can also utilize one or more LMS interfaces 88 to enable LMSs 14 to access the training system 86 and / or the SM-specific models 84 directly, to generate courses using both centrally-maintained models 84 as well as clientspecific models 98 (see FIG. 5a) discussed below.

[0058] FIG. 5a illustrates a configuration for an LMS 14, which enables a user to generate a client-specific model 98 using files and information internal to the organization that uses the LMS 14. The LMS 14 in this configuration includes a model training application 90 that can be accessed by users of the LMS 14 to gather available documents 92, such as, without limitation, PDFs, presentations, audio files, video files, documents, etc. that already exist and reflect client-specific knowledge that can be used to generate a client model 98 that provides the basis for leveraging Al and machine learning to generate courses.

[0059] The files and other information gathered by the model training application 90 can be placed in a model training repository 94 to provide a source for a training engine 96 to apply machine learning and other Al techniques to the contents of the repository 94 to create and train a client model 98 that can be used to infer content for generating a course. That is, the client models 98 may be used to generate organization-centric courses that use more specific knowledge bases than the wider and more general internet space that would be accessed by an LLM 22. The LMS 14 may permit multiple different client models 98 to be generated and stored in a client models database 100. The LMS 14 may interface with the system 12 by providing a course generator system interface 104 and by providing one or more client interfaces 106, such as an API for users of an organization to access the LMS 1410CPST Doc: 1410-2598-6841.1and the course generator system 12 to not only create new client models 98 but use them to create courses for the organization.

[0060] FIG. 5b illustrates a knowledge bank 300, which may be used by an LMS 14, e.g., as part of the training engine 96 or as a separate module, to facilitate skills gap assessments, learning pathways, and continuous learning based on content fed to that knowledge bank 300. It can be appreciated that the computing environment 10 (e.g., in / with an LMS 14 and / or course generator system 12) may include multiple knowledge banks 300 and one is shown in FIG. 5b for illustrative purposes only. The Al-powered knowledge bank 300 may be designed to centralize learning content from multiple supported sources 302, evaluate learner proficiency, and deliver adaptive training paths. The goal is to provide organizations with a dynamic, competency-based view of their learners’ knowledge and ensure continuous upskilling. Content aggregation can be achieved using a content aggregator 304, which ingests and aggregates data from any one or more supported sources 302. It can be appreciated that the supported sources 302 can include content from the content generator system 12, file storage platforms (e.g., Box.com, Dropbox, OneDrive, Egnyte, etc.), YouTube® videos and other video files (e.g., with closed captions), PDF documents, MS Word® and other documents, additional supported file types as needed, etc.

[0061] The interconnectivity of the components shown in FIG. 5b illustrate the process, wherein automated tools and / or users can drag and drop files into the repository, referred to herein as the Knowledge Bank 300. An Al engine 306 may be included or otherwise available to the knowledge bank 300 and its content aggregator 304 to perform an Al- powered categorization, wherein the Al engine 306 automatically scans and categorizes content from the supported sources 302 into a set of different levels of complexity, in this example, three levels of complexity, namely: Level 1 - Foundational Knowledge, Level 2 - Intermediate Knowledge, and Level 3 - Advanced Knowledge. These levels may be stored in separate datastores 308 as shown in FIG. 5b or filed and tagged in a single database.

[0062] A skills assessment application 312 may be made available to users, e.g., of the LMS 14. For each level, the Al engine 306 may also generate a proficiency challenge 310 that can be used to perform skills gap assessments of employees in one example. In one example, the knowledge bank 300 may generate for the skills assessment application 312, three challenges 310 in total (i.e., one per level), with, for example, ~50 questions each. The learner receives a score and a grading based on demonstrated knowledge.

[0063] Based on proficiency results, the Al engine 306 can build a customized course targeting knowledge gaps, accessible via a learning pathways module 314. Learners can11CPST Doc: 1410-2598-6841.1receive short, embedded questions during the course to confirm understanding and continuous feedback on progress and knowledge retention. Upon completion, learners retake the proficiency challenge 310. This creates a feedback loop such that if gaps remain, the Al engine 306 generates another personalized course. Or, if mastery is achieved, the learner receives a Green Light (Competent) status (see FIGS. 16-18 discussed later).

[0064] A continuous learning cycle 316 may additionally be implemented, to provide dynamic updates, where authors can add new content to the knowledge bank 300 at any time. Learners’ status may shift from Green (competent) to Amber (new learning required).

[0065] To account for knowledge depletion, the Al engine 306 can gradually reduce learners’ proficiency scores over time to encourage refresher content, ensuring ongoing engagement and upskilling.

[0066] The knowledge bank 300 can be hosted on any LMS 14 and can be used to track learner engagement, progress, retention, and areas of weakness. The skills assessment application 312 can then be used to provide organizational dashboards showing staff competency levels, and highlights both knowledge acquisition and knowledge decay over time.

[0067] In this way, the knowledge bank(s) 300 can provide organizations with a clear picture of workforce competency and ensure learners are continually upskilling and retaining critical knowledge. Consequently, organizations and learners can save time and resources by creating automated, adaptive training and improve ROI on training by linking learning directly to proven competence.

[0068] The ability to drag and drop from the supported sources 302 allows the system to integrate with information repositories where the Al engine 306 can then build a multimodal interactive course and a knowledge bank 300 from various internal sources and file types. Knowledge banks 300 can combine input files from various sources 302, e.g., content generated using the course generator system 12 (accessed and used by any LMS 14), and customer owned proprietary information. This also allows for cross-bank searches to be conducted, e.g., using an Al or other intelligent agent. The knowledge bank 300 can be made secure to maintain the integrity of original intellectual property (IP), e.g., translated, sliced and diced content, updated waterfall content to all sub courses on all LMSs 14, etc. The Al engine 306 can learn from internal proprietary information repositories and organizational knowledge, and should be secure from outside parties and also be trusted Al12CPST Doc: 1410-2598-6841.1generated content (i.e., as opposed to Al-generated web content or someone else's knowledge bank 300).

[0069] The skills assessment application 312 can ultimately create a skills matrix dashboard showing green, yellow or red for each person assigned to each knowledge bank 300. The system may include an unlimited number of banks 300 with small or large amounts of content. From a process perspective, the system can create just enough, just in time, “just for me” personalized learning in an instant and dynamic way. The Al-generated challenges 310 are more dynamic than traditional skills gap assessments and thus can be more efficient. The ROI of training can now be measured in verified knowledge or competence rather than simply content consumed in hours. From a technical perspective there are many technical hurdles from volume of content (data and storage), multiple types of files, security and integrity of IP and Al, etc.

[0070] As discussed further below, the system can include an intelligent agent which does Al searching (e.g. by accessing the LLM(s)) 22. This may be done within a course, within custom courses created in the course generator system 12 and allows for crosscourse searches between wrapped SCORM files which may be hosted on different LMSs 14.

[0071] The intelligent agent can be configured as a fully autonomous Al agent that can act as a teacher / instructor and create the end-to-end experience, including building a course automatically using the course generator system 12, creating new knowledge banks 300 (e.g., from scratch or by combining existing knowledge banks 300, etc.), use generative Al to create content and prepare learning objectives and learning components in a multi-modal fashion. That is, the system can utilize agentic Al technologies to further automate the process.

[0072] As discussed above, knowledge banks 300 are collections of courses and other content. A user can build any course in the course generator system 12, drags it into a knowledge bank 300 causing a number of things to happen. The Al agent (e.g., via the Al module 42) can read and understand the content and therefore return a summary of the content and provide the references and locations within the course content, video or PDF that the reference was located. The Al can divide the content into three levels, or as many levels as set by the user. These can range from fundamentals through to advanced as the numbering progresses. The learner can then challenge an Al generated exam based off the content within the level. These exams may be dynamic, meaning based on a previous answer, the next answer will be generated to challenge the user’s knowledge level of the concepts contained in a level. From these exams, the Al can give the user a score based on13CPST Doc: 1410-2598-6841.1what percentage of the concepts contained within the level does the Al agent believe the user understand. It may be noted that this is not what score achieved on the exam. From this, the Al provides a learning pathway 314, pulling from the content within the knowledge bank 300. The learner can challenge these learning components, and the Al provides “knowledge check questions” as it feels it needs to check on whether the learner is retaining the information learned.

[0073] Based on these knowledge checks, and moving through the course content, eventually the learning pathway 314 will be completed, and the exam can be reattempted to prove understanding of the concepts contained in the knowledge bank 300, thus generating another grade or score for the learner’s understanding. Another learning pathway 314 may be generated off of this. The process may be iterative until the user gets a “passing grade” or an appropriate level of understanding that has been prescribed in the knowledge bank 300 as a passing grade.

[0074] All of these grades can be displayed in a knowledge matrix (see FIG. 18 described below), which are colour coded to show groups of learner and levels cross on knowledge bank 300, individual learners across many knowledge banks 300 and levels, and one knowledge bank 300 with many learners and levels, etc. These matrices can be used to show learner readiness to engage on a certain project or if it is onboarding material it can be users to see when a learner is ready to work in the company etc.

[0075] Knowledge banks 300 may require that input come from the course generator system 12, creating course content using a course authoring tool and then dragging those courses into the knowledge bank 300. However, the system may be configure to link these knowledge banks 300 to any other learning source, for example, YouTube Videos, or documents and files contained in SharePoint, Box.com, Dropbox, Egnyte, etc. Safeguards may be provided for security, privacy of information, file sharing, etc. The system can keep the source documents in the knowledge bank 300 updated dynamically to avoid training users on outdated content. The system can also perform checks to ensure the information being used is a “final” version, ensuring not to train users on draft versions of documents that are undergoing a re-write but not finalized. With this, the system can use the Al assistant to provide the summaries of information, as well as use the knowledge banks 300 to divide the content up into the various levels as previously described, and assess knowledge of the information contained in the knowledge bank 300.

[0076] The system may also enable content to be created from videos, PDFs, or audio transcripts such a copilot, chat GPT chats or SIRI conversation, etc. This enables users to14CPST Doc: 1410-2598-6841.1easily convert documents of videos into courses. Documents can be understood by the Al and converted into text, audio, exams and if the documents contain images these too can be included in the courses. However, videos contain all the components required to make a multimodal course - that being a course containing, text, audio, video, images, practical examples, exams. The videos can have screen shots that can make up images, the audio can be converted to closed captions which can then be understood by Al to turn it into text, and the Al can understand the context of the videos to make the text and audio components of the courses understandable and digestible. The goal here being if a company has a series of videos (long or short) they can be automatically converted into a multimodal course with micro learning components. Alternatively, using the recoding functions already within the authoring tool, a user could record a multi-hour video if necessary and this can be automatically converted into a multimodal course with micro learning components that is all reportable. This can significantly speed up the generation of course content because the video does not even need to be perfect as even the audio over the video once divided and spliced can be edited by Al audio to explain the concepts being explained.

[0077] Here, the goal is having videos, PDFs documents, whether loaded or saved directly in the system, or hosted on any file sharing and management system converted into learning materials with multimodal learning and micro-credentialing components.

[0078] Agentic Al (e.g., by providing an agent via the Al module 42) can be used to set a goal and make a learning pathway 314. A goal can be to make Agentic Al create learning pathways 314 automatically. As such, the process may include enrolling a learner into a series of courses (however those course are created or authored), and the learner is matched to the course based on roles, goals, or personal preferences. However, Al can understand the skill required to do a particular job. Scope documents can be uploaded as a function of the course generator system 12 to outline - create learning, courses, curriculum and learning paths to achieve these goals.

[0079] Based of all the previous points (having access to the right files, videos being auto turned into courses, and access to premade content already), the Agentic Al can make the autonomous decision to create net new, or use existing course content or just modules from within existing courses to curate custom curriculum for every person or every role or every project. Once this is added to the knowledge banks 300, the goal may be to have all the user’s content tied to the course generator system 12 and then be able to simply type into a knowledge bank 300, “make me a bank to follow these learning goals”, based off of a scope document or just a simple goal there will be a curriculum developed, and knowledge15CPST Doc: 1410-2598-6841.1bank developed and therefore the multiple level assessments from the previous examples and Al assistance to answer questions.

[0080] An iterative process may be used on Agentic Al courses and curriculum. The process described above can be configured such that it is not a “one and done” approach. That is, the system can also be iterative so that the learner can: a) keep improving their skills as time changes and requirements evolve, and b) the amount of courseware and information fed to a learner at any one time is not overwhelming. Some projects may have hundreds of hours of training that will be de-motivational, so these need to be fed to the learner piece by piece to ensure they are progressing and moving through. And, if it takes the learner a long time to complete the training make sure the trainings remain up to date.

[0081] The goal for the reporting is not only to have a skills matrix but to be able to ask the Al anything such as: “tell me who has the following skills”, or “which members of my team require the minimum upskilling to be project ready per this scope”, etc.

[0082] With a system configured in this way, the approach can be: i) to be able to make content immediately to if not very efficiently, ii) ensure each learner only gets training on the skills gaps rather than knowledge they evidently have already acquired, and iii) enable anyone in an organization to pick the right staff for the job or find the most cost effective and time efficient process to have staff upskilled.

[0083] Delivery of content and ensuring understanding can be done through practical application. While items discussed above may focus on making and delivering and reporting on content consumption, in software the system can measure the application of this knowledge and course correct the learner during their use and application of the knowledge within the software. In this way, the system can teach someone, and show that theoretically they understand it.

[0084] Additionally, the system can provide a plugin (essentially an overlay to any software) that can watch and follow the process that the learner is taking within the software. Based on the course content that has been made available to the plugin, the plugin should know and understand the best practices within the software. The goal here being to provide real time feedback to software users based on an individual company specifications or required best practices, and a) correct behaviour when using a software and b) generate on the go courseware to correct those behaviours so there are learning materials available to ensure the learner has access to the right tool to do the job as the employer wants. This can then be extrapolated into reporting and assessments by providing a course, and then using16CPST Doc: 1410-2598-6841.1the plugin to score how well the learner uses the software to deliver a project or any deliverable per the employers requirements. Then, custom content can be provided, and skill gap insights can be reported back to the employer.

[0085] FIG. 6 illustrates a scenario in which a user can access both the SM models database 82 and an internal LMS 14 and internal client model 98 to leverage multiple models 84, 98 to generate a course. For example, the course being generated may related to an internal process at an organization that would also benefits from subject matter expertise that has been trained into an external SM model 84 held by the system 12. The course generator server application 34 may therefore utilize the Al module 42 to provide an Al course generator 110 that uses any model(s) available to the user based on the organization, the LMS 14 being used and any permissions or licenses required.

[0086] The Al course generator 110 may thus be utilized to leverage Al tools and knowledge or process specific models to generate a course 112.

[0087] A configuration for the skills assessment module 43 is shown in FIG. 7. The module 43 includes a skills gap assessment module 114 to enable the system 12 to analyze user answers to tests and quizzes to automatically construct a custom learning path. The module 43 also includes a training scheduler 116 to split the custom learning path into components that are not only digestible to a user but can be fit into a preferred schedule or existing calendar while enabling reminders and other notifications.Leveraging Al to Generate Courses and Assess Skills

[0088] FIG. 8 illustrates a process that may be used by the system 12 to generate an SM-specific model 84. At block 120, the SM model training system 86 may access available sources of data such as the data storage 20 to obtain files, data, information, and other content such as past and current courses and templates generated using the system 12. Additionally, if applicable (and optional), the LLM 22 may be used to obtain internet-based content at block 122. These sources of content are then used at block 124 by the SM model training system 86 to generate an SM-specific model 84 at block 124. The SM-specific model 84 is then stored with other SM-specific models 84 to provide a set of models 84 in a data storage 82 that can be accessed by users of the course generator system 12

[0089] FIG. 9 illustrates a process that may be used by the LMS 14 to generate a clientspecific model 98, e.g., as schematically illustrated in FIG. 5. At block 130, the model training app 90 may be used to locate the files 92 and other information and content. For example, a user may drag and drop content from various sources. The files and other17CPST Doc: 1410-2598-6841.1internal information are collected in the model repository 94 at block 132. At block 134, the training engine 96 is used to access the files and information in the model repository 94 to generate a clients-specific model 98, i.e., by creating and training the model 98 using machine learning and Al techniques. The client-specific model 98 is stored in the database 100 at block 136, which enables that model 98 to be used internally to generate courses at block 138. Optionally, as shown in dashed lines, at block 140, access to the client-specific model 98 may be provided to other LMSs 14 and / or the course generator system 12.

[0090] FIG. 10 illustrates a process that may be performed when block 140 shown in FIG .9 is enabled. At block 150, the LMS 14 provides access to the client model 98 on a marketplace. The marketplace may be provided by the content generator system 12 or some other entity within the computing environment 10. In this way, the client model 98, which has been trained based on content held or available to one organization may be offered to other organizations. At block 152, the LMS 14 receives a request to license the use of the client model 98 for generating a course, e.g., via system 12. At block 154, the request may be processed by the owner of the model 98 (e.g., via their LMS 14) to negotiate and agree upon license terms. At block 156, when such terms can be agreed upon, a copy of the model 98 or temporary access to the model 98 (e.g., via system 12) may be provided per the license terms. For example, the license may restrict the number of times the client model 98 can be accessed or a duration of time for which such access is granted. The other organization may therefore leverage that organization’s expertise and / or experience to create their own course 112.

[0091] FIG. 11 illustrates a process that may be used by the skills assessment module 43 to conduct a skills gap assessment 114. At block 160, an initial skills assessment quiz or test may be provided to the user. This allows the system 12 to determine gaps in the user’s knowledge or previous training to customize the learning path to that user rather than stick to a rigid linear learning path. At block 162 the module 43 may use Al, such as by accessing an SM-specific model 84 or client model 98 or LLM 22 to analyze the results of the quiz or test and prepare the custom learning path accordingly. This may be done by identifying the questions that were answered incorrectly and by what extent they were incorrect to emphasize more content on those particular areas in the learning path. At block 164, the custom learning path may be provided to the user and this may be reassessed and adjusted at any additional stage in the learning journey, e.g., by using tests or quizzes or answers to exercises to continually assess where gaps may be in that user’s knowledge or experience.18CPST Doc: 1410-2598-6841.1

[0092] With a learning path identified for a user, e.g., using the module 43 or a traditional linear path, the system 12 may enable automatic scheduling and delivery of the content as shown in FIG. 12. At block 166 the user’s schedule and their preferences (e.g., I only wish to do the course in 1 hour increments) are obtained. The schedule and preferences are compared to the suggested learning path at block 168 to determine when and how to divide the learning path into components at block 170. For example, by being restricted to 1 hour segments, the custom learning path may require intelligent divisions or may even need to be adjusted to fit the user’s preferred scheduling.

[0093] At block 172 the schedule for the components is determined and this allows the system 12 to populate a calendar or other organization tool for the user. It can be appreciated that such populating may be done by the system 12 or by an LMS 14 or other element of the user’s organization depending on access permissions. At block 174, appointments for consuming the learning content that are met or missed are tracked and the scheduling may be adjusted. Notifications may be sent per the schedule at block 176 and can be adjusted based on feedback (e.g., snooze or reschedule selections) from the user.

[0094] Referring now to FIG. 13, a process is illustrated that may be implemented by the system 12 in leveraging the LLM 22 to provide at least some automation into the course generation process. For example, the system 12 may access an LLM such as ChatGPT4 currently or later versions thereof or similar LLMs 22 as they become available. ChatGPT4 or another LLM 22 may be used to allow course authors to answer a few simple questions such as course title, length, level (e.g., beginner vs. advanced, etc.). At block 180, the Al module 42 may present the user with such questions, e.g., by being called by the server application 34 and sent to the user via the client device application 64 to obtain inputs to feed the LLM 22. The questions provide a way to obtain a preliminary structure for generating a prompt for the LLM 22. The preliminary structure and its questions may be predetermined or can be created in real time based on the context, e.g., the associated LMS 14, the industry, the user or user type, or other metadata that can be extracted from or by the client device 18 to seed the prompt generation process.

[0095] At block 182, the Al module 42, using the LLM interface 44, feeds the inputs (e.g., by way of a prepared prompt) to the LLM 22 and receives an auto-generated table of contents for the corresponding course. The LLM 22 would generate the table of contents, which may include lectures, topics, and slide titles to display to the author. At block 184, the table of contents is provided to the author and they are able to edit them. For example, the author can delete or add a lecture, topic or slide title to ensure that the course meets their19CPST Doc: 1410-2598-6841.1expectations, requirements, etc. This first pass using the LLM 22 can be used to determine a set of categories, stages, sub-processes or other modules, chapters or divisions of the topic or process being documented in the course. The LLM 22 may have access to other contextual data to assist in aligning the prompt inputs with the desired course structure, allowing the user to, with only minimal information, obtain the first pass at the high level structure, which may then be populated as discussed below.

[0096] At block 186, an option can be selected by the author to automatically generate content using the Al module 42. The author can apply this option to specific topics or slides or to the entire table of contents. In this way, the Al module 42 can be called to again access an LLM 22 via the LLM interface 44 and populate text, audio, etc. in a selected language at block 188. The Al-generated course may then be displayed to the author in the same format as any other course created using the system 12. That is, the Al-generated content may be used to populate any portion of a course as if the user had authored it personally, with the ability to modify, edit, discard, augment or otherwise refine the Al- generated content in multiple passes. At block 190 the system 12 may enable the course content to be edited, deleted, or supplemented with video or other text, audio, etc. In addition, more content can be added to the course at any time, either currently when generating the course, or later after the course has been deployed and used. Any such additions or revisions can again utilize the Al module 42, e.g., to update / refresh content, etc. The same editing processes shown in FIG. 14 can be utilized and the Al module 42 can cause the course to be updated not only locally but changes to be pushed out to any user that has downloaded and begun using the course (e.g., as a course update similar to an app update). To enable such additions or revisions, the system 12 can store a centralized version of the current content, once published, such that when any changes are made, records indicative of who has used the course can be used to push out the changes in realtime thus allowing any user in any LMS 14 to continually be updated by the system 12.

[0097] At block 192, the system 12 can provide mechanisms to control the updating of content over time. For example, if content is added later (e.g., 1 or 2 years after being originally authored), the Al module 42 can remember the context of the course so that the content that is being added to the course (whether at the beginning, middle or end) will remain with the same flow, avoid repetition, and be in line with the rest of the course for consistency.

[0098] While not shown in FIG. 13, it can be appreciated that in the background, the Al- related modules, models, and tools can be trained. For example, a current catalogue of20CPST Doc: 1410-2598-6841.1courses created manually may be used to initially train the LLM 22 for the authoring process (e.g., to match word counts, layout, flow, etc.) and to continually train the LLM 22 as more authors create content and edit the content using the system 12. That is, the Al module 42 can be used to track what authors do or do not like in a course as well as collect feedback from users and administrators, to continue to get better at automatically generating courses and content therefor.

[0099] As noted above, in one example, ChatGPT may be used as a publicly-available LLM 22. Because GPT-type language models tend to have a large number of parameters, these language models may be considered LLMs. An example GPT-type LLM is GPT-4. GPT-4 is a type of GPT language model that has been trained (in an unsupervised manner) on a large corpus derived from documents available to the public online. GPT-4 has a very large number of learned parameters (on the order of hundreds of billions), is able to accept a large number of tokens as input (e.g., up to 2048 input tokens), and is able to generate a large number of tokens as output (e.g., up to 2048 tokens). GPT-4 has been trained as a generative model, meaning that it can process input text sequences to predictively generate a meaningful output text sequence. ChatGPT is built on top of a GPT-type LLM, and has been fine-tuned with training datasets based on text-based chats (e.g., chatbot conversations). ChatGPT is designed for processing natural language, receiving chat-like inputs and generating chat-like outputs.

[0100] As discussed above, the system 12 may access a remote language model (e.g., a cloud-based language model), such as ChatGPT or GPT-4, via a software interface (e.g., an application programming interface (API) such as the LLM interface 44). Additionally or alternatively, such a remote language model may be accessed via a network such as, for example, the Internet. In some implementations such as, for example, potentially in the case of a cloud-based language model, a remote language model may be hosted by a computer system as may include a plurality of cooperating (e.g., cooperating via a network) computer systems such as may be in, for example, a distributed arrangement. Notably, a remote language model may employ a plurality of processors (e.g., hardware processors such as, for example, processors of cooperating computer systems). Indeed, processing of inputs by an LLM 22 may be computationally expensive / may involve a large number of operations (e.g., many instructions may be executed / large data structures may be accessed from memory) and providing output in a required timeframe (e.g., real-time or near real-time) may require the use of a plurality of processors / cooperating computing devices as discussed above.21CPST Doc: 1410-2598-6841.1

[0101] Inputs to an LLM 22 may be referred to as a prompt, which is a natural language input that includes instructions to the LLM 22 to generate a desired output. A computing system may generate a prompt that is provided as input to the LLM 22 via its API 44. As described above, the prompt may optionally be processed or pre-processed into a token sequence prior to being provided as input to the LLM 22 via its API 44. A prompt can include one or more examples of the desired output, which provides the LLM 22 with additional information to enable the LLM 22 to better generate output according to the desired output. Additionally or alternatively, the examples included in a prompt may provide inputs (e.g., example inputs) corresponding to / as may be expected to result in the desired outputs provided. A one-shot prompt refers to a prompt that includes one example, and a few-shot prompt refers to a prompt that includes multiple examples. A prompt that includes no examples may be referred to as a zero-shot prompt.Processing and Compiling Multiple Media Input File Types

[0102] FIG. 14 illustrates a process flow that enables the processing and compilation of various media input file types for published outputs, including the wrapping of content into shareable content objects (SCOs) 172, further details of which are provided below. As shown in FIG. 14, various media input file types can be handled by the system 12, for example, without limitation, text, audio, video, PDF, PowerPoint (PPT), comma-separated value (CSV), compressed (e.g., ZIP), shareable content object reference model (SCORM), etc. These various input file types may be subjected to translation, where necessary, as described herein, e.g., using the translation module 46. A data processing stage may then be applied to ensure consistent and compatible formatting, compression, file size, trimming, alignment, compatibility, hosting, etc. This enables data objects to be utilized by the system 12 as described herein, for example to utilize templates, enable compiling and publishing of courses, and allow for consistent and compatible contribution by multiple users. The course generator application 34 may then generate SCOs 172 that are SCORM compatible to enable them to be used with any LMS 14 and to enable two-way data flow with the LMSs 14. Also shown in FIG. 14 is a feedback loop that allows changes to templates and courses made from such templates to be captured at metadata 174 and be fed back to the processing flow to enable changes to be centrally stored and pushed out to other users and courses that have relied upon or otherwise utilized that content. For example, if a blurb of text is edited and updates saved to the system 12, the system 12 may push out the edits to any other courses that have used the same blurb (e.g., using the drag and drop methods described later). It can be appreciated that whether Al is used to author content or that22CPST Doc: 1410-2598-6841.1content is authored manually or incorporated from elsewhere, the published output process of the course generator 34, hosting on any LMS 14, ongoing updates and feedback plus metadata feedback 174 remains consistent.SCOs and Environment-Agnostic Remote Access of SCOs

[0103] The SCOs 172 enable parent, child, grandparent relationship, versioning, translation and all dynamic updates to the content on multiple LMSs 14. File size of the SCO 172 is greatly reduced compared to a SCORM file which is uploaded manually to each LMS 14 and must be updated in each instance, each time. The SCOs 172 also enable reverse flow of data from multiple LMS systems 14 back to content generator 34 in bite sized chunks. This granular data is then 'lined up' across multiple / all LMS systems 14 to enable aggregate data and information compiling when multiple systems are not built to align.

[0104] SCOs 172 generated in an e-learning course such as that described herein can create certain challenges, in particular when communicating such objects to third-party controlled file systems or endpoints, e.g., any LMS 14. It was found that once uploaded to such an endpoint, solutions are lacking to reliably update, regulate access to, or collect telemetry from the SCOs 172 in these circumstances. Moreover, while loose standards exist regarding the structure of SCOs 172, such standards were found to be unstructured with a wide variety in the variation in the content interpretation / parsing logic used by endpoints that severely constrains the system’s ability to utilize an ednpoint-agnostic file structure and set of communication methods. The SCOs 172 utilized herein address these and other challenges as described below.

[0105] The system 12 is configured to utilize techniques to update the content of- and manage access to- data stored on third-party LMS systems 14, which requires a robust telemetry and state-tracking system. To address this, a single data representation capable of representing an arbitrary sequence of multimedia using an XML-based encoding scheme was developed, and used to embed streaming machinery within a SCO 172 without unacceptable communications and authentication issues. A number of unpredictable, adverse interactions arising from mutually conflicting limitations imposed by the target endpoints were encountered, notably with respect to the structure and packaging of data and instructions, file size / bandwidth limitations, and data parsing and rendering issues. These issues are exacerbated by the fact that there is typically no direct control over the environment in which the data is hosted and presented. As a result, a file structure and communication system were needed that are capable of delivering and visualizing arbitrary23CPST Doc: 1410-2598-6841.1SCO-embedded multimedia content independently of client-side software environment used to access, render, and interact with this information.

[0106] The system 12 was developed to include a highly flexible method to structure data to accommodate a wide variety of sequences and representation structures, as well as to address issues arising from the fact that each endpoint has its own distinct methods of parsing SCOs 172. This includes the consideration of lightweight data compression and encoding methods, as well as techniques for embedding streaming software within the constraints of SCOs 172. It is recognized that, in some cases, issues pertaining to limitations on the number of data objects that can be encapsulated in any one content package may be encountered. These limits may be address via the use of a hierarchical data structure, however, this could conflict with endpoints’ parsing systems.

[0107] Additionally or alternatively, a method to expand the capabilities of the above streaming system, alongside a set of related software techniques to allow dynamic retrieval and presentation of content independent of the browser, document, container, or client-side software environment used for rendering and interaction may be employed. A technique for embedding streaming machinery within these SCOs 172, as updates to content may render existing state-tracking obsolete and, as such techniques to track and present the appropriate version may also be incorporated. Specifically, a method to infer the nature of an update as (i.e. destructive or non-destructive), and appropriately invalidate content can be employed. Moreover, to overcome limitations in authentication and connection management as well as in the tracking of sessions and the collection of telemetry data, the system 12 employs a bidirectional communication system to allow the interface described above to identify agents that were accessing this data and compute appropriate responses (as required to continue sessions, manage permissions, and track and respond to events and requests).

[0108] It has been found that in collecting and communicating events (e.g. inputs, timing information) arising from conflicting parsing logic or channel-management requirements between endpoints issues may be encountered, for example, in suspending or terminating a session without experiencing data loss due to the indirect nature of our interface with the individuals inputting data, in identifying the identity of individuals attempting to access our data through this proxy system, and the parsing of a variety of other metadata. To address this, a technique may be employed which utilizes a large number of extremely lightweight and granular updates, which can improve overall accuracy by ensuring that failures by third-party parsing and aggregation systems would have a smaller impact on accuracy and cause fewer artifacts.24CPST Doc: 1410-2598-6841.1

[0109] Accordingly, new techniques were identified to render data multimedia objects independent of the browser, document, container, or environment that the software is operating within. Specifically, methods for embedding streaming software within a SCO file 172 to be hosted on a third-party controlled system and overcoming the subsequent session management and authentication obstacles. Methods may also be employed to ensure that the above data is reliably rendered, and to ensure that interactions the events are appropriately processed despite variations in the container or context used to visualize this information. This may require bidirectional communication methods and highly robust event and metadata parsing techniques to ensure that telemetry can be captured, stored, and processed, regardless of the properties of the endpoint used to access the content.

[0110] The system 12 therefore incorporates environment-agnostic techniques for streaming SCO-data that addresses challenges with tracking and presentation of data due to issues with parsing and unpacking of hierarchical objects. These issues may inhibit the system 12 from operating when rendered on external endpoints and may inhibit these endpoints from parsing tracking data when rendered locally.

[0111] It was postulated that an xAPI-based communication system and endpointagnostic message-bus, however conventional record stores may be incapable of tracking many of the kinds of event data required by the system 12, due data formatting requirements that severely constrained the format and type of data can be communicated, notably with regard to timing information. Additionally, API statements by the system 12 may end up returning tracking data at a high granularity, (i.e. based on a large number of child-objects which separately retrieve and communicate telemetry data). However, due to the abovedescribed obstacles in resolving the structure of the object hierarchy, the data retrieved was unable to relate the tracked-events to source objects / components. To address this, software methods for improved mapping of child objects to categorical / topical obest were developed for the purposes of analysis. A method was then postulated for manipulating the parsing and grouping logic of endpoint unpacking systems, which would allow a single file to be uploaded by the system 12, and split on the receiving end into separately tracked objects. To address scalability challenges (in that a large hierarchy would require dozens or hundreds of separate encoding and uploading operations), a method was developed to identify, group, and wrap the SCOs 172 such that endpoints would unwind the hierarchy into the correct structure during processing.

[0112] To address potential adverse interactions between the target endpoints and the large numbers of individual data stores involved in the organization and distribution of our25CPST Doc: 1410-2598-6841.1content, which was observed by experimentally reseeding a database with a limited number of data sources, it was found that a conflict may arise from a constraint in message protocols limited in the volume of addressing data it was able to pass to endpoints. To address this challenge, the messaging system was configured as described herein to reduce the overhead required to communicate meta information and addresses. A data model was developed that can successfully package and manage this data with further considerations to support the filtering and analysis.

[0113] In addition, the system 12 can provide a capability to version and share components / content of the encoded SCOs 172, both to allow multiple separate SCOs 172 to share a single piece of content, as well as to allow automatic translation of a single piece of content into multiple languages while allowing these translated outputs to update themselves in response to changes to the master / source. As updates to content necessarily impact existing state-tracking, and as the system 12 may be configured to allow rejection of updates on a piecemeal basis, to address obstacles related to the deletion of dependencies, the system 12 may use a method to infer the nature of an update as (i.e. destructive or nondestructive) and allow the merging of changes, even where intermediate updates to the source were rejected / ignored. This drafting system can limit the scope of branches within these contents, and for handling direct, live references which draw from the same source data separately from indirect ones which require a publishing step.

[0114] Translation may present an obstacle in that the layout / timing and presentation of data changes when converted between languages. Conventional systems (e.g. PDF) are incapable of dynamically restructuring information, and as such improved methods for representing and positioning information within a document were developed. This was especially problematic in the case of synchronizing closed captions of audio and video information to the timing of the source speech.

[0115] Moreover, the system 12 may be configured to manage bandwidth associated with uploading, tracking, and processing large volumes of data to multiple endpoints.

[0116] To ensure that the above data is reliably rendered, and to ensure that interactions the events are appropriately processed despite variations in the container or context used to visualize this information, the system 12 adopts bidirectional communication methods and highly robust event and metadata parsing techniques to ensure that telemetry can be captured, stored, and processed, regardless of the properties of the endpoint used to access our content. The system 12 may also utilize techniques to render data multimedia objects independent of the browser, document, container, or environment that said software26CPST Doc: 1410-2598-6841.1is operating within. Specifically, methods for embedding streaming software within a SCO file 172 to be hosted on a third-party controlled system and overcoming the subsequent session management and authentication obstacles.

[0117] To inject multimedia data into a range of endpoints possessing mutually incompatible representation and communications mechanisms can give rise to unpredictable, adverse interactions between the system’s own injection software, and the SCO sources being encoded and injected to endpoints (which are outside of the system’s scope of control). These issues may not be capable of being resolved by conditional techniques and presents technological uncertainty regarding consistent SCO delivery via this injection pipeline regardless of the content of the source SCO 172, or the behavior of the various presentation endpoints regard to parsing, structuring, and rendering, which may be mutually incompatible and so prohibits static or otherwise conventional data-structuring methods. The system 12 may thus aim to use communication methods capable of delivering arbitrary, SCO-embedded data and instructions to these endpoints.

[0118] The system 12 encapsulates telemetry data obtained from endpoint-SCO module interactions, which serves as a feedback mechanism that informs a decisioning subsystem to control subsequent actions (i.e. as individual modules within a SCO may require specific objectives and data fields). The encapsulation system can be configured to ensure that linkages between discrete content elements be maintained (i.e. sub-SCO level data / software from which interaction telemetry is acquired). The system 12 should be configured to handle event sampling and data encapsulation techniques that prevent adverse interactions between the mappings occurring between discrete content elements as managed by various data hosts, in order to maintain consistency in the rendering of the data streams.

[0119] Mechanisms used by third-party management systems to communicate with endpoints may possess mutually incompatible representations and communications mechanisms to transport distinct types of data (e.g. SCOs 172, progress indicators, etc.). Common techniques to track binary interaction outcomes (e.g. multiple-choice questions) tracked by each endpoint data rendering system were identified and the system 12 may be configured for a generic progress-tracking technique capable of handling arbitrary multimedia and SCO-encoded object hierarchies. Designing conditional logic or branching techniques to handle the entire range of cases identified may be infeasible and highly sensitive to unforeseen conditions. To resolve this problem, a general-case method may be used which enforces a separation of concerns between progress-funneling actions and27CPST Doc: 1410-2598-6841.1telemetry acquisition operations. This can improve the system’s ability to deal with the variation between the data-unpacking and parsing logic across endpoint data parsing and administration systems, and by improving software for manipulating the parsing and grouping-logic of endpoint unpacking systems so as to maintain consistency while sharing arbitrary data objects with an endpoint.

[0120] Subsequent testing revealed fine-grained, data-object mapping inconsistencies between discrete elements at the object level (e.g. upon handling multiple transformations embedded within a single document) as managed and rendered by distinct endpoints, which can cause issues where the inheritance structure of the data being processed became compromised, while attempting to generate language translations whose root differed from the master / source copy.

[0121] It was also found that modifications to the mechanism employed to encode or decode SCO data at either end of the information delivery pipeline (outside of the system’s scope of control) manifested as broken xAPI dependencies leading to data processing aberrations at the presentation / abstraction layer due to the hierarchical nature of the content being handled. Furthermore, a subsequent analysis revealed that a time-tracking mechanism can be intertwined with the process execution thread of the host window / tab of the browser, and that any action enforced onto the corresponding container can have an adverse effect on the time-tracking capabilities of the system’s mechanism. To resolve this problem, a process isolation technique may be used to separate / seclude the time-tracking algorithm. It was determined that fine-grained time-tracking capability can become somewhat limited as compared to prior iterations due to unresolved inter-process conflicts with the corresponding web browser. However, it was determined that results remained acceptable, as time-tracking granularity remained above a critical threshold.

[0122] The system 12 may also adopt methods to deliver data authored and packaged by third-party tools, which would otherwise cause adverse interactions with the system’s own data wrapper. A data-object encapsulation approach may be used that enables the system 12 to bypass the encapsulation employed by a subset of the objects being streamed, which can prevent the system 12 from gaining control over specific data elements. This issue may be addressed by a modular pipeline design that employs branching logic to identify the variables present (i.e. endpoint type, web browser, SCORM specification, etc.) so as to determine which interfaces should be referenced to access specific elements and to track more granular interactions. Telemetry acquisition problems may also be encountered, given that the external data administration systems may not receive data as expected, for which is28CPST Doc: 1410-2598-6841.1proposed employing a shim layer to intercept and manage / wrap data with our codes before being forwarded. To provide a capability to read SCOs 172, the system 12 may adopt a new software method to interpret SCO data (i.e. telemetry), and to translate it into a format that was compatible with the proprietary system 12. This approach can enable the system 12 to send SCO objects in a way that can be seamlessly interpreted by the corresponding endpoint.

[0123] To address potential time-tracking issues that may be traced to disparate data formatting requirements which can constrain the type of child-objects capable of communicating telemetry information, telemetry data may be sent by referencing divergent encoding conventions depending on the communications mechanism being employed. To enable this to occur in a genericized approach, a branching-logic technique capable of resolving object hierarchy dependencies to track telemetry at a granular level, and that is compatible with most telemetry communications conventions may be utilized.

[0124] Mechanisms employed to encapsulate progress / interaction telemetry as enabled by time-tracking primitives available by third-party content authoring tools were considered. It has been determined that for some of the objects being presented, data encoding is rather static and does not allow time-based progress tracking or audio embedding. In addition, some of the object-creation tools encode data in a way which is incompatible with the present data streaming system, leading to broken mapping with specific content elements (granular, 1 :1 correlation is lost). To resolve this issue, a candidate technique can be used that enables audio embedding and splicing, which we believe we will be able to stream into an arbitrary presentation frame in a manner that is compatible with all forms of external endpoints without requiring additional provisions.

[0125] The system 12 can therefore employ methods for delivering arbitrary SCO- encoded data, and to enable seamless telemetry tracking and analysis with this information independently of time-invariable, client-side environment version updates. To this end, techniques may be used that enable an ability to measure regardless of arbitrary modifications at either end of the content delivery pipeline. Notably, a technique can be used that enforces a separation of concerns between progress funneling actions and telemetry acquisition operations by manipulating the parsing and grouping logic of endpoint unpacking systems so as to maintain consistency while sharing arbitrary data objects between SCO relational models and an endpoint. In addition, the system 12 can be configured to resolve timing-tracking problems, and to deliver data authored by third-party tools, which had been wrapped by SCO code that was incompatible with our proposed information delivery system.29CPST Doc: 1410-2598-6841.1These approaches may be capable of supporting more than 60 different endpoint systems and up to 3 different web browser technologies, in example implementations.

[0126] The system 12 may also adopt software to ensure that the event and telemetry data generated by these objects was received and processed accurately by both the system 12 and the hosting environment (e.g., any LMS 14). Specifically, telemetry analysis methods may be used that are resilient to adverse interactions between the mappings occurring between discrete content elements as managed by an endpoint. This may be further configured to enable audio embedding and splicing currently unsupported by static methods for data representation, and to provide a wider endpoint coverage.

[0127] With respect to environment-agnostic techniques for streaming SCO-data certain data acquisition obstacles were observed that could cause data loss during session termination, which can be exacerbated by the fact that intermittent and non-reproducible inconsistencies may be observed when executing API fetch operations with these remote endpoints. While reducing transmission overhead may be attempted by modifying the data- generation systems involved in the telemetry system, sufficient reduction / compression may not be possible without unacceptable loss to the ability to provide data descriptiveness. An alternative method and early-commit process may instead be used, whereby it is postulated that modification introduced at the session loading phase of the telemetry-logic could allow the system 12 to perform granular data deliveries across the whole session duration, instead of populating them all at once during session termination, and thereby reducing objectlifecycle management issues related to interference and inconsistent behaviours exhibited by browsers during the termination phase of a session.

[0128] To this end, a content handling mechanism can be employed that dispatches session data and events into dedicated stage chunks to represent specific phases (initiation, loading, closing, etc.) and asynchronously commit them to the network as they arrive. The discontinuous aspect of this mechanism may, in some implementations be incompatible with certain specific data objects constrained to a scheduling related completion paradigm (i.e., real-time evaluation results / quizzes need a wait for completion mechanism throughout the whole session prior to committing). To address this, the system 12 can be configured to attach data objects to relevant stage chunks, and compute the corresponding encapsulation logic at each phase. This can allow the system 12 to increase transmission frequency with little impact on bandwidth consumption.

[0129] The system 12 may also employ methods to deliver arbitrary SCO data, which can otherwise cause telemetry transmission problems. While one could compute correlated30CPST Doc: 1410-2598-6841.1data properties at the acquisition layer (i.e. process SCORM score based on SCORM response) and send only compliant data objects, this may require a redeployment of existing data repositories, which could disrupt data versioning. To resolve this issue, a software method was developed for the system 12 to dynamically encapsulate SCO data (i.e. telemetry) by overriding the static interpretation logic at the instantiation level. The method loads an existing static logic from the system, then applies additional computation algorithms to modify the logic’s behavior. A switching algorithm may be used that executes a dedicated process (either computing and sending only the SCORM score or the SCORM response) depending on the presence or absence of a special character in data objects.

[0130] The system 12 may also be configured to have the capability to version and share components / content of the encoded SCOs 172, both to allow multiple separate SCOs 172 to share a single piece of content while allowing outputs to update themselves in response to changes to the master / source. While a method to utilize a hierarchy of references to achieve this, it may cause node synchronization and orphaning issues upon deletion of source data, and overheads with managing highly branched reference trees with arbitrary complexity. To address these obstacles, the system 12 may use a method to allow the merging of changes, even where intermediate updates to the source were rejected / ignored. A drafting system can be used which limits the scope of branches within these contents, and handles direct, live references which draw from the same source data separately from indirect ones which require a publishing step. Other issues may be encountered such as conflicts with the pointer / referencing system used to retrieve and present content. A dual caching mechanism may be used that stores both the published and progress version of data objects, allowing ad-hoc computation of a diff of the edited content. A supplementary deletion checking can be encapsulated and abstracted for each content type (e.g. in the e-learning context, we may have presentation slides, videos, interactive quizzes, etc.). Additional data syncing constraints may be introduced on the data configuration / generation side, as part of that logic is abstracted away for each content type, however, content can also be edited from within the corresponding data object.

[0131] As such, the system 12 can be configured for delivering arbitrary SCO-encoded data, and to enable seamless telemetry-collection and analysis with this information independently of time-invariable, client-side environment version updates. To this end, techniques can be employed that enable data acquisition / manipulation regardless of arbitrary modifications at either end of the content delivery pipeline. Notably, a content handling mechanism may be used that dispatches session data and events into dedicated31CPST Doc: 1410-2598-6841.1stage chunks that represent specific phases, attach data objects to relevant chunks, compute the corresponding encapsulation logic at each phase and asynchronously commit them to the network in an iterative process. In addition, techniques to deliver arbitrary data authored and packaged by third-party tools with the objective to support special Unicode language scripts and characters can be utilized by the system 12.

[0132] Techniques may also be employed to compute diffs and version data objects independent of the browser, document, container, or environment generating these data. Specifically, a dual caching mechanism can be used, which stores both the published and progress version of data objects, allowing diff computation of modifications affecting the edited content. The mechanism assures data tracking in a branching hierarchy by measuring the distance between each changeset of data object and subsequently applies a deletion and node integrity verification algorithm that is encapsulated and abstracted for each type of content type (Slide, Video, Quiz). This can improve the system’s ability to achieve piecemeal and incremental updating of these SCOs, especially as it pertains to context-dependent application of updates to content.

[0133] It has also been found that the injection of SCO data into a range of disparate endpoints can present obstacles in that additional constraints enforced by third-party controlled data sources (e.g. data synchronisation / duplication between individual LMSs 14) would have counterproductive effects on the system’s integration designs patterns. Notably, the acquisition layer is composed of independent LMSs 14 that individually encapsulate the ingestion logic for input data (e.g. evaluation score, course evolution, etc.), resulting in siloed environments that generate unpredictable, adverse interactions between our own injection software, and the SCO sources being encode and injected to. These issues present technological uncertainties regarding the design of a mechanism that achieves consistent SCO delivery regardless of the content or the behavior of the source / presentation SCO 172 in regard to parsing, structuring, and rendering. For example, one should consider encapsulation and communication methods capable of delivering arbitrary, SCO-embedded data and instructions to these endpoints.

[0134] Encapsulation of SCO objects can also present challenges to seamlessly track and update nested / entangled data objects, due to the fact that SCOs impose complex interdependence, integrity and synchronisation requirements with regard to inheritance, translation, and management of data objects, which may interfere with the branching hierarchy of the SCO data management process. For example, one should consider versioning methods for data objects with arbitrarily complex inheritance and / or translation32CPST Doc: 1410-2598-6841.1relationships that could prevent adverse interactions between the mappings occurring between discrete content elements as managed by various data hosts, in order to maintain consistency in the rendering of our data streams.

[0135] In implementing the system 12, limitations at the data ingestion layer were encountered that were causing data / session association mismatch during instance / session mappings, wherein authored data (i.e. data received and compiled from LMS systems 14) would be inadvertently routed to the incorrect sub-processing instances on server-side. This can be exacerbated by the fact that intermittent and non-reproducible inconsistencies were observed when executing API fetch operations with these remote endpoints. Methods may be employed to reduce mapping inconsistencies by modifying the data-generation systems involved in the telemetry system for tracking and versioning author attributes from incoming data. To this end, a mechanism may be used for dynamic in-software access to specific instance data, which dynamically computes associative links at a more granular level (i.e. topics, course, questions, etc.) between each learning item and the corresponding profile accessing it. If faced with conflicts with the pointer / referencing system used to retrieve and present content, especially when these references are nested or when objects in the source SCO are reordered without explicit change to the child objects themselves, the system 12 may use modulation techniques to deliver data authored and packaged by third-party tools in a centralized environment (i.e. dynamically route / assign telemetry data to server instances for processing). A data-object aggregation approach can be used that enables the system 12 to bypass the encapsulation employed by a subset of the objects being streamed, which could prevent the system 12 from gaining control over specific data elements.

[0136] More specifically, a modular pipeline design can be employed that utilises a programmatic branching technique (i.e. sequentially / dynamically iterating through individual interacting LMS instances) to identify the variables present (i.e. endpoint type, web browser, SCORM specification, etc.) so as to determine which interfaces should be referenced to access specific elements and to track more granular interactions. In testing, additional limitations were discovered given that the external data administration systems were not receiving data as expected, for which the system may employ a shim layer to intercept and manage / wrap data with the system’s code before being forwarded, which may improve data ingestion into third-party data systems and enable the generator 34 to send SCO objects 172 in a way that can be seamlessly interpreted by the corresponding endpoint. To enable the content generator 34 to read SCOs 172, a new software method may be used to interpret SCO data (i.e. telemetry), and to translate it into a format that was compatible with33CPST Doc: 1410-2598-6841.1the proprietary system 12. The system 12 can therefore be configured for multiple endpoint SCO-streaming that is robust to inconsistencies (i.e. timing, file types, etc.) that arise between different SCORM systems.

[0137] It was also desirable to provide a capability to version and share components / content of the encoded SCOs 172, both to allow multiple separate SCOs 172 to share a single piece of content, as well as to allow automatic 1 :n (i.e. one to many) mappings between a single piece of content and multiple target objects, as to provide dynamic updates response to changes to the master / source (e.g. such as procedural updates to multi-language translations).

[0138] To resolve these versioning / time-tracking problems that can be traced to disparate data formatting requirements which severely constrained the type of child-objects capable of communicating telemetry information (i.e. telemetry data was sent by referencing divergent encoding conventions depending on the communications mechanism being employed, which limited the concurrency capabilities of any genericized approach), a branching-logic technique can be used that is capable of resolving object hierarchy dependencies to track telemetry data at a granular level, and that was compatible with most telemetry communications conventions.

[0139] Additionally, alternative mechanisms were developed to encapsulate progress / interaction telemetry as enabled by time-tracking primitives available by third-party content authoring tools. Certain obstacles may be encountered in that for some of the objects being presented, statically encoded data structures do not allow time-based progress tracking. In addition, some of the object-creation tools encode data in a way that is incompatible with the present data streaming system, leading to broken mapping with specific content elements (granular, 1 :1 correlation is lost). To resolve this issue, a technique may be employed that enables video embedding and splicing, which we believe will be able to stream into an arbitrary presentation frame in a manner that is compatible with all forms of external endpoints without requiring additional provisions. The proposed technique for remote-process manipulation and telemetry collection can be used to allow high-resolution data collection and comparison of SCO-encoded objects of arbitrary structure across a range of models and remote management endpoints. Further considerations may be directed to managing bandwidth and computation performances (CPU and memory utilization, query optimization, etc.) associated with uploading, storing, tracking, and processing large volumes of data to multiple endpoints. Moreover, various systems may be used for tagging and a skills assessment method to perform “gap analysis” to dynamically34CPST Doc: 1410-2598-6841.1identify and map profiles to corresponding courses, as required to support development and testing activities related to the communication, versioning, and data-injection systems described above.

[0140] As such, the system 12 can be configured for delivering arbitrary SCO-encoded data, and to enable seamless telemetry-collection and analysis with this information independently of time-invariable, client-side environment version updates. To this end, techniques have been developed that enable data acquisition / manipulation regardless of arbitrary modifications at either end of the content delivery pipeline. Notably, a content handling mechanism can be used that dispatches session data and events into dedicated stage chunks that represent specific phases, attach data objects to relevant chunks, compute the corresponding encapsulation logic at each phase and asynchronously commit them to the network in an iterative process. In addition, techniques are incorporated to deliver arbitrary data authored and packaged by third-party tools with the objective to support special Unicode language scripts and characters.

[0141] In addition, techniques to compute diffs and version data objects independent of the browser, document, container, or environment generating these data are proposed. Specifically, a time-tracking mechanism allows diff computation of modifications affecting the edited content. The mechanism guarantees data tracking in a branching hierarchy by measuring the distance between each changeset of data object and subsequently applies a deletion and node integrity verification algorithm that is encapsulated and abstracted for each type of content type (Slide, Video, Quiz). This can improve the system’s ability to achieve piecemeal and incremental updating of these SCOs, especially as it pertains to context-dependent application of updates to content.

[0142] The injection of SCO data into remote endpoints presents additional obstacles in that the acquisition layer is composed of independent LMSs 14 that individually encapsulate the ingestion logic for input data (e.g. evaluation score, course evolution, etc.), resulting in siloed environments that generate unpredictable, adverse interactions between the system’s own injection software, and the SCO sources the system 12 encodes and injects into. These issues present technological uncertainties regarding the design of a mechanism that achieves consistent SCO delivery regardless of the content or the behavior of the source / presentation SCO in regard to parsing, structuring, & rendering. Encapsulation and communication methods capable of consistently delivering arbitrary, SCO-embedded data & instructions to these endpoints should be considered.35CPST Doc: 1410-2598-6841.1

[0143] To improve the capacity to version and share discrete modular components / content of our encoded SCOs to multiple target objects presents an issue to seamlessly track / lineage and update nested data objects across non-hierarchical graph data structures, due to the fact that SCOs impose complex interdependence, integrity, and synchronisation requirements with regard to inheritance, translation, and management of data objects, which may interfere with the branching hierarchy of the SCO data management process. Lineage methods for data objects with arbitrarily complex inheritance and / or translation relationships may prevent adverse interactions between discrete content elements as managed by various data hosts, in order to maintain consistency in the rendering of our data streams.

[0144] To address issues related to the ability to consistently collect telemetry and interaction data from remotely rendered SCOs without requiring direct influence over the client-side rendering environment within an LMS host, one may identify injection issues across endpoints that ultimately caused missing progress and completion across the discrete LMS solutions. It has been found that intermittent and non-reproducible inconsistencies when executing API fetch operations with these remote endpoints may occur. It was found that those issues arose when users opened an additional browser tab from the original source tab, and deduced those issues arose from limitations at the data- ingestion layer, more specifically, that the system 12 was unable to embed at runtime the LMS data injection system during the creation of new tabs.

[0145] To address this, the system 12 could synchronize the data between the tabs and potentially launch the SCORM player to retrieve the information, however, it was found that the LMS 14 may only support links to the course launcher and may not open the SCORM player directly, making this option undesirable. Alternatively, one may synchronize learner ID information and related metadata between tabs, however, when a new tab is opened, the learner ID may not available. Further, one may transmit the learner ID from the SCORM player tab to the new tab using a get parameter on tab load but the system 12 may be unable to update progress and other information from the new tab back to the SCORM player tab, which is important for updating information in the LMS 14. Therefore, it was determined that launching the SCORM player independently in the application was not feasible due to the dependency on the LMS for information retrieval.

[0146] To address the issues above, one may persist the relevant data with either a new progress value through cookies, or local storage (cache) so that the tab with the SCORM player would then poll the cookie / cache to read the latest progress and commit it to36CPST Doc: 1410-2598-6841.1the LMS 14. Synchronization issues between the tabs may occur, especially when users could use multiple tabs simultaneously to make progress in the course, if there is not an ability to coordinate progress updates across multiple tabs due to the isolation of the discrete tabs (code running in one tab could not directly access all the variables, functions, or data of another tab), and the lack of synchronization mechanisms when multiple tabs attempt to update the data simultaneously.

[0147] Concurrently with the above, the system 12 may be configured to improve its capacity to version and share discrete modular components / content of the encoded SCOs 172, both to allow multiple separate SCOs 172 to share a single piece of content, as well as to allow automatic 1 :N (i.e. one to many) mappings between a single piece of content and multiple target objects, as to provide dynamic updates response to changes to the master / source (e.g. such as procedural updates to multi-language translations). A notable obstacle may be introduced, to maintain data consistency when dealing with discrete modular components that are being assembled and reassembled across different languages, and across topologically complex non-hierarchical structures, as this complicates an efficient, and accurate lineage / versioning determination, as (1) chain of translation can lead to loss of accuracy and clarity, (2) changes to one part of the structure might have cascading effects, and (3) in non-hierarchical structures, querying relationships may require traversing a large number of nodes, severely impacting performance, especially when handling entities with a large number of connections and interactions.

[0148] To this end, software methods may be employed to efficiently and accurately track / lineage modular pieces of our encoded SCOs 172 using prospective graph data models to represent the complex relationships between entities, such as Directed Acyclic Graph structure to prevent cycling when tracing the lineage, and versioned graphs to trace the evolution of content over time. Candidate metadata systems may also be used alongside unique identifiers to improve the system’s capacity to manage complex nonlinear directional relationships. It has been found that complex sets of relationships and inheritance conditions related to the non-hierarchical data structure may exist, and issues when querying complex relationships with a large number of nodes can arise. To address this, a process for attribute inheritance has been developed, where certain metadata attributes can be inherited by translated or derived content and a set of query techniques were created with discrete traversal paths / strategies, join strategies, and depth-limiting algorithms. These techniques can efficiently parse and retrieve data throughout the complex non-hierarchical graph topology, as to determine data lineage / versioning when handling arbitrarily complex37CPST Doc: 1410-2598-6841.1mappings between a single piece of content and multiple target objects, as well as related transitive relationships.

[0149] In summary, techniques to lineage methods for data objects with arbitrarily complex inheritance and / or translation relationships with the capacity to version and share discrete modular components / content of our encoded SCOs, both to allow multiple separate SCOs to share a single piece of content, as well as to allow automatic 1 :N (i.e. one to many) mappings between a single piece of content and multiple target objects has been developed using a graph data model to represent the complex relationships between entities, alongside a metadata system with unique identifiers to improve our capacity to manage complex nonlinear directional relationships. A process for attribute inheritance, where certain metadata attributes can be inherited by translated or derived content, as well as a set of query techniques with discrete traversal paths / strategies, join strategies, and depth-limiting algorithms to efficiently parse and retrieve data throughout our complex non-hierarchical graph topology has also been configured for the system 12.Example User Interface for using Al to Generate Course

[0150] FIGS. 15a-15g illustrate an example using the Al module 42 and, when applicable, using the LLM 22 to generate a course and its content. Referring first to FIG. 15a, a course builder page 200 is shown, which provides an “Ask Al” option 202. The Al module 42 may then ask the author one or more questions as shown in FIG. 15b for ‘context’ and fine tuning ChatGPT response. It can be appreciated that the option 202 may, additionally or alternatively, be used to select an option to use a client-specific model 14 and / or an SM-specific model 84 as shown in FIGS. 4-6 and described above.

[0151] For example, questions are optional except first: “Tell us about the course you want to build”. Also, “What is the subject for this course?” For example, the answers may include Revit Software, Baking cookies, Dog Grooming, etc. The Al module 42 can also ask the author to help the system 12 improve the quality of the results with answering the following questions: “ What is the ideal duration for this course?” (example answers: 1 Hr, Less than 5 Hrs, About 4hrs), and / or “How familiar is the audience with the topic being covered in the course?” (example answers: beginner, intermediate, advanced, general, etc.), and / or “What language should the course be built in?” (example answers: English, French). As noted above, the answers are used as inputs to the LLM 22. Limits may be placed by the system 12 and / or author, e.g., a maximum number of lectures, maximum number of topics in each lecture, maximum or minimum number of slides in each topic, etc. FIG. 15c illustrates an example of a course outline that is presented to the author based on a response obtained38CPST Doc: 1410-2598-6841.1from the LLM 22. Slides may then be generated from this prompt, which generates a skeleton for the course, as shown in FIG. 15d. The Al module 42 may be used to also generate content for individual topics and slides. An example of an introduction slide is shown in FIG. 15e, which includes text generated using the LLM 22 and is editable by the author. FIG. 15f illustrates audio being added to the slide and FIG. 15g illustrates an example of a published output.Example User Interface for using Knowledge Banks for Skills Assessments

[0152] Referring now to FIGS. 16-18 various Uls are shown that may be provided, for example, via the skills assessment application 312. In FIG. 16, a knowledge bank dashboard is shown. In this dashboard, access to the Al agent may be provided as well as a progress bar to shown progress to mastery. For this user, they have achieved 85% mastery at Level 1 , 65% mastery at Level 2, and 35% mastery at Level 3. These values may go up or down based on challenges 310 and may deplete automatically over time to encourage deployment of “refresher” courses. The goal here is to have mastery (e.g., green) at all levels based on the learning path created for that user as discussed above.

[0153] FIGS. 17a-17d illustrate an interactive content walkthrough, which may be accessed by selecting “View Learning Path” in FIG. 16 after taking a proficiency exam. In FIG. 17a, step 1 of 4 is shown illustrating “Understanding React State”. FIG. 17b illustrates a setup and collaboration page, which can be used to set up an manage a particular project in this example. FIG. 17c illustrates that the learning path is categorized “per topic”. After completion of the continuous learning path for a specific topic, the learner can be given a short (e.g., max 5 questions) exam such as that shown in FIG. 17d.

[0154] FIG. 18a illustrates a skills verification Ul, which includes a skills matrix for the different levels as well as a matrix for different groups (department, team, name). In this example, different knowledge banks 300 are used, namely onboarding, role, software, and project. The skills matrix allows an organization to provide verifiable competence, e.g., to demonstrate readiness for a project (e.g., for bidding purposes) or to meet regulatory standards or other requirements. That is, an organization can use the system to obtain the go-ahead for a project by demonstrating verified competence of team members. FIG. 18b illustrates other types of skill matrices, for example a team level performance matrix, a skills assessment matrix, and a team skills assessment matrix. These allow organizations to provide custom comparisons and gain valuable insight into skills adoption and retention.39CPST Doc: 1410-2598-6841.1

[0155] FIGS. 19a-19d summarize advantages that can be leveraged from using the system described herein, in particular by providing a knowledge management platform. FIGS. 20a and 20b compare current organizational workflows (FIG. 20a) to a new workflow paradigm that can be realized using the systems described herein. It can be observed that current siloed workflow elements shown in FIG. 20a can be broken down and realigned via the knowledge management platform, in particular by using the Al assistant and other Al features from the Al module 42.Technical Challenges and Implementation SolutionsPersonalizable Roles

[0156] The system described herein may require a comprehensive permission and access system which is both easy to use and powerful enough to provide fine-grained access controls. In one implementation, a permission system can be used for creating roles and resource groups. Resource groups are organized by type and have a tree hierarchy with a single root level node. Roles and groups would then be combined when assigning them to individual users. This implementation may encounter challenges. For example, assigning roles and groups to users may not be simple to understand as it involves knowing how both the roles and resource groups produced a unique combination of access at the time they are assigned to a user. Moreover, providing different permissions into different sets of resources required having multiple roles prepared and assigning each to different resource groups.

[0157] In an other implementation, to address these challenges, the system can be configured such that access could be easily granted to users without a deep understanding of the permission system. For example, a more traditional role-based approach can be used where users could be assigned one or more roles. Roles would contain all of the information necessary to control user access. Permissions and resources can be organized into their separate categories. Access may be granted by creating a “card” for a given resource type, which contains both a set of permissions and a set of related groups. Roles then become simple containers for cards, and cards would be combined to form a complete set of permissions for a role. The issue of separate permissions into separate resources can be overcome by allowing roles to contain multiple cards for the same resource type, as long as those resources were groupable. This approach can improve permission readability because they are naturally organized by resource type, and resources that were not organized into groups could simply forgo selecting resources.40CPST Doc: 1410-2598-6841.1

[0158] While addressing simplicity, the system should still needed to allow some degree of customization for individual users. To address this, the system may customize the role at the user level. The approach is that roles would have a limited hierarchy. Normal roles could be named and given to any user, but roles can also be customized, albeit only at the user level. Individual users are assigned a role, and then have that role personalized for that specific user. In this way, all access cards are inherited from the base role. Then, existing access cards can either be replaced or removed, and new access cards can be added. This effectively creates a single use role for a given user.

[0159] While addressing usability, permissions checking systems may become complicated. When making any API call, the system would need to check the users permissions, which was dependent on the resources being accessed. Checking those permissions can be significantly more complicated because cards could be removed or replaced. This meant the check may not be a simple matter of accumulating all permissions, but also involved excluding any that had a replacement. To overcome this, a caching system can be implemented for a user’s access. Once a user’s roles are set, their permissions no longer change; and that changing a user’s roles was an infrequent action. As such, a single table in the database can be implemented, which links a user with all of their active access cards directly. This would be computed once when a user’s roles are changed, and then used any time permission checks were required. The benefit of this approach was that the permission checking code could remain relatively similar to how it worked previously, and would only need to refer to a different table when checking permissions.Direct-Access Resources in a Group-Based System

[0160] In the system, resources may be organized by type and placed within group hierarchies. This provides a structured way to organize resources and select them when assigning access using a role. An issue with this approach can be that sometimes it is desirable to select only a specific set of resources when assigning access, rather than being forced to choose a group. We wanted to support some way of assigning access to resources directly. While adding a list of resources directly to access cards may be done, doing this may increase the complexity of all permission checking code, which was extensive. Permission checking code is executed during every request, so keeping the code simple and fast was a priority. To address this, the system may use a group-based approach. Instead of assigning access to individual resources, the system places those resources in a special “virtual” group. Virtual groups would exist outside the normal hierarchy, and would not be visible when traversing groups normally. Doing this allowed us to use the same permission41CPST Doc: 1410-2598-6841.1logic already in place without the need to modify it. This approach did however introduce two more issues.

[0161] One issue to address may be that it makes dealing with groups more complicated. Groups would need to be filtered based on their “virtual” status. To alleviate this, it has been recognized that the system can expose the virtual groups only to the access card routines and only internally. This means the system could have a list of resources at the access card level, without ever exposing the virtual groups to the user or APIs. Doing this allows the system to keep the original intention of the design while using a more convenient internal statistic in the system.

[0162] Another issue acknowledges that the system’s Ul may be capable of showing a set of actions available on a per-resource basis. The issue may arise when a user was given access using the new direct resource system and they were not able to access the actions which they had been granted. In this case, a solution may be to more thoroughly model the user’s permissions on the client side. Users logging in would be able to query a complete list of their access in the system; that would be cached during the user’s session and would be used to determine a user’s available actions.Simplifying Licenses-Based Access

[0163] It is recognized that with the structure above, there may be two ways to grant access within the system. The first way is through permissions and resources. The second way is a licensing system primarily used to sell course content. However, there may be a need to deliver access to external resources as well through the licensing system. This may be handled by delivering licenses to single sign-on applications, which could be embedded within the system or open in a new browser tab. Those would be assigned like licenses but grant access to applications outside the system. It has been found that some of these external sites may be more practically implemented as a perpetual license and unlimited access. It is also recognized that the system may wish to have finer-grained control over what permissions a user was granted into these external websites. This may not fit existing license models well, since assigning licenses with unlimited access may be undesirable, and the addition of permissions may not be compatible with the license-based system.

[0164] To alleviate this issue, the system can allow licenses to be assigned at the role level. This was done by having a custom access card type which was associated with a given licensed product. However, this may not be enough to allow fine grained control over the permissions in those external systems. To support that, the system can make the42CPST Doc: 1410-2598-6841.1permission system extensible. In this way, instead of having a fixed set of permissions in the system, it may be possible to create new ones. The system can also add the ability for those new permissions to be associated with different custom applications. When a license was going to be assigned to a role, the system can now present the same type of access card normally used by roles, but with the custom permissions available instead. This matches an existing role-based system such as that described above.

[0165] With these adaptations, the system may have two different permission models, a role-based one and a license-based one. This may not be desirable in some implementations, since it can make the permission checking code more complicated, something to avoid. It is note that the license-based system can be simpler than the role based one, but they both work to achieve the same type of objective, namely granting access to specific resources. It is noted that the system could reuse the role-based access cards but instead place them at the user level. This effectively means reworking the licensebased system with a role-based access card system. For example, the system may deploy a single, access-card based system used to implement roles, permissions, user licenses, and access to external resources.Synchronizing Course Libraries Between Two Systems

[0166] It may be desired to have the system act as the “front door” for users, even when accessing content hosted in an existing LMS 14. It may be desired that all of the administrative tasks should be handled within the system, and any actions should then be synchronized with an existing LMS 14. In this way, the system may be able to support course enrollments within the system. Supporting course enrollments means that those courses should be present within the system.

[0167] One challenge is that courses already exist in two systems, the content development and delivery platform (the course generator system 12), and the LMS 14. Moreover, the system may have three different tiers of courses to be categorized, namely courses available in the system’s library and should be accessible to all customers, reseller courses created by reseller partners (and may need to be added to the customer’s available courses when a purchase is made through that reseller), and customer courses that the customer creates themselves and makes available to their staff.

[0168] One challenge faced can be in determining how to represent the courses within the overall system. Because these courses already exist in a content delivery platform, there may be a desire to implement them as courses linked to the ones in that system. This can43CPST Doc: 1410-2598-6841.1provide a one-to-one mapping of the courses within the system, and the ones on a hosting platform. An issue to address may be where those courses should be hosted. For example, two LMSs 14 may be used to host content, one managed by the host, and the other managed by one of the resellers. Both of these systems have slightly different course libraries available on them, and both of them have completely different IDs for their courses. Since the system may need to synchronize enrollments between the overall system and both of the LMSs 14, one may map the courses as external courses. This means having two sets of unique courses for each LMS 14.

[0169] The subscription management system can be made responsible for synchronizing courses and mapping them between the overall system and both LMSs 14 since it acts as a middle-man between both. Courses would be uploaded to the LMS 14 first, and made available to existing customers. Then, a subscription management system can query both LMSs 14 via their APIs and create any missing courses in the overall system through its API. Neither the overall system, nor the LMSs 14 need to be aware of one another.

[0170] One issue to address may be with the LMS’s API, if it supports querying what courses are available to the currently logged in administrator, and supports querying what courses are available to a specific user. The overall system may include a subscription system to know which courses are part of a global library, which ones are part of reseller libraries, and which ones belong to customers. In the LMS 14, this may be determined by making courses available to specific departments. Those departments form a tree, with the main library assigned to the highest level, with resellers below that and customer departments at the bottom.

[0171] To implement this, the system may select candidate users and use them as the subject when querying with the LMS API. A challenge can be that different user statuses can make some courses unavailable. For example, a customer might have no users actively set up to view a course, but the system would still want to query that course via the API. It may be difficult or even impossible to determine the best user (assuming one event existed) to query to get an accurate view of every customer’s course library.

[0172] To address this, a dummy user can be made active in the LMS 14. The dummy user can have all the correct permissions enabled for viewing courses. The API would then move that user into a department and query it for what courses were available within that specific department, then quickly move it. This strategy may have an issue when moving a user to a new department, since there may be some form of internal delay in the LMS 14,44CPST Doc: 1410-2598-6841.1where the user does not have access to any courses. In this case, it can take some amount of time, often more than 5 minutes, before the LMS 14 completes its internal processing and the user’s courses are available to them. The dummy user would be exposed to customers during this time which could pollute tracking data and allow administrators the ability to modify them. To address these additional challenges, two approaches may be used. First, a dummy user can be placed at the top level department, outside the reach of customers and resellers. This would be used to query a main catalog of courses. Then, for both reseller and customer courses, the system can have them include a specific identifier in the vendor field for their courses. The system may query the entire LMS 14 for courses, and use the vendor field to determine which reseller or customer the course belonged to.Translation Editing and Reuse

[0173] With prior systems, it was recognized that users did not have the ability to edit translations generated by a course generation tool. With the system described herein, users can edit the translation. A course may be shown in all its supported languages (vice only the authored language). From a drop down menu, the user can select the language they want to view the course in and, once the course is displayed in that language, they are able to click “Edit” and update the translation accordingly.

[0174] In one implementation, users may be allowed to edit translations and keep the link to the parent translation. However, merging these changes may become challenging. As such, in another implementation, the system can break off the connection to the original “parent” as soon as a translation is edited. This means that any future updates to the original content will no longer cascade to that edited slide upon republishing. Instead, the user will need to manage updates manually for that content, since the link to the parent no longer exists.

[0175] The system may also provide the ability for the users to drag / drop translations into courses. The prerequisite for this feature is that the course needs to be authored in the language in which the user is trying to drag. That is, in this example, there can only be one authored language for the course. In some implementations, only the authored languages could be dragged into a course, but with these revised features, any translated language can also be dragged / dropped into the course as long as the course’s authored language matches the translated language in which the user is trying to drag / drop. For example, say a course A is in French and there is another course B in English with a French translation. The user could drag the French translation of course B into Course A. This gives the user more features and allows them to re-use the translated content.45CPST Doc: 1410-2598-6841.1

[0176] When translated content is dragged and dropped from one course into another, and a translation of the receiving (“child”) course is later requested, the workflow may be configured so that the content is traced back to its original source. For example, suppose Course A is authored in English and translated into French. If an author then builds Course B in French, they can drag the French translation from Course A directly into Course B. Later, when Course B needs a German translation, the system may want to deliberately avoid translating the French text in Course B into German because it could degrade translation quality. Instead, the present process locates the original English content in Course A, generates a German translation from that canonical source, and applies those German translations within Course B. This approach preserves translation accuracy and consistency across reused content.

[0177] Another feature may allow for multiple authored languages to be used in courses. This, however, may create challenges in tracing back to the original parent language for each piece of content, adding complexity. For example, if one course is in French, English, Spanish and then a user wants to translate that content into German, the system would need to track the language for each content (vice the course) and then translate from that language to German. And, consequently, if the user updates the language and it was mismatched with what the software had, with all the versioning, further complexities can arise.

[0178] The system may also be configured to restrict the changes to the course to only occur in the authored language. Otherwise, again, the versions can be complicated to manage with addition and deletion of slides as each ‘language’ would have its own ‘version’ along with the course itself having many versions due to republishing. So, to maintain consistency and reduce technical complexity, structural content changes such as adding or deleting slides may only be allowed in the original authored language. While translations can be edited for clarity and accuracy, all structural changes can be made in the source language.

[0179] In one implementation, the system may translate content on the fly as original content was being added to the authored language outline. However, running translations every time a user added content can be costly as they could keep changing their mind on the content. Moreover, this may gate the Ul so the user would have to wait for the translations to finish before they could continue editing again. In an alternative implementation, the system can include a button to the Ul to trigger the translation for new46CPST Doc: 1410-2598-6841.1content. Making it a manual step can reduce cost, while also allowing users to preview and edit translations.Insights Assessment Re-use

[0180] There may be a desire for the knowledge assessment process to have quizzes provided “out of the box” that cannot be modified or reused. That is, there may be a desire to use them exactly as delivered, which limits flexibility and customization.

[0181] To address this, the system can provide functionality that allows quizzes to be dragged and dropped into their own custom assessments, where they can then be fully edited. Customers can then add their own questions, remove questions they do not find relevant, and adjust the quiz content to better align with their specific training goals.Intelligent Assistant in Course Search

[0182] The system may provide the ability to perform searches within a course. The main challenge with enabling content search is that while all course content is stored on the system’s servers, enrollment and authentication might be managed through the LMS 14. This may create a challenge since when a learner performs a search, it may be difficult to confirm whether they actually have access to that content through the LMS 14. Since the system is to remain LMS-agnostic, this challenge can be addressed by enabling in-course search only. That is, if a learner is already inside a course, the system can safely assume they have access. To deliver this functionality, an intelligent assistant allows learners to type a question or prompt, and the assistant searches across the course text and video closed captions to return a summary response. The assistant can also provide a list of references within the course for learners who want to dive deeper.

[0183] The intelligent assistant can also be designed with guardrails, e.g., by not allowing access the internet. Thus, if the requested information is not part of the course, it responds with a clear message stating “this content is not part of the course” instead of hallucinating an answer. In an implementation of in-course Al generation, the system may pass the whole course outline to the LLM prompt. This worked well for smaller courses, but may provide additional challenges with larger courses as the system hits the character and context-window limits. To get around the limits, the system can switch to a different LLM 22. This may work until but may have yet further challenges with video subtitles but may be addressed by using vector stores.47CPST Doc: 1410-2598-6841.1

[0184] The system may thus allow a user to upload the course outline and subtitle files into vector stores. In this way, the system does not need to include the course content in the prompts and instead may use tool calling to use the vectors and return relevant generation.

[0185] As part of the search generation the system may also wish to retrieve references to point users to content for further learning. To avoid the LLM 22 returning incorrect ids from the course outline, the system may go through the references on the backend, iterate through the course outlines, and filter for references that matched the content on the different types of ids.Prod Support

[0186] a) PDF Translations:

[0187] The system may translate PDFs as part of the translation service. The system can parse the PDF into HTML, translate the text, and recreate the PDF file. This may prove challenging with significantly large documents. The translation process happens in a job.Due to a serverless architecture, the processing may time out after, e.g., 15 minutes. As a result, the translation can be retried in an infinite loop.

[0188] b) Performance Consideration:

[0189] If the system encounters slow speeds while loading course content, content may be served on cloud front and SCORM commits can block the load. To address this, the system can update a content wrapper to delay SCORM reporting until after the course load.Cross Course Search

[0190] Enabling cross course search can create challenges stemming from the way course content is hosted on servers and since learner enrollment and access may be managed through the LMS 14. This can create a gap because if the content itself does not reside within the LMS 14, the LMS 14 cannot provide granular search capabilities. At the same time, a lack of enrollment and access data needed to ensure access to the content can create issues, unless the system directly integrates with the LMS 14.

[0191] If the system is configured to include LMS integration, it may compromise a goal of remaining LMS agnostic, since that would require building integrations with every LMS 14 in the market. Uses may wish to have a holistic search across all the courses they are entitled to access. As such, an “everything” library may be used to provide the one collection of content the system knows every customer and learner has access to, which means the system can deliver cross-course search without access conflicts. That said, the same48CPST Doc: 1410-2598-6841.1challenge would remain for custom or client specific courses, where verifying access still depends on LMS integration. One of the issues that may arise in a cross course search is the amount of data needed to search text, closed captions PDFs etc. Even with vectors, there is an upper limit to the number of files that can be included per vector. One way to address this is to not search for closed captions since a cross course search is a ‘high level’ search and then once the user is in the course they can further do a deep dive search.Knowledge Banks

[0192] With the advancements in Al, the future of learning could look significantly different where traditional courses may eventually give way to Al driven search and on demand answers. Learners may expect information at their fingertips. The knowledge banks 300 describe herein can be developed with three groups in mind: authors, learners, and managers.

[0193] For authors, it is an attempt to simplify course creation. Instead of having to design slides, outlines, and structured modules, an author could simply upload a wide range of assets including Powerpoints, Word documents, snippets, audio, video, sharepoint, box.com etc into a single knowledge bank 300. Al would then process the materials and assess learners against the topics and concepts embedded in the bank, eliminating the need for manual creation of a course.

[0194] For learners, the bank 300 would reorganize training into a streamlined, proficiency based approach. Content within the bank 300 could be grouped into three levels (Level 1 , Level 2, Level 3). Learners would take a proficiency exam for each level, which would generate a personalized Custom Learning Path (CLP). Their objective would be to earn a “green light” across all three levels, with the threshold set at, say, 80% pass mark for green. Rather than sitting through a 40-hour course, a learner could take the exam, complete the targeted CLP, retake the exam, and move forward once proficiency is demonstrated, saving significant time while focusing only on areas where improvement is needed.

[0195] For managers, the knowledge bank 300 simplifies readiness checks. Instead of going through transcripts, course completions, and scores across dozens or even hundreds of courses, a manager would only need to see whether the learner has earned three green lights in the relevant bank. If they have, that learner is considered project ready. In this way, training shifts away from a focus on transcripts and scores toward a more meaningful measure of project readiness while still allowing hours and other metrics to be tracked. This49CPST Doc: 1410-2598-6841.1is what may be described herein as a skills matrix where it shows the manager an overview of whether the learner has proven their skill at all three levels in the required banks and gotten the green light to work on the project.

[0196] One challenge is maintaining LMS agnostic behavior while ensuring secure access control. Many customers may want to upload confidential or proprietary data into their banks 300, which means strict access controls may be needed. The system should consider whether the banks continue to be hosted within an LMS 14, as that is the familiar infrastructure for most customers.General

[0197] For simplicity and clarity of illustration, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth in order to provide a thorough understanding of the examples described herein. However, it will be understood by those of ordinary skill in the art that the examples described herein may be practiced without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the examples described herein. Also, the description is not to be considered as limiting the scope of the examples described herein.

[0198] It will be appreciated that the examples and corresponding diagrams used herein are for illustrative purposes only. Different configurations and terminology can be used without departing from the principles expressed herein. For instance, components and modules can be added, deleted, modified, or arranged with differing connections without departing from these principles.

[0199] It will also be appreciated that any module or component exemplified herein that executes instructions may include or otherwise have access to computer readable media such as transitory or non-transitory storage media, computer storage media, or data storage devices (removable and / or non-removable) such as, for example, magnetic disks, optical disks, or tape. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. Examples of computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transitory computer readable medium which can be used50CPST Doc: 1410-2598-6841.1to store the desired information and which can be accessed by an application, module, or both. Any such computer storage media may be part of the computing environment 10, any component of or related thereto, etc., or accessible or connectable thereto. Any application or module herein described may be implemented using computer readable / executable instructions that may be stored or otherwise held by such computer readable media.

[0200] The steps or operations in the flow charts and diagrams described herein are provided by way of example. There may be many variations to these steps or operations without departing from the principles discussed above. For instance, the steps may be performed in a differing order, or steps may be added, deleted, or modified.

[0201] Although the above principles have been described with reference to certain specific examples, various modifications thereof will be apparent to those skilled in the art as having regard to the appended claims in view of the specification as a whole.51CPST Doc: 1410-2598-6841.1

Claims

Claims:1 . A computer-implemented method of generating models to be used in automatically generating electronic content, comprising: obtaining content from existing sources; training a subject matter specific model using the content; and providing access to the subject matter specific model to leverage the subject matter knowledge an expertise trained into the model to generate a course.

2. A computer-implemented method of generating models to be used in automatically generating electronic content, comprising: enabling local content to be placed into a repository; having a training engine access the content in the repository to train a client specific model; storing the client specific model; and enabling the client specific model to be used internally by an organization to generate a course.

3. The method of claim 2, further comprising enabling the client specific model to be licensed.

4. The method of claim 3, further comprising: providing access to the client specific model in a marketplace; receiving a request to license use of the client specific model for course generation; processing the request to determine agreed upon license terms; and providing a copy or access to the client specific model according to the license terms.

5. A computer-implemented method of generating learning paths, comprising: providing an initial skills assessment quiz or test; using one or more artificial intelligence techniques to analyze the results of the quiz or test to prepare a custom learning path that focuses on skills or knowledge determined to be lacking; and providing the custom learning path while reassessing the path as components of the learning path are provided to a user.52CPST Doc: 1410-2598-6841.

16. A computer-implemented method of scheduling course content, comprising: obtaining a schedule and preferences associated with a learner; comparing the schedule and preferences to a suggested learning path; dividing the learning path into a plurality of components; determining a schedule for the components and adding to a calendar; tracking appointments for the components as they are met or missed and adjust scheduling; and providing notifications according to the schedule and obtaining feedback to adjust the schedule prior to a scheduled component.

7. A computer-implemented method of generating a continuous learning pathway for electronic content, comprising: enabling content to be added to a knowledge bank from one or more sources; aggregating the content and separating the content into a plurality of levels; for each level, using an Al engine to generate a proficiency challenge; and enabling the proficiencies challenges to be used to assess knowledge of corresponding content.

8. The method of claim 7, further comprising: displaying a progress towards an acceptable level of knowledge for each level.

9. A computer readable medium comprising computer-executable instructions for generating electronic content, comprising instructions that, when executed by a processor of a computing system, cause the computing system to perform the method of any one of claims 1 to 8.

10. A computing system for generating electronic content, the system comprising at least one processor and at least one memory, the memory storing computer executable instructions that, when executed by the at least one processor, cause the computing system to perform the method of any one of claims 1 to 8.53CPST Doc: 1410-2598-6841.1

Images