A software framework for tracking progress in a classroom environment

Abstract

To provide a method for instructors to track whether students actually read through the entire article.SOLUTION: In a handout service, a handout comprises a placeholder for hand-in and / or an attachment file that specifies an activity to be performed by a student as part of assignment. The activity is performed using a third-party application implementing a software framework that causes the application to generate progress tracking information. A daemon and / or client application interfaces with one or more network services to make the progress tracking information available to an instructor. The network services comprise the handout service that is used to generate the handout assigned to groups of students. The network services also comprise a progress pipeline that includes several services for processing the progress tracking information received from a plurality of client devices.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The described embodiments relate generally to a software framework configured to implement various technologies in a classroom environment. In particular, the embodiments relate to tracking and reporting information related to the progress of completing online assignments for multiple students. [Background technology]

[0002] Since the invention of audiovisual (AV) devices such as television, projection systems, and tape decks, school districts and teachers have utilized these devices to incorporate instructional materials into lesson plans that present information to students in new and engaging ways. Teachers are no longer limited to simply textbooks, lectures, and written assignments. With the advent of the information age, the Internet has opened up new dimensions for the classroom. Teachers can now draw from nearly limitless information resources to create engaging, productive, and interactive lesson plans.

[0003] However, the tools available to instructors when developing these lesson plans are limited. For example, Wikipedia® articles may provide useful background material on a particular subject. Nevertheless, instructors may not have good options for sharing articles with students. Instructors can print articles on paper handouts, which is a traditional method of distributing such materials to students. However, this method is expensive and loses some of the article's interactive elements (e.g., animations, hyperlinks, etc.) as presented online. Alternatively, instructors can email the article's hyperlink to each of their students to view on their home or school-provided computing devices. While such methods allow students to view the article in the appropriate context and interact with the interactive elements embedded in the article, there is no way for instructors to track whether each student clicked on the hyperlink, or even whether each student actually read the entire article. Summary of the Invention

[0004] This document describes various embodiments of a client-server architecture that enables progress tracking associated with instructor-generated assignments. Handouts may include placeholders for hand-ins and / or attachments that specify activities to be performed by students as part of an assignment. Some activities may be performed using a third-party application that implements at least a portion of a software framework that causes the application to generate progress tracking information that is provided to a daemon running in the background of the client device. The daemon and / or client application interface with one or more network services to make the progress tracking information available to the instructor. The network services may include a handout service that is utilized to generate handouts assigned to groups of students. The network services may also include a progress pipeline that includes several services configured to process progress tracking information received from multiple client devices.

[0005] In some embodiments, the client device includes a processor, memory, and a display. The memory can include an operating system, a client application, a daemon, and one or more third-party applications, including at least one progress application that implements at least a portion of a software framework. The software framework enables the progress application to generate progress tracking information associated with one or more contexts that the progress application provides to the client application and / or the daemon. The contexts generally define the structure of at least a portion of the progress application such that the client application can define activities that may be completed by a student within the progress application.

[0006] In some embodiments, the daemon is configured to run as a background process and receive progress tracking information from one or more applications, filter the progress tracking information, store the progress tracking information in a remote database, generate metadata associated with the progress tracking information, and transmit the metadata to at least one service. The daemon can also implement a synchronization mechanism that aligns versions of resources stored locally on the client device with versions stored in the remote database. The synchronization mechanism enables resources to be accessible on two or more client devices.

[0007] In some embodiments, the software framework causes progress applications to generate API calls that include progress tracking information and are sent to a daemon. The daemon may be configured to filter the progress tracking information based on whether progress tracking is enabled or disabled for one or more progress applications. Alternatively, the daemon may be configured to filter the progress tracking information based on a list of active contexts.

[0008] In some embodiments, the remote database is a network-based storage service divided into multiple zones that store different data, and the scope of each zone is limited to any combination of at least one of an organization identifier, a class identifier, a user identifier, a context identifier, or a handout identifier. Metadata corresponding to the progress tracking information is generated by a daemon and sent to the progress pipeline. The metadata includes a reference to the progress tracking information in the remote database and at least one of a class identifier, a handout identifier, a user identifier, or a context identifier.

[0009] In some embodiments, the progress pipeline includes several services structured into a pipeline architecture. The progress pipeline can include an on-ramp service configured to receive metadata corresponding to progress tracking information, de-reference a user identifier included in the metadata by replacing the user identifier with a directory service identifier or an internal identifier, determine whether progress tracking is enabled or disabled for one of the user identifier, the directory service identifier, or the internal identifier, and filter the metadata based on the determination. The progress pipeline can include an enrichment service configured to enrich the metadata with at least one of an organization identifier, a directory service identifier, a user identifier, or a handout identifier. The progress pipeline can include an aggregation service configured to retrieve a subset of the metadata corresponding to the tracking window from a distributed file system including at least one journal node, generate a data structure including the subset of metadata associated with a particular organization identifier included in the subset of metadata, store the data structure in a memory, and push a reference corresponding to the data structure to a queue. The progress pipeline can include an off-ramp service configured to process the data structure to generate sorted metadata based on at least one of a class identifier, a handout identifier, a context identifier, a user identifier, a directory service identifier, or an internal identifier. The progress pipeline can include a storage service configured to associate progress tracking information with a progress identifier associated with one of the user identifier or the directory service identifier, and store the progress tracking information and the progress identifier in a network-based storage service.

[0010] The client-server architecture and / or software framework, individually or in combination, may enable teachers and students to interact using interactive lesson plans that incorporate functionality from one or more third party applications to enhance the learning experience. Other aspects and advantages of the present invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of the described embodiments.

[0011] This Summary is provided merely for the purpose of summarizing some example embodiments so as to provide a basic understanding of some aspects of the subject matter described herein. Accordingly, it should be understood that the above-described features are examples only and should not be construed as narrowing the scope and spirit of the subject matter described herein in any way. Other features, aspects, and advantages of the subject matter described herein will become apparent from the following detailed description, drawings, and claims. [Brief explanation of the drawings]

[0012] The present disclosure will be readily understood by the following detailed description in conjunction with the accompanying drawings, in which like reference numerals indicate like structural elements, and in which:

[0013] [Figure 1] 1 illustrates a network environment designed to operate within a classroom environment, according to some embodiments.

[0014] [Figure 2] 1 illustrates a client-server architecture implemented to enable progress tracking of client devices within a classroom environment, according to some embodiments.

[0015] [Figure 3] 1 illustrates a graphical user interface of a client application, according to some embodiments.

[0016] [Figure 4]1 illustrates a client-server relationship between a handout service and a client application and / or daemon, according to some embodiments.

[0017] [Figure 5] 1 illustrates a synchronization mechanism implemented by a daemon, according to some embodiments.

[0018] [Figure 6A] 1 illustrates a graphical user interface of a client application, according to some embodiments. [Figure 6B] 1 illustrates a graphical user interface of a client application, according to some embodiments. [Figure 6C] 1 illustrates a graphical user interface of a client application, according to some embodiments. [Figure 6D] 1 illustrates a graphical user interface of a client application, according to some embodiments. [Figure 6E] 1 illustrates a graphical user interface of a client application, according to some embodiments.

[0019] [Figure 7] 1 illustrates a client-server relationship between a handout service and several client application(s) and / or daemon(s) associated with multiple students in a class, according to some embodiments.

[0020] [Figure 8A] 1 illustrates a graphical user interface implemented by a client application on a client device associated with a student, according to some embodiments. [Figure 8B] 1 illustrates a graphical user interface implemented by a client application on a client device associated with a student, according to some embodiments. [Figure 8C] 1 illustrates a graphical user interface implemented by a client application on a client device associated with a student, according to some embodiments.

[0021] [Figure 9] 1 illustrates a client-server relationship between a hand-in service and several client application(s) and / or daemon(s) of several students in a class, according to some embodiments.

[0022] [Figure 10] 1 illustrates a client-server relationship between a progress pipeline and client applications and / or daemons, according to some embodiments.

[0023] [Figure 11] 10 illustrates filtering of progress tracking information by a daemon, according to some embodiments.

[0024] [Figure 12] 1 illustrates a progress pipeline according to some embodiments.

[0025] [Figure 13] 1 illustrates a client-server relationship between a reporting service and a client application and / or daemon, according to some embodiments.

[0026] [Figure 14A] 1 illustrates a graphical user interface of a client application, according to some embodiments. [Figure 14B] 1 illustrates a graphical user interface of a client application, according to some embodiments.

[0027] [Figure 15]1 is a flowchart of a method for tracking student activity on a client device according to some embodiments.

[0028] [Figure 16] 1 is a flowchart of a method for processing progress tracking information using one or more services available over a network, according to some embodiments.

[0029] [Figure 17] FIG. 1 shows a detailed diagram of an exemplary computing device that may be used to implement various apparatus and / or methods described herein, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0030] Representative examples of applications of the methods and apparatus according to the present application are described in this section. These examples are provided solely to add context and to aid in understanding the described embodiments. Thus, it will be apparent to one of ordinary skill in the art that the described embodiments can be practiced without some or all of these specific details. In other instances, well-known process steps have not been described in detail to avoid unnecessarily obscuring the described embodiments. Other applications are possible, and therefore the following examples should not be construed as limiting.

[0031] In the following detailed description, reference is made to the accompanying drawings which form a part of the description, and in which is shown by way of illustration specific embodiments in accordance with the described embodiments. These embodiments are described in sufficient detail to enable those skilled in the art to practice the described embodiments, but these examples are not to be construed as limiting, and therefore other embodiments may be used, and changes may be made without departing from the spirit and scope of the described embodiments.

[0032] The digital classroom offers students new opportunities to learn within a rich environment of interactive applications installed on student-provided computing devices. While devices may allow students to interact online with information accessed through a browser or provided through specialized applications, the benefits of these activities are typically lost due to the difficult nature of tracking these activities. As a result, new techniques for incorporating these educational materials into a unified classroom architecture are desirable.

[0033] A classroom environment can provide several devices and applications to instructors and students, which allow the instructor to assign student activities using these applications that can be easily monitored by the instructor. The instructor creates handouts to assign to students that incorporate activities defined by contexts provided by one or more third-party applications implementing a software framework. The third-party applications are configured to track students' progress completing the activities associated with these contexts and provide the progress tracking information to a background process on the device. The background process and / or client application can interface with one or more network services designed to process the progress tracking information. The processed progress tracking information can be queried to generate reporting data for display to the instructor showing the progress of each student assigned the handout.

[0034] These and other embodiments are discussed below with reference to Figures 1-17. Those skilled in the art will readily appreciate that the detailed description provided herein with respect to those figures is for purposes of illustration only and should not be considered limiting.

[0035] FIG. 1 illustrates a network environment 100 designed to operate within a classroom environment, according to some embodiments. Network environment 100 includes several separate and distinct computing devices configured to communicate over a network 150. Network environment 100 includes a server device 110 and client devices 120. As shown in FIG. 1, client devices 120 may include, but are not limited to, laptop computers 120-1 and 120-5, tablet computers 120-2, 120-3, and 120-4, and a mobile device 120-6. It will be understood that network environment 100 may include other types of client devices 120, such as desktop computers, wearable devices (e.g., smart watches), personal digital assistants, etc.

[0036] The following terms may be used throughout this document: Class refers to a group of individuals who meet regularly to be taught a particular subject or activity. A class can include one or more teachers and a number of students enrolled in the class. Classroom environment refers to a group of related devices utilized by a group of users associated with a class. School refers to a managed collection of several classes taught by a group of teachers. School district refers to an organization that manages one or more schools.

[0037] In some embodiments, a teacher distributes educational materials to students in a class using one or more devices. As shown in FIG. 1 , a teacher can create educational materials for students using a laptop computer, such as client device 120-1, or a tablet computer, such as client device 120-2. In some embodiments, the laptop computer and / or tablet computer include an application configured to create interactive assignments for students. The assignments can include links to educational materials available over network 150, such as hyperlinks to websites available from first server device 110-1 or documents stored on second server device 110-2. The application can also enable the teacher to easily distribute assignments to students. Students can access assignments and view the educational materials included in each assignment using tablet computers, such as client devices 120-3 and 120-4, laptop computers, such as client device 120-5, and / or mobile devices, such as client device 120-6.

[0038] In some embodiments, the third server device 110-3 hosts one or more services utilized by applications included on the various client devices 120. For example, the third server device 110-3 may host a service that uploads data structures or files defining assignments to a network-based storage resource. As another example, the third server device 110-3 may host a service that tracks which client devices 120 are enrolled in a particular classroom environment. As yet another example, the third server device 110-3 may host a service that tracks each student's progress on a particular assignment. In some embodiments, services utilized by the application(s) may be hosted on more than one server device 110. In addition, the services may be hosted by different service providers. For example, a first server device 110-2 maintained by a first service provider may be configured to host a network-based storage service, and a second server device 110-2 maintained by a second service provider may be configured to host a school management service.

[0039] Set forth below is a description of the software framework of the applications executed by the various client devices 120 in the classroom environment. Additionally, set forth below is a description of the various services utilized by the applications described herein and hosted on one or more server devices 110.

[0040] 2 illustrates a client-server architecture 200 implemented to enable progress tracking of client devices 120 within a classroom environment, according to some embodiments. Client-server architecture 200 includes a client application 205 implemented on client device 120 and configured to interact with several services implemented on server device 110 accessible via network 150. Client-server architecture 200 enables instructors to create assignments that include rich interactive material provided by one or more third-party applications installed on client device 120, as well as integrate seamless progress tracking so that instructors can monitor students' efforts in keeping up with class assignments.

[0041] In some embodiments, client application 205 is a binary executable configured to be executed by a processor included in client device 120. The binary executable may be designed to run within a particular operating environment, such as a target hardware platform designed to run a target operating system. As a result, client application 205 may be implemented using multiple versions of source code compiled for different operating environments. For example, a first version of client application 205 may be designed for laptop computer 120-1, and a second version of client application 205 may be designed for tablet computer 120-2.

[0042] In some embodiments, client application 205 is designed to connect with one or more services over network 150. The services may include, but are not limited to, handout service 220, handin service 230, school administration service 240, progress pipeline 250, identification service 260, and reporting service 270. Client application 205 may be configured to access the services through an application programming interface (API) implemented by the service. For example, a web-based service may implement an API using Simple Object Access Protocol (SOAP) or using the REpresentational State Transfer protocol (REST), which allows client application 205 to access and / or operate on resources associated with the service. The API may allow client application 205 to send data to and / or receive data from the service. API calls may also cause the service to perform various operations and / or invoke additional services using additional API calls.

[0043] In other embodiments, client application 205 is a web-based application that is displayed within a browser on client device 120. Client application 205 may include a set of resources (e.g., Hypertext Markup Language (HTML) documents, images, scripts, etc.) that are requested from a server associated with a website. The browser receives the resources, interprets the resources, and displays a representation of the website on the display of client device 120. Thus, client application 205 is platform-independent in that client application 205 can be displayed on multiple different client devices 120 running different operating systems.

[0044] In some embodiments, client application 205 interfaces with one or more services through API gateway 210. API gateway 210 is implemented by server device 110 and redirects API calls received from client device 120 to various services within client-server architecture 200. In other embodiments, API gateway 210 is omitted or selectively bypassed, and client device 120 sends API calls directly to the services.

[0045] In some embodiments, the client application 205 enables instructors to create handouts for their classes. A handout refers to a file or data structure containing information related to an assignment intended to be made available to students in a class. A handout may contain several fields, including a handout identifier, a title of the assignment, a body of the assignment including text-based instructions for students on how to complete the assignment, a due date for the assignment, and a list of attachments associated with the assignment. Handouts may be created and stored locally on a particular client device 120. Attachments may include files (e.g., documents, images, videos, etc.), placeholders for files that students are to submit, and activities that students are to complete as part of the assignment. Activities are performed using a third-party application that implements a portion of the ClassKit software framework. Examples of activities may include, but are not limited to, reading a chapter from a digital book or textbook, taking a quiz or answering a problem set, and tracking time spent performing a task such as playing an interactive game or conducting a digital experiment.

[0046] In some embodiments, the client-server architecture 200 includes a handout service 220. The handout service 220 is configured to manage handouts. In some embodiments, the handout service 220 is configured to synchronize handouts created on one client device 120 with another client device 120 to allow instructors to work on multiple devices. In some embodiments, the handout service 220 is configured to allow handouts to be published to a list of students. Publishing a handout refers to making the information in the handout available for viewing by students on separate client devices 120 and handling various back-end operations related to handout attachments.

[0047] In some embodiments, the client-server architecture 200 can also include a hand-in service 230. A hand-in refers to a placeholder for a file or data structure, indicating that a student is instructed to create and submit the file or data structure to complete an assignment. A student can create a file or data structure and submit the file or data structure to the hand-in service 230 to fulfill the requirements for completing an assignment.

[0048] In some embodiments, the client-server architecture 200 may also include a school administration service 240 configured to manage administrative information for the school district related to the structure of classes. For example, the school administration service 240 may maintain records indicating which teachers are assigned to each of a plurality of classes. The records may also indicate which students are enrolled in each of a plurality of classes. Each teacher or student may be assigned a teacher identifier or student identifier, respectively. Each class created by the school district may be associated with a class identifier. A separate record may then be created for each class identifier that lists a roster of student identifiers for students enrolled in the class. In some embodiments, a relational database associates teacher identifiers and / or student identifiers with class identifiers in one or more tables. The relational database may be queried using Structured Query Language (SQL) or some other type of query language to return information identifying the structure of various classes.

[0049] In some embodiments, school administration services 240 include an administrative interface that allows a school district administrator to create classes and specify students to be enrolled in the classes. The administrative interface can be, for example, a web-based interface that requires the administrator to provide credentials to modify each class's roster. In other embodiments, school administration services 240 include an interface for downloading data from a separate and distinct school information system that the school district maintains separately from client-server architecture 200. Classes and rosters can be downloaded automatically from the school information system.

[0050] In some embodiments, the client-server architecture 200 also includes a progress pipeline 250. The progress pipeline 250 provides a secure architecture for managing information related to progress tracking as students complete assignments set forth in handouts published by instructors. The student's client device 120 may include a background process (i.e., a daemon) configured to monitor activity associated with one or more third-party applications installed on the client device 120. The daemon tracks progress made by students in completing assignments specified in one or more published handouts and sends information related to each student's progress to the progress pipeline 250. The progress pipeline 250 aggregates and stores the progress information, allowing an instructor to view comprehensive reports for one or more students in a class.

[0051] In some embodiments, the progress pipeline 250 is implemented as several separate services running on different server devices 110 and is structured to process progress information in a pipelined data flow. In other embodiments, the progress pipeline 250 may be implemented by several services running on a single server device 110.

[0052] In some embodiments, client-server architecture 200 may also include an identity service 260. Identification service 260 allows data related to particular client devices 120 to be associated with particular people (e.g., teachers or students) within a classroom environment. In some embodiments, identity service 260 associates various user or device identifiers with directory service identifiers that are scoped to the server side of client-server architecture 200. Use of identity service 260 protects data integrity by isolating personally identifiable information collected by the daemon from any identifiers utilized outside a particular scope (e.g., a classroom environment). Thus, data such as a user's test scores cannot be linked to other information outside that scope, such as what type of music a student prefers to listen to on a music sharing service offered by the same service provider. The integrity of such personally identifiable information may be particularly important given that most students are minors and there may be special considerations to take into account when protecting the data of vulnerable users of a certain age.

[0053] In some embodiments, a directory service identifier is a static identifier associated with a user of one or more devices by a particular service provider. A directory service identifier may be associated with a user identifier used by a client device to associate a user with a particular user account. A directory service identifier may be associated with an external identifier associated with a user of a particular user account and published to the client device. A directory service identifier may be associated with an internal identifier of a particular user. The internal identifier may be disassociated from the directory service identifier, and a new internal identifier may then be associated with the directory service identifier. In some embodiments, replacing an internal identifier with a new internal identifier may be used to sever an association between particular stored data and a particular user or user account without having to delete these data from one or more server devices 110.

[0054] In some embodiments, the client-server architecture 200 may also include a reporting service 270 that enables aggregate data related to the progress of activities completed by several students to be compiled and provided to the client device 120 for review by an instructor. The reporting service 270 may also analyze progress information received from multiple students to generate statistical data related to the progress information. For example, the reporting service 270 may generate statistics related to how many students completed a particular activity. The reporting service 270 may also track results related to the activity (e.g., the number of questions answered correctly in a quiz activity, the time taken to complete a quiz, etc.). In other embodiments, the client application 205 receives individual student reporting data from the reporting service 270, and the client application 205 aggregates the reporting data and / or calculates class statistics.

[0055] It will be understood that client application 205 may be configured to interact with some or all of the described services of client-server architecture 200. For example, client application 205 may be configured to generate calls to handout service 220, handin service 230, progress pipeline 250, and reporting service 270. In some embodiments, client application 205 is not configured to interact with a subset of services, such as school administration service 240 and / or identification service 260. Instead, other services may generate calls to these services in response to receiving a call from client application 205.

[0056] 3 shows a GUI 300 of a client application 205, according to some embodiments. The GUI 300 may be presented to an instructor on a display 302 of a client device 120. As shown in FIG. 3, the GUI 300 is displayed on a tablet computer, such as client device 120-2. The GUI 300 may include several elements, including graphics, text, icons, shapes, input fields, output fields, display areas, etc.

[0057] In some embodiments, GUI 300 includes a first display area 310, a second display area 320, and a third display area 330. First display area 310 is utilized to display a representation of a handout previously created by an instructor. Second display area 320 is utilized to display a representation of a class associated with a particular instructor. As shown in FIG. 3 , first display area 310 occupies a first portion of the pixels contained in display 302, and second display area 320 occupies a second portion of the pixels contained in display 302.

[0058] The first display area 310 includes a header showing elements 312 representing separate and distinct handouts associated with the user account. Handouts may be created by accounts associated with teachers, administrators, or other roles. In some embodiments, each element 312 is defined by a box that includes the title of the handout, a descriptor of the due date associated with the handout, and an indication of how many students have completed the handout. For example, the first element 312-1 is associated with a first handout for the assignment "Flora and Fauna," which is due March 12 and has been completed by all students; the second element 312-2 is associated with a second handout for the assignment "Test Prep," which is due March 15 and has been completed by only seven of the 22 students assigned the handout. It will be understood that in other embodiments, the elements 312 may take different forms. For example, the element 312 may omit the due date and / or completion indicator. Alternatively, element 312 can include information in addition to or instead of a due date and / or completion indicator. Additionally, element 312 can include graphics, such as pictures or icons, that represent different activities assigned to the handout. In some cases, the graphics can be animated, such as by playing a video as the background of element 312 or showing a slideshow of images attached to the handout.

[0059] The second display area 320 includes a header showing elements 322 representing separate and distinct classes associated with a particular instructor's user account. In some embodiments, each element 322 is defined by a box containing the title of the class and a descriptor of the number of handouts assigned to the class. For example, the first element 322-1 is associated with the class "Science A," which has two handouts assigned to it; the second element 322-2 is associated with the class "Math A," which has three handouts assigned to it; and the third element 322-3 is associated with the class "Math B," which has no handouts assigned to it (zero handouts). It will be appreciated that in other embodiments, the elements 322 may take different forms. For example, the element 322 may omit an indicator of the number of handouts assigned to the class. Alternatively, the element 322 may include information in addition to or instead of the class title and an indicator of the number of handouts assigned to the class. For example, the element 322 may include the number of students enrolled in the class. Additionally, elements 322 may include graphics, such as images or background colors coded to indicate particular subjects or subjects associated with the class.

[0060] In some embodiments, the instructor can select a particular class (or classes) using the input interface of the client device 120. For example, the instructor can use a touch interface to select an element 322 associated with one of the available classes associated with that instructor. Upon selecting a particular element 322, the selected element 322 is activated, and the element 312 in the first display area 310 may be updated to reflect the handout associated with the class represented by the selected element 322 in the second display area 320. The selected element 322 (e.g., the currently activated element) may be highlighted to indicate that the element 322 is active. The highlighting may be implemented by increasing the size of the element, changing the color associated with the element 322, changing the color or thickness of the border of the element 322, displaying a box around the selected element 322, etc. As shown in FIG. 3 , the first element 322-1 is activated and is shown larger than the second element 322-2 and the third element 322-3.

[0061] It will be appreciated that in some embodiments, there may not be enough room in the first display area 310 and / or the second display area 320 to present all classes and / or all handouts assigned to a particular class within the confines of the display 302. In such cases, a user of the client device 205 can use an input command, such as a touch gesture, to display additional elements. For example, an icon can be displayed indicating that some elements are not being displayed. The additional elements are displayed in the first display area 310 or the second display area 320 in response to the user selecting the icon. In another example, a user can use a swipe touch gesture to scroll horizontally to display additional elements, while other elements are moved within or removed from a particular display area.

[0062] Third display area 330 is located near the top edge of display 302. Third display area 330 includes elements that allow a user of client application 205 to sign in to a user account and create new handouts of a particular class or edit handouts that were previously created and stored locally on client device 120 and / or remotely on server device 110. In some embodiments, third display area 330 is hidden and may be displayed in response to a user input, such as, for example, using a vertical swipe touch gesture to reveal third display area 330.

[0063] A first icon 332 is displayed near the left side of the third display area 330. When selected by a user, the first icon 332 can cause the client application 205 to prompt the user to enter credentials to sign in to a user account associated with the faculty member. In some embodiments, the first display area 310 and the second display area 320 are hidden until the user signs in to a particular user account using the first icon 332. In some embodiments, when the client application 205 launches, a separate interface is displayed to the user that prompts the user to enter credentials to sign in to the user account. The GUI 300 is displayed on the display 302 only when the user has successfully signed in to a particular user account. The first icon 332 can then be used to sign out of the user account and / or to sign in to a different user account.

[0064] A second icon 334 is displayed near the right side of the third display area 330. When selected by a user, the second icon 334 can cause the client application 205 to create a file or data structure in the file system of the client device 120 to store information related to the new handout. In some embodiments, the new file or data structure is automatically associated with the active class selected in the second display area 320. The instructor can then add information to the file or data structure using a different GUI, which will be described in more detail below with reference to Figures 6A-6E.

[0065] A third icon 336 is displayed near the right side of the third display area 330. When selected by a user, the third icon 336 can cause the client application 205 to retrieve information stored in a file or data structure of a previously generated handout. In some embodiments, one of the elements 312 in the first display area 310 is activated either automatically based on selection criteria or manually in response to input provided by the user. For example, when an element 322 representing a particular class is activated, the element 312 may be automatically selected for activation based on the smallest difference between the due date of the handout and the current date among all handouts stored for that class. Alternatively, the element 312 may be manually selected for activation in response to a touch gesture input (e.g., a tap input) on an area of ​​the display 302 corresponding to the particular element 312. Although not explicitly shown in FIG. 3 , a selected element 312 in the first display area 310 (e.g., the currently active element) may be highlighted to indicate that the selected element 312 is active. In response to selection of the third icon 336, a different GUI, described in more detail below with reference to Figures 6A-6E, may be displayed by the client device 120 and populated with data from a file or data structure associated with the active element 312.

[0066] In some embodiments, the first display area 310 includes representations of handouts in preparation and handouts that have been published to a list of students. In some cases, the element 312 may include an indication of whether the handout represented by the element 312 has been published or has not yet been published (i.e., is still being prepared by the instructor).

[0067] It will be understood that GUI 300 described above and illustrated in FIG. 3 is merely an exemplary embodiment of a GUI designed for client application 205. In other embodiments, the GUI of client application 205 may take other forms. The look and feel of the GUI may also correspond to standard display elements associated with a particular operating system or target environment. In still other embodiments, client application 205 is a web-based application displayed in a browser on client device 120. In such embodiments, GUI 300 may include multiple Hypertext Markup Language (HTML) elements contained in one or more HTML documents. The representation of such elements may depend on the implementation of the browser application on client device 120 and, in some cases, on user preferences associated with the browser application.

[0068] 4 illustrates a client-server relationship between the handout service 220 and the client application 205 and / or daemon 440, according to some embodiments. The handout service 220 provides a means for instructors to create and assign handouts to a list of students enrolled in classes associated with the instructor. As described above, the client application 205 is configured to prompt the user for credentials associated with a user account. If the user account is associated with the instructor, the client application 205 enables the instructor to create handout files or data structures in the memory 420 of the client device 120. In some embodiments, these files or data structures can be synchronized to a network-based storage service, thereby allowing the instructor to access information related to the handouts from multiple client devices 120.

[0069] 4, client device 120 includes a processor 410, a display 302, and a memory 420. Memory 420 includes an operating system 430, a client application 205, a daemon 440, and a local database 445. Daemon 440 is a background process that implements an API that allows client application 205 to asynchronously communicate with daemon 440 to perform specific tasks. One of those tasks operates to create a file in local database 445.

[0070] In some embodiments, the daemon 440 maintains a local database 450. In some embodiments, an instructor can create a new file or data structure for a handout using a client application 205. The client application 205 requests the daemon 440 to create a file or data structure for the handout in a local database 450 maintained by the daemon 440. An instructor can populate the file or data structure for the handout using a GUI 600, which is discussed in more detail below with reference to Figures 6A-6E.

[0071] In some embodiments, daemon 440 is configured to synchronize files or data structures of local database 450 with remote database 460. Remote database 460 may be a distributed database, such as a cloud-based database service, implemented on several server devices 110. Daemon 440 interfaces with remote database 460 to synchronize files or data structures of local database 450 with remote versions of the files or data structures so that the files or data structures may be accessed and / or modified using multiple client devices 120. The synchronization mechanism is described in more detail below with reference to FIG. 5.

[0072] Once the handout has been synchronized to the remote database 460, the instructor can then switch to a different client device 120 using a different version of the client application 205 and use the new client device 120 to access the version of the handout stored on the remote server 110. Any changes to the local copy of the resource stored on the new client device 120 are synchronized to the remote database 460 to update the version of the handout on the remote database. These changes are then pushed to the client application 205 on the previous client device 120 to update the copy of the local resource in the local database 450 on the initial client device 120.

[0073] For example, a faculty member may open client application 205 on laptop computer 120-1 and create a local version of a handout that is stored in the memory of laptop computer 120-1. Daemon 440 on laptop computer 120-1 sends a copy of the local version of the handout to remote database 460, synchronizing the handout with the version of the handout stored in remote database 460. The faculty member then opens a separate client application 205 on tablet computer 120-2 and provides credentials for the faculty member's user account. By opening client application 205 on tablet computer 120-2, daemon 440 downloads the remote version of the handout stored in remote database 460 into the memory of tablet computer 120-2, whereupon client application 205 on tablet computer 120-2 can access and modify the local copy of the handout in the memory of tablet computer 120-2. Any modifications to the local copy of the handout can be synchronized to remote database 460 so that the version of the handout stored in remote database 460 is up to date. When the instructor then opens client application 205 on laptop computer 120-1, client application 205 requests from daemon 440 the current version of the handout from remote database 460, which sends the version of the handout stored in remote database 460 to daemon 440 on laptop computer 120-1 for storage in laptop computer 120-1's memory as the local version of the handout.

[0074] In some embodiments, the handout service 220 is implemented as a set of instructions stored in a memory of the server device 110, configured to be executed by a processor of the server device 110. The handout service 220 manages the distribution of handouts between client devices 120 associated with instructors and client devices 120 associated with students. In some embodiments, the handout service 220 implements the distribution of handouts using a remote database 460. The handout service 220 creates zones 465 in the remote database 460. Each zone 465 is scoped to a particular class. Alternatively, each zone 465 is scoped to a group of students. Each class associated with an instructor may be associated with a different group of students.

[0075] A teacher can create a handout in the client device's local database 450. The handout is synchronized to a personal zone 465-1 for the teacher in the remote database 460. Once a particular handout is prepared and ready to be assigned to a group of students, the teacher can use the client application 205 to publish the handout and assign it to the group of students. The client application 205 requests the handout service 220 to publish the handout. The handout service 220 then copies a copy of the handout to a new zone 465-2 in the remote database 460 that is scoped to the group of students specified in the handout. Each student within the new zone 465-2 subscribes to the new zone 465-2 so that the daemon 440 on the client device 120 controlled by each student in the student group synchronizes a copy of the published handout to the local memory of that student's client device 120.

[0076] It will be appreciated that the handout service 220 also generates a handout identifier that can be utilized to identify the handout resources stored in the remote database 460. In some embodiments, the handout identifier is generated when the handout is published. In other embodiments, the handout identifier is generated when the handout is created. The handout resources are then copied to the new zone 465-2 and deleted from the instructor's personal zone 465-1 once the handout is published.

[0077] FIG. 5 illustrates a synchronization mechanism implemented by daemon 440, according to some embodiments. As shown in FIG. 5, daemon 440 is configured to interface over network 150 with database service 510, which provides a front end for remote database 460. Database service 510 may be implemented on server device 110. In some embodiments, remote database 460 is limited in scope to a particular organization or school district. Thus, all information stored in remote database 460 pertains to a particular school district. In such embodiments, database service 510 is configured to manage multiple remote databases 460 for different organizations or school districts. In various embodiments, remote database 460 is a cloud-based, distributed database service managed by a service provider.

[0078] In some embodiments, the database service 510 is configured to store files in multiple different zones 465 implemented within the remote database 460. Each teacher and / or student in the school district may be assigned a different zone 465 as a personal zone within the remote database 460. A personal zone 465 may store files or data structures associated with a particular user account. Additionally, each class created by the school district may be assigned a class zone 465 for sharing resources with all students and teachers associated with the class. In alternative embodiments, zones 465 may be created for groups of students and / or teachers, which may be a subset of a class or may be enrolled in more than one class. A class zone 465 may be used to publish handouts to students and / or other teachers associated with a particular class. A zone 465 may be scoped to an organization, a class, or an individual (e.g., a teacher or a student). Zones 465 may also be used for different data.

[0079] In some embodiments, database service 510 maintains a subscription for each zone 465 that lists the users (e.g., faculty and / or students) within each zone 465. Database service 510 can push a notification to each client device 120 of users that subscribe to zone 465 whenever a resource in zone 465 is updated. Daemon 440 on a particular client device 120 receives the notification, downloads the updated resource from zone 465, and updates its local copy of resource 522 in local database 450.

[0080] It will be appreciated that different types of zones can be created and synchronization mechanisms can be utilized to implement several functions required by one or more services of the client-server architecture 200. For example, zones can be created to store handouts, hand-ins, bookmarked context associated with third-party applications, class roster information, progress tracking information, and the like. Furthermore, each of the above zones can be scoped to a single user (i.e., as a personal zone) or to multiple users. For example, a handout zone 465 can be created to store data related to a handout (e.g., attachments, hand-in resources uploaded by students, etc.) and can be scoped to a group of students identified in the handout. A progress zone 465 can be created to store progress information for several students. A progress zone 465 can be scoped to a class or to a particular handout published to a class. A roster zone 465 can be created to store data related to an organization's class structure. A roster zone 465 can be scoped to an organization.

[0081] 6A-6E illustrate a GUI 600 of a client application 205, according to some embodiments. The GUI 600 may be used to edit information stored in a handout that an instructor is preparing to assign to a class. The GUI 600 may be presented on the display 302 of a client device 120. As shown in FIG. 6A, the GUI 600 is displayed on a tablet computer, such as client device 120-2. The GUI 600 may include several elements, including graphics, text, icons, shapes, input fields, display areas, etc. In some embodiments, the GUI 600 is displayed in response to an instructor selecting the second icon 334 or the third icon 336 of the GUI 300.

[0082] GUI 600 includes a first display area 610, a second display area 620, and a third display area 630. First display area 610 is utilized to display information related to a handout created by an instructor for a particular class or group of students. Second display area 620 is utilized to display attachments to the handout. As shown in FIG. 6 , first display area 610 occupies a first portion of pixels contained in display 302, and second display area 620 occupies a second portion of pixels contained in display 302. Third display area 630 occupies a third portion of pixels contained in display 302. In some embodiments, third display area 630 may be hidden or displayed in response to a user input, such as using a swipe touch gesture to cause third display area 630 to appear near the top edge of display 302.

[0083] 6A, a first display area 610 includes an interface for indicating a group of students to whom the handout is assigned. The interface shows a label (e.g., "To:"), a user interface element 612, and a user interface element 614. An instructor can specify a group of students to whom the handout is assigned by either typing an identifier for a student or students into user interface element 612 or by selecting user interface element 614 and selecting a group of students from a list.

[0084] In some embodiments, selecting user interface element 614 causes client application 205 to request roster data for classes associated with the instructor who signed in to client application 205. The roster data includes identifiers for all students enrolled in any classes associated with the instructor. Client application 205 can request the list of identifiers from school administration service 240. Alternatively, client application 205 can obtain the list of identifiers from local memory 420 of client device 120. For example, the roster service can periodically request roster data from school administration service 240, which stores the roster data in roster zone 465 of remote database 460. Roster zone 465 can be scoped to specific classes, and instructors can subscribe to roster zone 465 for those classes associated with that instructor. Thus, through a synchronization mechanism, daemon 440 downloads the roster data to local database 450 of client device 120, where it can be accessed by client application 205.

[0085] The first display area 610 also includes an interface that indicates a due date for students to complete the activities attached to the handout and submit any hand-ins assigned to the handout. The interface shows a label (e.g., "Due Date:") and a user interface element 616. The user interface element 616 may display a date (e.g., a period) for completing the assignment. Selecting the user interface element 616 may display a representation of a calendar that allows the instructor to select a date.

[0086] The first display area 610 also includes several text fields that can be edited by the instructor to change the title and body of the handout. For example, as shown in Figure 6A, the instructor specifies the title of the handout, "Flora & Fauna," and writes a set of instructions to be given to students on how to complete the assignment.

[0087] The second display area 620 includes a pair of user interface elements referred to as a first icon 622 and a second icon 624. The first icon 622 may be selected by the instructor to add a placeholder for a hand-in to the handout. The second icon 624 may be selected by the instructor to add an activity to the handout. Any attachments to the handout may be listed below the first icon 622 and the second icon 624. As shown in FIG. 6A , this handout currently has no placeholders or activities attached for hand-ins.

[0088] As shown in FIG. 6B , by attaching a placeholder for a hand-in to a handout, box 640 in second display area 620 is populated with several user interface elements. For example, first user interface element 642 allows a user to edit options for the hand-in, second user interface element 644 displays information related to the hand-in, third user interface element 646 allows a user to view more detailed information about the hand-in, and fourth user interface element 648 allows a user to reorder the handout's attachments or remove (e.g., unattach) the placeholder for the hand-in from the handout. In some embodiments, second user interface element 644 includes a thumbnail image of a representation of a resource for the hand-in and text located next to the thumbnail image that provides information related to the hand-in. For example, the text may indicate the title of the hand-in and the type of resource being generated for the hand-in. It will be understood that this information may be omitted or additional information may be included in place of the information described herein.

[0089] 6A, selecting the second icon 624 causes the client application 205 to display an interface including user interface elements that allow the instructor to attach activities to the handout. The types of activities may include activities completed using a third-party application, activities performed by viewing or editing an attachment, activities completed by viewing a network resource associated with a bookmark or link in a browser application, and activities completed by viewing a multimedia file (e.g., an image, a video, etc.).

[0090] As shown in FIG. 6C , in response to a user selecting a second icon 624 of the GUI 600, a box 650 can be displayed. The box 650 includes a first user interface element 652 that enables an instructor to associate application activity with a handout. For example, the third-party application can be configured to present a quiz to students and prompt the students to answer questions included in the quiz. As another example, the third-party application can provide students with an interactive lesson that includes the students following instructions to perform a task (e.g., writing source code for a program, playing a game, matching pictures to words in a foreign language, etc.). As yet another example, the third-party application can present the text of a book or textbook to students for reading one or more sections of the digital book or textbook.

[0091] Box 650 includes a second user interface element 654 that allows an instructor to attach a file to the handout. The file can include a document, a spreadsheet, a presentation (e.g., a slide deck), source code, or any other type of file that can be viewed by an application on client device 120. Attached files allow an instructor to distribute educational materials that are typically read or viewed by students. In some embodiments, an instructor can specify whether a single file is shared among all students in a class (referred to as a collaborative option) or whether clones (e.g., copies) of the file are shared with each student individually. An instructor can also specify permissions associated with the attached file, making it read-only or allowing students read-write access to edit the file.

[0092] Box 660 includes a third user interface element 666 that allows the instructor to attach a bookmark or link to the handout. A bookmark or link refers to a hyperlink that specifies a Uniform Resource Locator (URL) of a resource accessible over a network such as the Internet. The link may allow students to visit a website that may be related to the lesson.

[0093] Box 650 includes a fourth user interface element 658 that allows the instructor to attach multimedia files to the handout. The multimedia files may include photos, slideshows, videos, or other types of multimedia, such as virtual reality data or three-dimensional models. The multimedia files may then be viewed by students using an application, such as an image viewer application, installed on client device 120.

[0094] When a user selects first user interface element 652 to attach an activity associated with a third-party application to a handout, box 650 may be replaced with box 660, as shown in Figure 6D. Box 660 allows the instructor to select a third-party application for the activity from a list of applications.

[0095] In some embodiments, box 660 includes a first user interface element 662 for selecting any application from a list of available applications. However, in some cases, some third-party applications may implement at least a portion of the ClassKit software framework, which enables the third-party applications to provide progress data to one or more services for tracking student progress completing activities within the third-party applications. These third-party applications that implement at least a portion of the ClassKit software framework may be referred to herein as progress applications. Progress applications that implement at least a portion of the software framework may be listed separately in individual user interface elements below first user interface element 662. For example, second user interface element 664 allows the instructor to attach activities from the first progress application, third user interface element 666 allows the instructor to attach activities from the second progress application, and fourth user interface element 668 allows the instructor to attach activities from the third progress application.

[0096] In some embodiments, a third-party application implements at least a portion of the ClassKit software framework, which provides a means for incorporating interactive functionality into the third-party application. For example, the ClassKit software framework allows software developers to define contexts associated with an application. As used herein, context refers to metadata that describes the structure of the application. For example, a digital textbook may be described using contexts that define chapters, sections, pages, problem sets, and similar structures within the digital textbook. In general, a context defines metadata that allows the activity of the third-party application to be defined.

[0097] Context also allows tracking the progress of an activity within an application. For example, referring back to the digital textbook example, an instructor may define an activity related to reading a chapter of the textbook. Context allows the instructor to specify the particular chapter or section of the digital textbook that the instructor wants to assign when defining the activity. Thus, when specifying an activity using GUI 600, client application 205 can examine the context to determine which activity can be completed within the third-party application.

[0098] In some embodiments, an instructor can manually search a list of available contexts associated with one or more third-party applications and select the set of contexts that the instructor wants to assign to various handouts. The set of contexts can be stored in memory 420 and associated with the client application 205 to enable the instructor to attach activities associated with those contexts to handouts.

[0099] In other embodiments, each progress application installed on the client device 120 can notify the daemon 440 of the available contexts provided by the progress application. An instructor can utilize the client application 205 to browse a list of available contexts exposed by one or more progress applications and select several contexts as favorite contexts. For example, by selecting the first user interface element 662 in box 660, the instructor can browse all of the available contexts provided by the progress applications installed on the client device 120. The instructor can then select specific contexts as the instructor's favorite contexts. The subset of available contexts selected as favorites by the instructor can be stored in the local database 450 of the client device 120. The favorite contexts selected by the instructor can also be synchronized to the personal zone 465 of the remote database 460 scoped to the instructor. Additional user interface elements in box 660 can then be displayed, with each user interface element corresponding to a specific favorite context. The instructor can then select a specific favorite context to attach to a handout.

[0100] As shown in FIG. 6E, selecting second user interface element 654 causes client application 205 to display box 670 containing two user interface elements that allow the instructor to specify options associated with the attached file. In some embodiments, the options include file sharing permissions and whether the file is collaborative. For example, as shown in FIG. 6E, user interface element 672 allows the instructor to indicate whether the file is collaborative. This option determines whether the file is shared as a single resource among multiple users or whether copies of the file are shared with each user individually. User interface element 674 allows the instructor to change the resource permissions of the file. If the resource is read-only, students cannot edit the resource, but if the resource is read-write, students can edit the file.

[0101] It will be understood that additional user interface elements not explicitly shown in Figure 6E may be included in box 670. For example, another user interface element may allow the instructor to specify a file stored in local memory 420 to attach to the handout.

[0102] 7 illustrates a client-server relationship between a handout service 220 and several client application(s) 205 and / or daemon(s) 440 associated with multiple students in a class, according to some embodiments. As shown in FIG. 7, a class can include L students enrolled in the class, with each student having access to one or more client devices 120. As described above, the handout service 220 is configured to enable an instructor to create and manage handouts using multiple different client devices 120 controlled by the instructor. In addition, the handout service 220 is also configured to assist in distributing handouts to client applications 205 included on multiple additional client devices 120 associated with groups of students.

[0103] In some embodiments, an instructor can finish preparing a handout and then publish it for assignment to a group of students to view and complete. The handout service 220 copies the handout to a handout zone 465 of the remote database 460 scoped to the group of students specified in the handout. In response to the instructor publishing the handout, the handout service 220 notifies each client device 120 associated with the students via various paths 710 that the handout zone 465 of the remote database 460 has been updated. The paths 710 can refer to network paths associated with the destination addresses of the network interfaces of particular client devices 120.

[0104] The notifications may be received by a daemon 440 included on each client device 120. The daemon 440 then downloads the data from the handout zone 465 to a local database 450 on the client device 120. In some embodiments, the handout service 220 utilizes the school administration service 240 to obtain user identifiers for students in a class. The user identifiers may be utilized to send notifications using a separate notification service to each client device 120 associated with students in a given class.

[0105] In some embodiments, the group of students may be identified by school administration service 240, which is configured to return a list of identifiers of students enrolled in a particular class in response to receiving a request from handout service 220. In other embodiments, the directory service may interface with school administration service 240 to create directory data in directory zone 465 of remote database 460 scoped to the organization. Handout service 220 may then access the directory data from remote database 460 instead of retrieving the directory data directly from school administration service 240.

[0106] In response to a request to publish a handout, the handout service 220 can copy the handout from a personal zone 465 scoped to the instructor to a handout zone 465 scoped to a group of students in a class. Once a copy of the handout is stored in the handout zone 465, the handout service 220 sends a notification of the change to any client devices associated with students that subscribe to the handout zone 465. In response to receiving the notification from the handout service 220, a daemon 440 included in a student's client device 120 associated with the handout zone 465 automatically downloads the copy of the handout stored in the handout zone 465 to the local memory of the client device 120. The handout can then be viewed by a client application 205 on the student's client device 120 so that the student can complete activities and / or view educational materials attached to the handout.

[0107] 8A-8C show a GUI 800 implemented by a client application 210 on a client device 120 associated with a student, according to some embodiments. GUI 800 differs from GUI 600. Client application 210 may be configured to display GUI 800 in response to a user signing in to a user account associated with the student rather than the instructor. GUI 800 allows students to view class handouts assigned to groups of students by the class instructor.

[0108] In some embodiments, GUI 800 includes a first display area 810, a second display area 820, and a third display area 830. First display area 810 is utilized to display a representation of a handout assigned to a student in a particular class. Second display area 820 is utilized to display a representation of a class in which the student is enrolled. As shown in FIG. 8A , first display area 810 occupies a first portion of the pixels contained in display 302, and second display area 820 occupies a second portion of the pixels contained in display 302.

[0109] The first display area 810 includes a header showing elements 812 representing separate and distinct handouts assigned to students in the selected class. In some embodiments, each element 812 is defined by a box that includes the title of the handout, a descriptor of the due date associated with the handout, and an indication of whether the student has completed the handout. For example, a first element 812-1 is associated with a first handout for the assignment "Flora and Fauna," which is due March 12 and has been completed by the student, and a second element 812-2 is associated with a second handout for the assignment "Test Prep," which is due March 15 and has not been completed by the student. It will be appreciated that in other embodiments, the elements 812 may take different forms. For example, the element 812 may omit the due date and / or completion indicator. Alternatively, the element 812 may include information in addition to or instead of the due date and / or completion indicator. Additionally, the element 812 may include graphics, such as images or icons, representing different activities assigned to the handout. In some cases, the graphic may be animated, such as by playing a video as the background of element 812 or showing a slideshow of images attached to the handout.

[0110] The second display area 820 includes a header that shows elements 822 that represent separate and distinct classes in which the student is enrolled. A student can select a particular element 822 in the first display area 810 to view which handouts have been assigned to the student by the class's instructor. In some embodiments, each element 822 is defined by a box that includes the title of the class and a descriptor of the number of handouts assigned to students in the class. For example, the first element 822-1 is associated with the class "Science A," which has two handouts due; the second element 822-2 is associated with the class "Math A," which has no handouts due (zero handouts); and the third element 822-3 is associated with the class "English A," which has one handout due. It will be understood that in other embodiments, the elements 822 may take different forms. For example, the element 822 may omit an indicator of the number of handouts assigned to students in the class.

[0111] In some embodiments, a student can select a particular class using the input interface of client device 120. For example, a student can use a touch interface to select an element 822 associated with one of the available classes in which the student is enrolled. When an element 822 is selected, the element 822 is activated, and the element 812 in the first display area 810 may be updated to reflect the handout associated with the class represented by the selected element 822 in the second display area 820. The selected element 822 (e.g., the currently activated element) may be highlighted to indicate that the element 822 is active. The highlighting may be implemented by increasing the size of the element, changing the color associated with the element 822 or the border of the element 822, displaying a box around the selected element 822, etc. As shown in FIG. 8A , a first element 822-1 is activated and is shown larger than the second element 822-2 and the third element 822-3.

[0112] Third display area 830 is located near the top edge of display 302. Third display area 830 includes elements that allow a user of client application 210 to sign in to a student's user account. In some embodiments, third display area 830 is hidden and may be displayed in response to a user input, such as, for example, using a vertical swipe touch gesture to reveal third display area 830.

[0113] A first icon 832 is displayed near the left side of the third display area 830. When selected by a user, the first icon 832 can cause the client application 210 to prompt the user to enter credentials to sign in to the student's user account. In some embodiments, the first display area 810 and the second display area 820 are hidden until the user signs in to a particular user account using the first icon 832. In some embodiments, when the client application 210 launches, a separate interface is displayed to the user that prompts the user to enter credentials to sign in to the user account. The GUI 800 is displayed on the display 302 only when the user has successfully signed in to a particular user account. The first icon 832 can then be used to sign out of the user account and / or to sign in to a different user account.

[0114] By selecting a particular element 812 representing a handout for a particular class, the GUI 800 is updated with data representing the handout, as shown in Figure 8B. A first display area 840 presents information contained in the handout, such as the title of the handout, the due date of the handout, an indication of the class to which the handout is assigned, and any instructions included in the handout by the instructor. A second display area 850 presents information related to any handouts and / or activities attached to the handout by the instructor.

[0115] Selecting an activity in the second display area 850 opens the activity in a different application. If the activity is an attachment of a particular file type, an application corresponding to that file type is opened to view the attachment. If the activity is a bookmark or link, a web browser application is opened and the resource specified by the URL of the bookmark or link is retrieved. If the activity is a multimedia file, a multimedia application is opened to view the multimedia file. However, if the activity is associated with a progress application (e.g., if the activity is associated with a context provided by the progress application and selected by the instructor as a favorite context), the progress application is opened to complete the activity. In some embodiments, the progress application implements at least a portion of the ClassKit software framework for tracking student progress as they complete activities.

[0116] As shown in FIG. 8C , a GUI 860 of a first progress application is presented to a student on the display 302. The GUI 860 may provide output in the form of information presented as pixel data. As shown in FIG. 8C , the GUI 860 of the first progress application may provide text and graphics to assist a student in learning a particular subject. The GUI 860 may include elements, some of which may be interactive. For example, a student may select an image of a tree to display more detailed information about the tree. It will be understood that the third-party software shown in FIG. 8C is shown for illustrative purposes only, and the design of the software is not limited to any particular type of learning software. Indeed, the variety of software that can implement the ClassKit software framework is large.

[0117] 9 illustrates a client-server relationship between a hand-in service 230 and several client application(s) 210 and / or daemon(s) 440 of multiple students in a class, according to some embodiments. As shown in FIG. 9, a class may include L students enrolled in the class, each student having access to one or more client devices 120. The hand-in service 230 is configured to assist in managing hand-ins included in one or more handouts assigned to a corresponding group of students.

[0118] In some embodiments, when a handout that includes a placeholder for a hand-in is published, the placeholder in the handout indicates that the student is requested to submit a file corresponding to the hand-in. Again, the student generates the file for the hand-in using one or more applications on the client device 120. The student can then submit the hand-in using the client application 205. Alternatively, the client application 205 can create a GUI element (e.g., an application extension) added to a third-party application that allows the student to submit a hand-in from the third-party application.

[0119] In some embodiments, documents created by students may be synchronized to a personal zone 465-1 in the remote database 460 so that students can access the documents from different client devices 120. When a student selects to submit a document as a Hand-in, the Hand-in service can copy the document from the student-scoped personal zone 465-1 to the class-scoped Hand-in zone 465-2, and optionally to a specific handout corresponding to the Hand-in. Moving the document from the student's personal zone 465-1 to the Hand-in zone 465-2 allows the instructor to view the document. In some embodiments, the document is also deleted from the student-scoped personal zone 465-1, and the student is granted read-only access to the copy of the document stored in the Hand-in zone 465-2. As a result, once the document is submitted as a Hand-in, the student can no longer edit the document.

[0120] In some embodiments, the group of students associated with a particular handout may be identified by the school administration service 240, which is configured to return a list of identifiers of students enrolled in a particular class in response to receiving a request from the hand-in service 230. In other embodiments, the roster service may interface with the school administration service 240 to create roster data in a zone 465 of a remote database 460 that is scoped to the organization. The hand-in service 230 may then access the roster data from the remote database 460 instead of retrieving the roster data directly from the school administration service 240.

[0121] 10 illustrates a client-server relationship between the progress pipeline 250 and the client application 210 and / or daemon 440, according to some embodiments. To fully realize the benefits of the ClassKit software framework within a classroom environment, a means is required for tracking the activity of each student using an associated client device 120. The daemon 440 and progress pipeline 250 provide such functionality to any third-party application that implements at least a portion of the ClassKit software framework.

[0122] 10, client device 120 includes a processor 410 and memory 420 containing software and data. Memory 420 of client device 120 includes an operating system 430, a client application 210, a daemon 440, and one or more progress applications 1010. Daemon 440 implements an API to communicate with progress applications 1010 via the ClassKit software framework.

[0123] In some embodiments, one or more progress applications 1010 each implement at least a portion of a ClassKit software framework for tracking progress associated with one or more contexts provided by the progress application 1010. The ClassKit software framework includes at least one of source code added to the source code of the progress application 1010, source code of a library of functions utilized by the progress application 1010, and a library of compiled functions that can be linked to the source code of the progress application 1010. The software framework provides functionality included in the source code of the progress application 1010 that makes API calls to a daemon 440 running in the background of the client device 120. The API calls include progress tracking information related to actions performed by a user within the progress application 1010.

[0124] A software developer can implement API calls in the source code of the progress application 1010 to implement progress tracking functionality within the progress application 1010. For example, in a document viewing application, the source code can be configured to generate an API call each time the user navigates to a new page indicating that the user has read a particular page of the document. A software developer can also define one or more contexts for the progress application 1010 that specify the structure of the progress application 1010 to be attached to handouts in the client application 210 and to allow instructors to create activities that utilize the progress application 1010. Each context is a data structure that defines the structure of the progress application 1010 using standard language (e.g., predefined keywords) and / or formatting.

[0125] It will be appreciated that a single progress application 1010 may be associated with more than one context. For example, a textbook application may be implemented to view multiple textbooks stored in different files or data structures. A different context may be defined for each different textbook. Each context may define a particular layout for the corresponding textbook, such as defining the textbook's table of contents, including the title of the textbook, the author of the textbook, the edition of the textbook, the number of chapters in the textbook, a description of each chapter of the textbook, a description of the sections contained in each chapter, a description of each page contained in each section, a description of the problem sets within each chapter or section, etc. An instructor may then use these contexts to select specific activities associated with a particular context and attach these selected activities to handouts assigned to students in a class.

[0126] Daemon 440 is configured to receive API calls from progress applications 1010. The API calls may include progress tracking information generated within the progress application 1010. In some embodiments, progress tracking may be enabled or disabled. Daemon 440 is configured to ignore progress tracking information received from progress applications 1010 when progress tracking is disabled. However, when progress tracking is enabled, daemon 440 is configured to process the progress tracking information. In some embodiments, daemon 440 aggregates progress tracking information received via two or more API calls before processing the aggregated progress tracking information received from one or more progress applications 1010.

[0127] In some embodiments, progress tracking may be enabled or disabled for individual progress applications 1010. Thus, progress tracking may be enabled for a first application and disabled for a second application. Thus, daemon 440 is configured to filter out any progress tracking information from applications for which progress tracking is disabled and to process only progress tracking information for applications for which progress tracking is enabled.

[0128] In some embodiments, progress tracking for all progress applications 1010 installed on client device 120 is automatically enabled upon installation of client application 210. Thus, a user of client device 120 is required to disable progress tracking for client device 120 or for individual third-party applications installed on client device 120 to prevent collection of progress tracking information, which may be referred to as an opt-out policy. In other embodiments, progress tracking for all third-party applications 1010 installed on client device 120 is automatically disabled upon installation of client application 210. Thus, a user of client device 120 is required to enable progress tracking for client device 120 or for individual third-party applications installed on client device 120 to enable collection of progress tracking information, which may be referred to as an opt-in policy.

[0129] Progress tracking information may include, but is not limited to, information identifying actions completed by a user of client device 120 using progress application 1010. For example, progress tracking information may include information indicating that a user has viewed or read a particular chapter of a digital textbook, completed a number of questions presented to the user by the GUI of progress application 1010, clicked on a link in or interacted with an interactive element of progress application 1010, or completed other actions.

[0130] The progress tracking information may include personally identifying information (PII). This PII may be considered sensitive information, and therefore, care is taken by the daemon 440 to secure this information when transmitted from the client device 120 to the server device 110. In some embodiments, a progress zone 465 scoped to a specific student is utilized to store progress tracking information for each student associated with a user account. The progress tracking information is stored and locally encrypted in the memory 420 of the client device 120. The progress tracking information may also be remotely encrypted in the progress zone 465 of the remote database 460. The progress tracking information may be locally or remotely encrypted with different encryption keys. Additionally, the daemon 440 may be configured to encrypt the progress tracking information during transmission to the remote database 460. The encryption key used for encryption during transmission may be different from the encryption key utilized locally or remotely. As a result, progress tracking information can be secured locally using various forms of authentication (e.g., biometric authentication, requiring the user to enter a passcode, etc.), and progress tracking information can be secured remotely (i.e., on the server side) using additional forms of authentication such as mutual TLS as a secure transmission mechanism.

[0131] FIG. 11 illustrates filtering of progress tracking information by a daemon 440, according to some embodiments. In some embodiments, the daemon 440 is configured to filter progress tracking information received from the progress application 1010 based on a set of active contexts associated with one or more handouts assigned to the student by one or more instructors. The daemon 440 can track a list of active contexts 1170 based on attachments to one or more handouts currently assigned to the student (or class). In some embodiments, handout attachments that have already passed their due date can be closed (e.g., deactivated) by an instructor and removed from the list of active contexts 1170. The daemon 440 is configured to filter progress tracking information received from the progress application 1010 based on the list of active contexts 1170.

[0132] For example, a first instructor can assign a first handout to a particular student, the handout including activities associated with a first context of a first progress application 1010-1, and a second instructor can assign a second handout to a particular student, the handout including activities associated with a second context of a second progress application 1010-2. The first context of the first progress application 1010-1 and the second context of the second progress application 1010-2 are considered active contexts. However, a student may use a third progress application 1010-3 that is not associated with any active context, and the daemon 440 can be configured to filter out (e.g., ignore) any progress tracking information 1140 received from the third progress application 1010-3 to avoid collecting unnecessary progress tracking data. Similarly, the daemon 440 can also filter out any progress tracking information associated with a different, inactive context and received from the first progress application 1010-1 and / or the second progress application 1010-2. For example, an active context may define the structure of a Macbeth e-book, and an inactive context may define the structure of a Romeo and Juliet e-book. Any progress tracking information received while the Macbeth e-book is open in the progress application 1010 is processed by the daemon 440, and tracking information received while the Romeo and Juliet e-book is open in the progress application 1010 is filtered out and ignored.

[0133] 11 , first progress tracking information 1110 received from a first progress application 1010-1 and associated with a first context included in the list of active contexts 1170 is processed by daemon 440. In some embodiments, daemon 440 stores first progress tracking information 1110 in local database 450 as progress tracking information 1150-1. Progress tracking information 1150-1 is uploaded to progress zone 465 of remote database 460 scoped to the student via a synchronization mechanism. Similarly, second progress tracking information 1120 received from a second progress application 1010-2 and associated with a second context included in the list of active contexts 1170 is processed by daemon 440. Second progress tracking information 1120 is stored in local database 450 as progress tracking information 1150-2. Again, the progress tracking information 1150-2 is uploaded via a synchronization mechanism to the progress zone 465 of the remote database 460 scoped to the student.

[0134] However, third progress tracking information 1130 received from the second progress application 1010-2 and associated with a third context that is not included in the list of active contexts 1170 is ignored by the daemon 440. Similarly, fourth progress tracking information 1140 received from the third progress application 1010-3 and associated with a fourth context that is not included in the list of active contexts 1170 is also ignored by the daemon 440.

[0135] The daemon 440 also creates metadata 1160 associated with the encrypted progress tracking information 1150. The metadata 1160 may include a reference to the encrypted progress tracking information 1150 in the remote database 460 and a context identifier for the context associated with the progress tracking information. In some embodiments, the metadata 1160 includes additional information that may be utilized by the progress pipeline 250, such as a class identifier, a handout identifier corresponding to the context, an application identifier corresponding to the progress application 1010 that generated the progress tracking information 1150, or a user identifier associated with the user signed in to the user account on the client device 120. As a result, the progress pipeline 250 does not process the actual progress tracking information, which may include PII. Instead, the progress pipeline 250 only processes the metadata 1160 associated with the progress tracking information, and the actual progress tracking information remains encrypted and stored in the personal progress zone 465 of the remote database 460.

[0136] It will be appreciated that in other embodiments, the actual progress tracking information, rather than the metadata 1160, may be provided to the progress pipeline 250 either encrypted or unencrypted, but these embodiments may provide less security for the PII contained in the progress tracking information.

[0137] It will be appreciated that receipt of an active context implicitly authorizes the daemon 440 to track progress associated with that particular context. When a context is removed from the list of active contexts 1170, the implicit permission is revoked and therefore the daemon will not track progress associated with that context.

[0138] 12 illustrates a progress pipeline 250 according to some embodiments. The progress pipeline 250 is implemented on one or more server devices 110. In some embodiments, each component of the progress pipeline 250 is implemented on a different server device 110. In some cases, particular components of the progress pipeline 250 are implemented on a group of server nodes, sometimes referred to herein as a cluster.

[0139] 12, progress pipeline 250 receives progress tracking metadata 1202 over network 150 from daemons 440 included on one or more client devices 120. Progress pipeline 250 may be implemented as one or more services associated with a network address, where messages sent to the network address are processed by the corresponding service.

[0140] In some embodiments, the on-ramp service 1210 receives progress tracking metadata 1202 from one or more daemons 440. The progress tracking metadata 1202 includes information related to a particular student's progress tracking information 1150. In some embodiments, the progress tracking metadata 1202 includes a user identifier associated with the user account of the client application 210 and a context identifier for a particular context that corresponds to the progress tracking information 1150.

[0141] In some embodiments, the on-ramp service 1210 is configured to check whether progress tracking is enabled for a particular user. An administrator can change a particular student's preferences for permission to track the given student's progress through the school administration service 240. The on-ramp service 1210 may receive notification about a change in the status of a given student's permission preferences before the change is received by the daemon 440 of a particular client device 120. As a result, the daemon 440 can continue to collect progress tracking information for a given student even after the student has disabled progress tracking altogether. Thus, the on-ramp service 1210 implements a gated function for progress tracking metadata received by the progress pipeline 250 based on the set of flags stored in the remote database 460.

[0142] In some embodiments, the roster service is configured to use the school administration service 240 to periodically pull the set of permission flags for each student in a particular school district and store the set of permission flags in a permission zone 465 of an organization-scoped remote database 460. The on-ramp service 1210 can then query the set of permission flags for each item of progress tracking metadata 1202 received using a user identifier included in the progress tracking metadata 1202.

[0143] In some embodiments, the on-ramp service 1210 invokes the identity service 260 and requests a directory service identifier using the user identifier received in the progress tracking metadata 1202. The directory service identifier may be registered to be associated with the user identifier when the user account is created. The directory service identifier remains constant for the life of the user account and is used only within the server side of the client-server architecture 200, such that the client application 210 and / or the daemon 440 never receive the directory service identifier. As a result, anyone snooping on the traffic between the daemon 440 and the progress pipeline 250 cannot obtain the directory service identifier to associate specific PII stored on one or more servers with a specific user's identifier. The identity service 260 returns the directory service identifier to the on-ramp service 1210, which de-references the user identifier included in the progress tracking metadata 1202 and replaces the user identifier with the directory service identifier. In some embodiments, the set of permission flags corresponds to the directory service identifier, not the user identifier of the user account. As a result, the user identifier is de-referenced before the on-ramp service 1210 performs its gate-keeping function. Once the on-ramp service 1210 has finished processing the progress tracking metadata 1202 , the progress tracking metadata 1202 is sent to a queue 1215 .

[0144] In other embodiments, an internal identifier may be utilized instead of a directory service identifier. Thus, progress tracking metadata 1202 is tracked throughout the progress pipeline 250 using the internal identifier rather than the directory service identifier. This allows the progress pipeline 250 to sever all associations between progress tracking information and a particular user simply by removing the association between the internal identifier and the directory service identifier.

[0145] In some embodiments, queue 1215 is a queuing service implemented on server device 110. Queue 1215 may be implemented within the same server device 110 as on-ramp service 1210 or within a different server device 110. In some embodiments, queue 1215 is implemented as a distributed streaming service, such as the Apache® Kafka distributed streaming platform. Queue 1215 allows on-ramp service 1210 and enrichment service 1220 to run asynchronously.

[0146] In some embodiments, the enrichment service 1220 is implemented on one or more server devices 110. The enrichment service 1220 is configured to receive the progress tracking metadata 1202 from the queue 1215 and enrich the progress tracking metadata 1202 with additional information. In some embodiments, the enrichment service 1220 requests roster data from the school administration service 240 based on a directory service identifier included in the progress tracking metadata 1202. The roster data may include one or more class identifiers for any classes in which the student is enrolled and a list of student identifiers for the students enrolled in each class. The school administration service 240 may utilize the student's directory service identifier rather than the user identifier provided by the daemon 440 in the raw progress tracking metadata 1202. In other embodiments, a user identifier rather than an internal identifier may be utilized to query the school administration service 240.

[0147] The enrichment service 1220 can also retrieve all active handouts of a particular class from the corresponding zone 465 of the remote database 460. The roster data and handouts are cross-referenced with the class identifier and directory service identifier included in the progress tracking metadata 1202 to add additional information to the progress tracking metadata 1202 if not already included in the progress tracking metadata 1202. In some embodiments, the enrichment service 1220 ensures that the progress tracking metadata 1202 includes the organization identifier, class identifier, handout identifier, and context identifier, as well as a reference to the encrypted progress tracking information 1150.

[0148] In some embodiments, progress tracking metadata 1202 can be associated with multiple class identifiers and / or multiple handout identifiers. For example, if two different instructors from two different classes assign activities to a particular student in two different handouts that reference the same context, when the student completes the activity for one class, the student is credited with simultaneously completing the same activity for the other class. Once enrichment service 1220 enriches progress tracking metadata 1202, enrichment service 1220 sends the enriched progress tracking metadata 1202 to queue 1225.

[0149] In some embodiments, queue 1225 is similar to queue 1215. Queue 1225 may be implemented within the same server device 110 as enrichment service 1220 or within a different server device 110.

[0150] In some embodiments, progress tracking metadata 1202 is queued and stored in a distributed file system 1230. In some embodiments, distributed file system 1230 is the Apache® Hadoop Distributed File System (HDFS) as implemented on a collection of server nodes. Each server node includes a set of inexpensive disk drives on which data can be stored. In some circumstances, distributed file system 1230 includes at least one journal node configured to track changes to progress tracking metadata 1202 stored in distributed file system 1230. The journal node protects the integrity of progress tracking metadata 1202 in the event of a server node crash.

[0151] It will be appreciated that progress pipeline 250 may be utilized by multiple organizations, even if each organization's data is stored in a separate and distinct remote database 460 scoped to the particular organization. As a result, distributed file system 1230 stores a large amount of progress tracking metadata 1202 for one or more organizations. Aggregation service 1240 is designed to sort through all of the progress tracking metadata 1202 and organize the progress tracking metadata 1202 received by an organization during a tracking window. In some embodiments, aggregation service 1240 is a computational service for processing large datasets. For example, but not limited to, aggregation service 1240 may be implemented as an Apache Spark service implemented on a cluster of server nodes. The progress tracking metadata 1202 is then processed in one or more Spark jobs spawned by aggregation service 1240.

[0152] In some embodiments, the tracking window may be, for example, minutes or hours. For example, the aggregation service 1240 may sort the progress tracking metadata 1202 received at the distributed file system 1230 in 10-minute windows. In some embodiments, the aggregation service 1240 is configured to execute a routine upon expiration of a timer set based on the tracking window. Once the routine is executed, the timer is reset to await the expiration of the next tracking window.

[0153] In some embodiments, the aggregation service 1240 consults a journal node of the distributed file system 1230 to collect information related to all of the progress tracking metadata 1202 stored in the distributed file system 1230 during a tracking window. The journal node tracks changes to the distributed file system 1230 and can return a list of all progress tracking metadata 1202 stored in the distributed file system 1230 during a previous tracking window. The aggregation service 1240 is configured to pull the progress tracking metadata 1202 received during the tracking window and generate a data structure for each organization for one or more organizations that submit progress tracking metadata 1202 to the progress pipeline 250 during the tracking window. The data structure may be referred to herein as a cube.

[0154] In some embodiments, the aggregation service 1240 generates separate and distinct cubes scoped to a single organization. All progress tracking metadata 1202 corresponding to a particular organization identifier is sorted into the corresponding cube. The progress tracking metadata 1202 stored in a given cube may be sorted by class identifier, handout identifier, context identifier, and student identifier (e.g., user identifier, directory service identifier, etc.).

[0155] In some embodiments, cubes are stored in block-based storage 1250. In some embodiments, block-based storage 1250 may be HDDs included in server devices 110 that implement aggregation service 1240. In other embodiments, block-based storage 1250 may be a virtual block-based storage device implemented on one or more sets of physical storage devices included in one or more server devices 110. In other embodiments, block-based storage 1250 may be replaced by a distributed database service, such as Apache Cassandra, implemented on a group of server nodes. In still other embodiments, block-based storage 1250 may be replaced by a distributed file system similar to distributed file system 1230 described above.

[0156] The aggregation service 1240 generates one or more cubes containing the progress tracking metadata 1202 received during the tracking window for one or more corresponding organizations. The aggregation service 1240 adds a reference to each cube to a queue 1245. In some embodiments, the queue 1245 is similar to queues 1215 and 1225 and allows the off-ramp service 1260 to operate asynchronously with the aggregation service 1240.

[0157] In some embodiments, the off-ramp service 1260 is configured to process each cube reference as it leaves the queue 1245. In some embodiments, the off-ramp service 1260 is simply configured to read the progress tracking metadata 1202 stored for a given cube in block storage 1250 and write each progress tracking metadata 1202 entry to a particular zone 465 of the remote database 460 that is scoped to a particular class, context, and optionally, student based on a class identifier, context identifier, and / or directory service identifier / user identifier included in the progress tracking metadata 1202. As a result, the off-ramp service 1260 sorts the progress tracking metadata 1202 to a granularity finer than a cube.

[0158] In other embodiments, the off-ramp service 1260 de-references the progress tracking metadata 1202 by replacing the directory service identifier utilized by a previous service in the progress pipeline 250 and replaces the directory service identifier with the progress identifier. While the progress identifier may be associated with the directory service identifier in the identity service 260, the progress identifier may be disassociated from the directory service identifier to sever any connection between the progress tracking data 1150 and a particular user. In some embodiments, the progress identifier is discarded in response to a user opting out of progress tracking (e.g., disabling progress tracking entirely). For example, an organizational administrator can use the school administration service 240 to disable progress tracking for specific students, classes, schools, and / or the entire school district. These changes allow the identity service 260 to disassociate the progress identifier from the directory service identifier for those students for whom progress tracking has been disabled. While the progress tracking information may still exist in the remote database 460, the progress tracking information cannot be associated with a particular student once the association between the progress identifier and the directory service identifier is severed.

[0159] Once the off-ramp service 1260 has finished processing the progress tracking metadata 1202, the progress tracking metadata 1202 is sent to an escrow service 1270. In some embodiments, the escrow service 1270 is implemented on at least one server device 110. The escrow service 1270 is configured to pull a reference to the encrypted progress tracking information 1150 by an entry in the progress tracking metadata 1202 and store the progress tracking information 1150 in progress storage 1280. In some embodiments, the progress storage 1280 is a network-based storage service. The progress tracking information 1150 is stored in a zone similar to zone 465, scoped to a particular class identifier, context identifier, progress identifier, and (optionally) handout identifier.

[0160] In some embodiments, progress storage 1280 is separate and distinct from remote database 460 and may only be accessed on the server side of client-server architecture 200. Thus, daemon 440 does not have direct visibility into progress tracking information 1150 archived by progress pipeline 250. In other embodiments, progress storage 1280 may be implemented in a new zone 465 of remote database 460 and synchronized to client device 120 using reporting service 270.

[0161] Although not explicitly shown, various services in the progress pipeline 250 may be configured to reuse particular progress tracking metadata 1202 being processed by the service if the service encounters a particular problem during processing. In some embodiments, a retry queue is associated with each service, and progress tracking metadata 1202 may be pushed to the retry queue for processing at a later time. For example, a service such as the identification service 260 may experience downtime that prevents the progress pipeline 250 from completing a particular operation. In such a case, the data processed by the service may be stored in the retry queue for processing at a later time when the downed service may become available. Alternatively, a service may simply be configured to return progress tracking metadata 1202 to a queue that interfaces with the service's input. For example, the enrichment service 1220 may be configured to push progress tracking metadata 1202 back to the queue 1215 for processing at a later time.

[0162] The reporting service 270 may then pull the progress tracking information 1204 from the progress storage 1280 in response to requests from the client application 210 and / or the daemon 440 to generate reporting data that is provided to the client application 210 and / or the daemon 440.

[0163] 13 illustrates a client-server relationship between a reporting service 270 and a client application 210 and / or daemon 440, according to some embodiments. The client application 210 and / or daemon 440 included in the client device 120 interfaces with the reporting service 270 of the server device 110. After handouts are assigned to students in a class, an instructor may want to track the students' progress in completing the assignments. More specifically, the GUI of the client application 210 may display a representation of progress tracking information associated with a particular handout. Attachments, handouts, and / or activities may each be displayed along with information related to each student's completion progress.

[0164] In some embodiments, client application 210 receives input to display a representation of the handout on display 302 of client device 120. In response to the input, client application 210 requests a report from reporting service 270. As used herein, a report may refer to a file or data structure that contains reporting data derived from progress tracking information 1204 pulled from progress storage 1280.

[0165] In some embodiments, reporting service 270 receives a request for a report from client application 210. The request may include a class identifier, a context identifier, a user identifier, and (optionally) a handout identifier. Reporting service 270 may utilize identification service 260 to find a progress identifier corresponding to the user identifier and then query progress storage 1280 to retrieve progress tracking information 1204 corresponding to the class identifier, context identifier, progress identifier, and (optionally) handout identifier.

[0166] Additionally, the request may specify a time frame associated with the request. For example, the request may specify whether the query should limit progress tracking information to the time before the handout's due date, the time after the handout's due date, or the time up to (and including) the handout's due date. As used herein, a due date may refer to the time when a context attached to a handout is deactivated, thereby blocking further progress tracking related to that context after the due date unless another handout containing the context has been assigned to the student. As a result, the reporting service 270 generates a set of progress tracking information 1204 for a particular context corresponding to students in a particular class and assigned to a particular handout.

[0167] In some embodiments, the daemon 440 aggregates the progress tracking information 1204 for a class and calculates various statistics for the class based on the reported data. For example, the daemon 440 can calculate the total number of students in the class and the total number of students who completed a hand-in or activity. The daemon 440 can also use timestamps associated with different progress tracking information 1204 to calculate the total time it took each student to complete a hand-in or activity (e.g., by calculating the difference between the first and last timestamps for all progress tracking information for a particular activity or hand-in). As another example, the daemon 440 can calculate each student's score based on the correct / incorrect answers provided for a quiz or problem set. The daemon 440 can calculate the ratio of passing scores to failing scores, the ratio of the number of students with passing scores to the total students in the class or the number of students with failing scores, the average score for the class, the minimum and maximum scores for the class, the median score and standard distribution of the set of scores, the distribution of scores mapped to letter grades (e.g., the number of A scores, the number of B scores, etc.), and any other relevant statistics related to the scores and / or the progress tracking information 1204.

[0168] In some embodiments, a student may have several attempts to complete a particular activity. For example, a student may take a quiz multiple times. In such embodiments, the reporting request may specify whether the reporting data should include progress tracking information 1204 for each attempt completed by the student or only the last attempt completed by the student. If the progress tracking information includes data for multiple attempts, the daemon 440 may be configured to calculate statistics related to the multiple attempts, such as an average score, a highest score across all attempts, a ratio of successful attempts to unsuccessful attempts, etc.

[0169] In some embodiments, the daemon 440 may also be configured to download roster information from the roster zone 465 of the remote database 460, for example, to associate user identifiers associated with progress tracking information with student names.

[0170] In other embodiments, client application 210 may be configured to perform functions that aggregate progress tracking information 1204 and / or calculate statistics related to progress tracking information 1204 in local database 450. Daemon 440 merely facilitates the download of progress tracking information 1204 from zone 465 of remote database 460 to local database 450 so that client application 210 can access the progress tracking information 1204. Client application 210 is then configured to perform any necessary analysis using the progress tracking information 1204 to generate values ​​that populate fields in a GUI presented to the instructor.

[0171] 14A-14B illustrate a GUI 1400 of a client application 210, according to some embodiments. The GUI 1400 may be used to present reporting data related to assigned handouts to an instructor. The GUI 1400 may be presented on the display 302 of a client device 120. As shown in FIG. 14A, the GUI 1400 is displayed on a tablet computer, such as client device 120-2. The GUI 1400 may include several elements, including graphics, text, icons, shapes, input fields, display areas, etc. In some embodiments, the GUI 1400 is displayed in response to an instructor selecting an assigned handout element 312 in the GUI 300.

[0172] GUI 1400 includes a first display area 1410 and a second display area 1420. First display area 1410 presents information contained in the handout, such as the title of the handout, the due date of the handout, an indication of the class to which the handout is assigned, and any instructions included in the handout by the instructor. Second display area 1420 presents information related to any handouts and / or activities attached to the handout by the instructor.

[0173] Each hand-in or activity attached to a handout and presented in second display area 1420 may also be presented with reporting data related to the hand-in or activity. The reporting data may be displayed in one or more elements 1422. For example, as shown in FIG. 14A , an activity associated with Application 2 includes a first element 1422-1 that displays the ratio of passing scores to failing scores for students who completed the activity, a second element 1422-2 that displays the average time it took students to complete the activity, and a third element 1422-3 that displays the ratio of the number of students who completed the activity to the total number of students in the class.

[0174] Additionally, an icon 1424 is presented near the hand-in or activity that allows the instructor to view more detailed information about the reported data. As shown in FIG. 14B , by selecting the icon 1424, the GUI 1400 may be updated to display the detailed information near the hand-in or activity in a second display area 1420. It will be appreciated that the second display area 1420 may automatically adjust to cover pixels of the display 302 that were previously covered by both the first display area 1410 and the second display area 1420. Of course, the extent of the second display area 1420 may remain the same as in FIG. 14A or may be manually updated in response to a touch input gesture that repositions the first display area 1410 and / or the second display area 1420 up or down relative to the pixels of the display 302. For example, a swipe gesture may be utilized to scroll the first display area 1410 and / or the second display area 1420 up or down.

[0175] As shown in FIG. 14B , reporting data related to each individual student enrolled in the class may be presented in GUI 1400. For example, detailed information entry 1426 may include the student's name, an indication of whether the student passed or failed the activity, the time it took each student to complete the activity in the third-party application, if completed, and an indication of whether the activity was completed. It will be understood that the information shown in FIG. 14B is only one example of the type of detailed information that may be presented in GUI 1400. In other embodiments, the detailed information may include other types of statistics or raw progress tracking information (e.g., a breakdown of each action tracked by daemon 440, along with the time the action was received by daemon 440 and information related to the action). In still other embodiments, the detailed information may include a graphical representation of the reporting data, such as a graph or chart showing a student's progress over time, a link to the answers submitted by the student to a series of questions, a link to a record of hand-ins or activities performed by the student, etc.

[0176] In some embodiments, the detailed information may present a detailed breakdown of progress tracking information for subsections of a particular context. For example, a context may describe the structure of a problem set assigned to a student. The progress tracking information may include information related to the student's answers provided to each question in the problem set. Thus, even if the answers to a problem set are considered the parent context, child contexts of each problem in the problem set may track the answers to each individual question provided by the student. Thus, the detailed information allows the instructor to view the specific answers given to each question and whether the answers were correct or incorrect. GUI 1400 may allow the instructor to easily view such visual representations of the progress tracking data.

[0177] In some embodiments, the client application 210 can output the results of the reporting data to an archived file or data structure for subsequent review, for example, by faculty and administrators, when determining the final grade for each student in the class.

[0178] 15 is a flowchart of a method 1500 for tracking student activity on a client device, according to some embodiments. Method 1500 is performed by client device 120. In some embodiments, method 1500 may be implemented as logic configured to monitor activity associated with one or more applications. The logic may include instructions executed by processor 410 of client device 120 to monitor a student's progress completing activities attached to handout resources. In some embodiments, method 1500 is implemented within logic implemented by daemon 440.

[0179] At 1502, progress tracking information is received from one or more applications. In some embodiments, the progress tracking information includes at least one of an application identifier corresponding to the application that generated the progress tracking information, a context identifier corresponding to a particular context associated with the application, a timestamp indicating when the progress tracking information was generated, or information indicating actions performed within the application and / or results of the actions. In some embodiments, the progress tracking information is received by a daemon within an API call generated by the application.

[0180] At 1504, the progress tracking information is filtered. In some embodiments, the progress tracking information is filtered based on a determination of whether progress tracking is enabled or disabled for each of one or more applications installed on the client device. The progress tracking information is discarded when progress tracking information is received from any application for which progress tracking is disabled and processed when progress tracking information is received from any application for which progress tracking is enabled. In other embodiments, a list of active contexts associated with one or more applications is received, and the progress tracking information is compared to the list of active contexts. For example, a context identifier included in the progress tracking information is compared to certain context identifiers included in the list of active contexts. The progress tracking information is discarded when the progress tracking information corresponds to a context not included in the list of active contexts and processed when the progress tracking information corresponds to a context included in the list of active contexts.

[0181] At 1506, the progress tracking information is stored in a remote database accessible from a network. In some embodiments, the remote database is a network-based storage service that uses different zones to store different types of data limited to specific scopes. Each zone that stores progress tracking information may be protected by disk-based encryption, and access to the data may be limited to trusted servers using authentication and message signing techniques. The progress tracking information may be stored in a personal zone within the network-based storage service, which is associated with a specific user identifier for a user account associated with a client application installed on the client device.

[0182] Metadata associated with the progress tracking information is generated at 1508. The metadata may include a subset of information in the progress tracking information, such as a user identifier and / or a context identifier, and information such as a reference to progress tracking information stored in remote database 460.

[0183] At 1510, the progress tracking information metadata is sent to a progress pipeline. In some embodiments, the progress pipeline consolidates and aggregates progress tracking information received from multiple different client devices associated with multiple different students in one or more classes within one or more institutions. The progress pipeline can include several services implemented on several different server devices.

[0184] 16 is a flowchart of a method 1600 for processing progress tracking information using one or more services available over a network, according to some embodiments. Method 1600 is performed by progress pipeline 250. In some embodiments, method 1600 may be implemented as logic configured to process at least one of the progress tracking information or metadata corresponding to the progress tracking information. The logic may include instructions executed by processor 410 of server device 110 for processing the progress tracking information and / or the metadata. In some embodiments, method 1600 is implemented within logic implemented by one or more services, including on-ramp service 1210, enrichment service 1220, distributed file system 1230, aggregation service 1240, off-ramp service 1260, and custodial service 1270.

[0185] At 1602, metadata corresponding to progress tracking information is received from a plurality of client devices. In some embodiments, the metadata may include a reference to progress tracking information stored in a remote database. The metadata may also include at least one of a class identifier, a user identifier, or a context identifier corresponding to the progress tracking information.

[0186] The metadata is enriched with additional information at 1604. In some embodiments, the metadata supplemented with the additional data correlates the progress tracking information to an organization identifier, a class identifier, a directory service identifier, a context identifier, and optionally a handout identifier.

[0187] At 1606, a data structure is generated that includes a subset of the metadata received by the progress pipeline during the tracking window for a particular organization. In some embodiments, the distributed file system utilizes at least one journal node that enables retrieval of all of the metadata received during the tracking window from the distributed file system. The service sorts the metadata into different data structures that correspond to each of one or more organizations that submitted metadata to the progress pipeline during the tracking window.

[0188] At 1608, the data structure is processed to generate sorted metadata based on a class identifier and / or a context identifier. Sorting can also be based on a directory service identifier or an internal identifier. The operation at 1608 can be performed for each data structure created for one or more organizations.

[0189] Progress tracking information corresponding to the sorted metadata is retrieved from the remote database at 1610. A reference included in the metadata can be used to request the progress tracking information from the remote database.

[0190] At 1612, the progress tracking information is stored in memory accessible by the reporting service. In some embodiments, the progress tracking information is dereferenced from the directory service identifier or internal identifier, and the progress identifier replaces the directory service identifier or internal identifier in the metadata associated with the progress tracking information. The progress identifier and the progress tracking information may be stored in a network-based storage service accessible by the reporting service. The network-based storage service need not be accessible by the client device except indirectly through the reporting service.

[0191] 17 shows a detailed diagram of an exemplary computing device 1700 that may be used to implement various apparatus and / or methods described herein, according to some embodiments. In particular, the detailed diagram illustrates various components that may be included in the computing devices shown in FIGS. 1-14 and described herein. For example, one or more of server device 110, client device 120, or any other device, including any network appliance and / or consumer electronics device, may include the components of computing device 1700.

[0192] As shown in FIG. 17 , computing device 1700 may include processor 1702, which represents a microprocessor or controller for controlling the overall operation of computing device 1700. Computing device 1700 may also include user input devices 1708 that allow a user of computing device 1700 to interact with computing device 1700. For example, user input devices 1708 may take various forms, such as buttons, keypads, dials, touchscreens, audio input interfaces, visual / image capture input interfaces, inputs in the form of sensor data, etc. Furthermore, computing device 1700 may include display 1710 (screen display) that may be controlled by processor 1702 to present visual information to the user. Data bus 1716 may facilitate data transfer between at least storage device 1740, processor 1702, and controller 1713. Controller 1713 may be used to interface with and control various devices via device control bus 1714. Computing device 1700 may also include a network / bus interface 1711 that connects to data link 1712. In the case of a wireless connection, the network / bus interface 1711 may include a wireless transceiver.

[0193] Computing device 1700 also includes a storage device 1740. This device may include disk(s) (e.g., hard disk) and a storage management module that manages one or more partitions within storage device 1740. In some embodiments, storage device 1740 may include flash memory, semiconductor (solid-state) memory, or the like. Computing device 1700 may also include random access memory (RAM) 1720 and read-only memory (ROM) 1722. ROM 1722 may store programs, utilities, or processes to be executed in a non-volatile manner. RAM 1720 may provide volatile data storage and stores instructions related to the operation of computing device 1700.

[0194] As mentioned above, one aspect of the present technology is the collection and use of available data from various sources to track a student's progress as they complete assignments. The present disclosure contemplates that, in some examples, this collected data may include personal information data that uniquely identifies a particular person or that can be used to contact or locate a particular person. Such personal information data may include demographic data, location-based data, phone numbers, email addresses, Twitter IDs, home addresses, data or records regarding a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), birth date, or any other identifying or personal information.

[0195] This disclosure recognizes that the use of such personal information data in the present technology can be for the benefit of the user. For example, personal information data can be used to enhance the educational experience of individuals attending a school. Thus, using such personal information data allows teachers to tailor their lessons or individual attention to the needs of their students. Moreover, other uses of personal information data that benefit the user are also contemplated by this disclosure. For example, health and fitness data can be used to provide insight into the user's overall wellness or as proactive feedback to individuals using the technology to pursue wellness goals.

[0196] This disclosure contemplates that entities involved in the collection, analysis, disclosure, transmission, storage, or other use of such personal information data will adhere to robust privacy policies and / or practices. Specifically, such entities should implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or government requirements for maintaining the strict confidentiality of personal information data. Such policies should be easily accessible to users and should be updated as data collection and / or use changes. Personal information from users should be collected for the entity's lawful and legitimate use and should not be shared or sold except for those lawful uses. Furthermore, such collection / sharing should be carried out after the user's informed consent is obtained. Furthermore, such entities should consider taking all necessary measures to protect and secure access to such personal information data and to ensure that others with access to that personal information data comply with their privacy policies and procedures. Furthermore, such entities may be able to undergo third-party assessments to demonstrate their adherence to widely accepted privacy policies and practices. Furthermore, policies and practices should be tailored to the specific types of personal data collected and / or accessed and should comply with applicable laws and standards, including jurisdiction-specific considerations. For example, in the United States, the collection of or access to certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA), while health data in other countries may be subject to other regulations and policies and should be addressed accordingly. Therefore, different privacy practices should be maintained with respect to different types of personal data in each country.

[0197] Notwithstanding the foregoing, the present disclosure also contemplates embodiments in which a user selectively blocks use of or access to personal information data. That is, the present disclosure contemplates that hardware and / or software elements may be provided to prevent or block access to such personal information data. For example, in the case of a progress tracking service, the present technology may be configured to allow a user to “opt in” or “opt out” of participating in the collection of personal information data during registration for the service or at any time thereafter. In another example, a user may choose not to provide progress tracking information for a particular third-party application. In yet another example, a user may choose to limit the scope of data collected or to completely prohibit the collection of progress tracking information. In addition to providing “opt-in” and “opt-out” options, the present disclosure contemplates providing notifications regarding access or use of personal information. For example, a user may be notified upon downloading an app that will access the user's personal information data, and then again immediately before the app accesses the user's personal information data.

[0198] Furthermore, it is the intent of this disclosure that personal information data should be managed and processed in a manner that minimizes the risk of unintentional or unauthorized access or use. Risk can be minimized by limiting data collection and deleting data when it is no longer needed. Furthermore, where applicable, de-identification of data can be used to protect user privacy in certain health-related applications. De-identification can be facilitated, where appropriate, by removing certain identifiers (e.g., date of birth, etc.), controlling the amount or specificity of data stored (e.g., collecting location data at a city level rather than a street address level), controlling how data is stored (e.g., aggregating data across users), and / or other methods.

[0199] Thus, while this disclosure broadly encompasses the use of personal information data to implement one or more various disclosed embodiments, this disclosure also contemplates that the various embodiments may be implemented without requiring access to such personal information data. That is, various embodiments of the present technology are not rendered inoperable by the absence of all or part of such personal information data. For example, progress tracking may be implemented based on minimal personal information, such as non-personal information data or other non-personal or publicly available information available to the progress tracking pipeline.

[0200] Various aspects, embodiments, implementations, or features of the described embodiments can be used individually or in any combination. Various aspects of the described embodiments can be implemented using software, hardware, or a combination of hardware and software. Furthermore, the described embodiments can be embodied as computer-readable code on a non-transitory computer-readable medium. The non-transitory computer-readable medium is any data storage device that can store data that can thereafter be read by a computer system. Examples of non-transitory computer-readable media include read-only memory, random-access memory, CD-ROMs, HDDs, DVDs, magnetic tape, and optical data storage devices. The non-transitory computer-readable medium can also be distributed over network-coupled computer systems so that the computer-readable code is stored and executed in a distributed fashion.

[0201] In the foregoing description, for purposes of explanation, specific terminology was used to provide a thorough understanding of the described embodiments. However, it will be apparent to those skilled in the art that specific details are not required to practice the described embodiments. Thus, the foregoing descriptions of specific embodiments are presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the described embodiments to the precise forms disclosed. It will be apparent to those skilled in the art that many modifications and variations are possible in light of the above teachings.

Claims

1. 1. A method for managing applications on a client device, the method comprising, by a background process running on the client device: receiving from each of one or more applications on the client device a separate context defining a separate structure of progress tracking information provided by the application relating to activities performed within the application; receiving individual progress tracking information from the one or more applications, the individual progress tracking information being provided by the applications according to the individual context; filtering the progress tracking information based on the individual context to generate filtered progress tracking information, wherein filtering the progress tracking information includes: receiving a list of active contexts associated with the one or more applications; comparing a context identifier included in the progress tracking information to the list of active contexts; discarding the progress tracking information if the context identifier is not included in the list of active contexts, and processing the progress tracking information to generate the filtered progress tracking information if the context identifier is included in the list of active contexts; and displaying information within a user interface that is based at least in part on the filtered progress tracking information; and A method comprising:

2. The method of claim 1 , wherein each application of the one or more applications implements at least a portion of a software framework shared by the background processes.

3. filtering the progress tracking information determining whether progress tracking is enabled or disabled for each of the one or more applications; discarding the progress tracking information if the progress tracking information is received from any application for which progress tracking is disabled, and processing the progress tracking information if the progress tracking information is received from any application for which progress tracking is enabled to generate the filtered progress tracking information; The method of claim 1 , comprising:

4. The method of claim 1 , further comprising storing the progress tracking information in a personal zone of a network-based storage service corresponding to a user account associated with the client device.

5. 10. The method of claim 1, further comprising: generating metadata based on the filtered progress tracking information, the metadata including a reference to the progress tracking information, a user identifier, and a context identifier.

6. receiving, via the user interface, a selection of at least one user interface element corresponding to at least a portion of the filtered progress tracking information; displaying additional information associated with the filtered progress tracking information within the user interface; and The method of claim 1 further comprising:

7. 1. A system comprising: receiving from each of one or more applications on a client device a separate context defining a separate structure of progress tracking information provided by the application relating to activities performed within the application; receiving individual progress tracking information from the one or more applications, the individual progress tracking information being provided by the applications according to the individual context; filtering the progress tracking information based on the individual context to generate filtered progress tracking information, wherein filtering the progress tracking information includes: receiving a list of active contexts associated with the one or more applications; comparing a context identifier included in the progress tracking information to the list of active contexts; discarding the progress tracking information if the context identifier is not included in the list of active contexts, and processing the progress tracking information to generate the filtered progress tracking information if the context identifier is included in the list of active contexts; and displaying information within a user interface that is based at least in part on the filtered progress tracking information; and 1. A system comprising: a client device configured to manage applications that run on them by implementing a background process configured to perform steps including:

8. The system of claim 7 , wherein each application of the one or more applications implements at least a portion of a software framework shared by the background processes.

9. filtering the progress tracking information determining whether progress tracking is enabled or disabled for each of the one or more applications; discarding the progress tracking information if the progress tracking information is received from any application for which progress tracking is disabled, and processing the progress tracking information if the progress tracking information is received from any application for which progress tracking is enabled to generate the filtered progress tracking information; The system of claim 7, comprising:

10. The system of claim 7 , further comprising storing the progress tracking information in a personal zone of a network-based storage service corresponding to a user account associated with the client device.

11. 8. The system of claim 7, further comprising generating metadata based on the filtered progress tracking information, the metadata including a reference to the progress tracking information, a user identifier, and a context identifier.

12. receiving, via the user interface, a selection of at least one user interface element corresponding to at least a portion of the filtered progress tracking information; displaying additional information associated with the filtered progress tracking information within the user interface; and The system of claim 7 further comprising:

13. 1. A client device configured to manage applications executing on the client device, comprising: receiving from each of one or more applications on the client device a separate context defining a separate structure of progress tracking information provided by the application relating to activities performed within the application; receiving individual progress tracking information from the one or more applications, the individual progress tracking information being provided by the applications according to the individual context; filtering the progress tracking information based on the individual context to generate filtered progress tracking information, wherein filtering the progress tracking information includes: receiving a list of active contexts associated with the one or more applications; comparing a context identifier included in the progress tracking information to the list of active contexts; discarding the progress tracking information if the context identifier is not included in the list of active contexts, and processing the progress tracking information to generate the filtered progress tracking information if the context identifier is included in the list of active contexts; and displaying information within a user interface that is based at least in part on the filtered progress tracking information; and 1. A client device comprising: at least one processor configured to implement a background process configured to perform steps including:

14. The client device of claim 13 , wherein each application of the one or more applications implements at least a portion of a software framework shared by the background processes.

15. filtering the progress tracking information determining whether progress tracking is enabled or disabled for each of the one or more applications; discarding the progress tracking information if the progress tracking information is received from any application for which progress tracking is disabled, and processing the progress tracking information if the progress tracking information is received from any application for which progress tracking is enabled to generate the filtered progress tracking information; The client device of claim 13 , comprising:

16. The client device of claim 13 , further comprising storing the progress tracking information in a personal zone of a network-based storage service corresponding to a user account associated with the client device.

17. 14. The client device of claim 13, further comprising: generating metadata based on the filtered progress tracking information, the metadata including a reference to the progress tracking information, a user identifier, and a context identifier.

Images