Systems, methods, and devices for vocational training

Abstract

In one aspect, an example method includes: (a) retrieving unstructured training set data, wherein the unstructured training set data comprises data indicative of one or more vocational activities; (b) transforming the unstructured training set data into structured training set data, wherein transforming the unstructured training set data comprises one or more of: (i) converting one or more attributes of the unstructured training set data and (ii) identifying an anomaly in the unstructured training set data; (c) generating a vocation recommendation model using the one or more machine learning models, wherein the one or more machine learning models are configured to generate the vocation recommendation model using the structured training set data; (d) identifying one or more vocation recommendations, wherein the one or more vocation recommendations are based on at least the generated vocation recommendation model; and (e) transmitting instructions that cause the client computing device to display a graphical representation of the one or more vocation recommendations.

Description

SYSTEMS, METHODS, AND DEVICES FOR VOCATIONAL TRAININGCROSS-FERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of co-pending U.S. Provisional Patent Application Serial No. 63 / 625,187, filed January' 25, 2024, which is hereby incorporated by reference its entirety7.USAGE AND TERMINOLOGY

[0002] In this disclosure, unless otherwise specified and / or unless the particular context clearly dictates otherwise, the terms “a” or “an” mean at least one, and the term “the” means the at least one.SUMMARY

[0003] In one aspect, an example computing system for structuring training set data for one or more machine learning models in is disclosed. The example computing system comprises a client computing device. The example computing system further comprises a modeling computing device, wherein the modeling computing device comprises one or more processors and a non-transitory computer-readable storage medium comprising instructions that cause the one or more processors to perform a set of operations comprising: (a) retrieving unstructured training set data, wherein the unstructured training set data comprises data indicative of one or more vocational activities; (b) transforming the unstructured training set data into structured training set data, wherein transforming the unstructured training set data comprises one or more of: (i) converting one or more attributes of the unstructured training set data and (ii) identifying an anomaly in the unstructured training set data; (c) generating a vocation recommendation model using the one or more machine learning models, wherein the one or more machine learning models are configured to generate the vocation recommendation model using the structured training set data; (d) identifying one or more vocationrecommendations, wherein the one or more vocation recommendations are based on at least the generated vocation recommendation model; and (e) transmitting, to the client computing device, instructions that cause the computing device to display, via the user interface of the client computing device, a graphical representation of the one or more vocation recommendations.

[0004] In another aspect, an example computer-implemented method is disclosed. The method includes (a) retrieving unstructured training set data, wherein the unstructured training set data comprises data indicative of one or more vocational activities; (b) transforming the unstructured training set data into structured training set data, wherein transforming the unstructured training set data comprises one or more of: (i) converting one or more attributes of the unstructured training set data and (ii) identifying an anomaly in the unstructured training set data; (c) generating a vocation recommendation model using the one or more machine learning models, wherein the one or more machine learning models are configured to generate the vocation recommendation model using the structured training set data; (d) identifying one or more vocation recommendations, wherein the one or more vocation recommendations are based on at least the generated vocation recommendation model; and (e) transmitting, to a client computing device, instructions that cause the client computing device to display, via the user interface of the client computing device, a graphical representation of the one or more vocation recommendations.BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 is a simplified block diagram of an example computing device.

[0006] Figure 2 is an example vocation recommendation computing system.

[0007] Figure 3 is an example flowchart for a vocation recommendation system.

[0008] Figure 4 is an example graphical user interface according to an example embodiment.

[0009] Figure 5 is a flow chart of an example method.DETAILED DESCRIPTIONI. Overview

[0010] In the modern-day dynamic and fast-paced labor market, popular attention is often focused on jobs that require a four-year college degree, despite the fact that a majority of American workers do not have a four-year college degree. While programs exist for those without a four-year college degree who wish to further their careers through job training, certification programs, “boot camps,” or other vocational programs, obtaining information about these programs can often be difficult for interested parties.

[0011] Many different public and private entities run vocational programs, from the traditional community college model to other government-run programs, as well as nonprofits and commercial entities. However, due to the broad variety of programs and the multitude of entities that provide them, information regarding those programs can be fragmented. This creates an obstacle to those interested in the vocational programs. For example, if they are only able search for programs that are offered by their local community college, a prospective applicant may miss out on another opportunity that is a better fit for their career goals.

[0012] If, however, there existed a centralized database containing information regarding vocational programs from all of the above entities, then it would be possible to provide an efficient, effective, and novel solution for generating vocation recommendations that take into account the myriad options available to prospective applicants. These recommendations would, in turn, allow for the prospective applicants to make better-informed decisions about their careers and access opportunities that they may have not been aware of previously.

[0013] Accordingly, features of the present disclosure can help to address these and other issues to provide an improvement to select technical fields. Specifically, features ofthe present disclosure help address issues within and provide improvements for select technical fields, which include, for example, computer-based systems for collecting and structuring data from a variety of sources relating to vocational training, thus creating a centralized and organized source of information. Additionally, the present disclosure helps address issues in processing such centralized data for usage in machine learning models to generate recommendations relating to vocational training based upon the collected, structured, and processed data.

[0014] Embodiments of the present invention provide methods, systems, and devices that allow prospective applicants, as well as other parties such as researchers, to better access relevant information regarding vocational training, as well as generate intelligent recommendations for vocational training programs relating to their career goals or other personal characteristics.

[0015] In some examples, a computing system for structuring data for use in one or more machine learning models may be implemented. The computing system may comprise a client computing device to allow user interaction with the data and generated recommendations and a modeling computing device.

[0016] In some examples, the modeling computing device may be configured to perform operations including retrieving unstructured data relating to vocational training. In the context of this disclosure, unstructured data is data that is not optimized for machine readability and processing, but rather human consumption. In some embodiments, this data may be text data on a website that may be retrieved by the modeling computing device using a web crawler. In other embodiments, this data may be in the form of a picture or other image format.

[0017] In some examples, the modeling computing device may retrieve unstructured data, in the variety of forms described above, from many different sources. Forinstance, the unstructured text data could be retrieved from a government database. One such example is the Integrated Postsecondary Education Data System (IPEDS) administered by the United States Department of Education.

[0018] In some examples, the modeling computing device may retrieve the unstructured data in accordance with preferences specified by the user. In order to obtain these user preferences, the modeling computing device may communicate with the client computing device of the computing system in order to allow the user to specify their preferences with regards to the data. In some examples, the client computing device may present a user interface to the user for this purpose.

[0019] For instance, a user may only desire to obtain information and recommendations related to vocational programs run by government entities. Thus, the user interface could present a checklist of available sources of information to the user. If the user selected only government-run programs, the client computing device would thus communicate this preference to the modeling computing device, which would then only retrieve unstructured data relating to vocational programs run by government entities.

[0020] In some examples, the user preferences could include other characteristics, such as location, cost, and duration. For example, a user could set their preference to programs within fifty- miles of their current location, and thus the modeling computing device would only retrieve unstructured data relating to vocational programs within a fifty-mile radius. A similar process would occur for other characten sties specified by the user in their preferences.

[0021] In some examples, the modeling computing device may retrieve the unstructured data according to characteristics and / or criteria that are in addition to and / or alternative to user-provided preferences. Such characteristics and / or criteria may be predetermined and / or pre-defined or otherwise provided to the modeling computing devicebefore a user interacts with the modeling computing device. In some examples, these characteristics and / or criteria may include one or more of: (i) location of a program; (ii) cost of a program; (iii) industry of a program; (iv) format of a program (e.g. in-person, hybrid, or online); (v) the time it takes to complete the program (e.g., months, hours); (vii) age of the average attendee and / or graduate of a program; (viii) educational or other requirements for participation in the vocational program; (ix) expected salary forthose who complete a program; and / or (x) the type of vocational program (e.g. apprenticeship, community college program, certification program, etc.).

[0022] In some examples, the modeling computing device may simply retrieve unstructured data, potentially in bulk, from all available sources and / or a subset of sources that match one or more particular types of content and / or sources from which the content originates (e.g., all publically available community' college website information). In some examples, the modeling computing device may simply retrieve unstructured data, potentially in bulk, from all available sources and / or a subset of sources without consideration for user-defined and / or other criteria.

[0023] After collecting the unstructured data, the next step is for the modeling computing device to transform the unstructured data into structured data such that the data is better-suited for being used by a machine learning model. In some examples, this process may involve converting attributes of the unstructured training data. This may involve, if the unstructured data is numeric, converting each numeric value in the unstructured data set such that they each share a common characteristic. This could be as simple as ensuring that each number is on the same scale or accounting for regional differences in number punctuation (e.g.. the usage of commas versus periods) or a more complex operation such as normalizing the numerical values such that they remaining useful for statistical analysis.

[0024] This normalization with regard to statistical analysis could involve identifying an anomaly within the data, such as an outlier value that could skew the average away from the actual average. The normalization process as part of the overall transformation from unstructured data into structured data could thus include, in some examples, removing outlier values and / or other anomalies in order to ensure statistical accuracy within the dataset.

[0025] Other anomalies identified and removed in some examples could include a redundant word in unstructured text data, such as an accidentally repeated word or otherwise superfluous text.

[0026] In some examples, user preferences could also be incorporated into the transformation step. While this disclosure has previously described how user preferences may be used to determine which data to obtain from a variety of sources, they may also be used to selectively filter the full data set.

[0027] For instance, user preferences could be obtained through the same method described above, through the user interface of the client computing device and then transmitted to the modeling computing device. However, rather than using those preferences to inform the obtaining of the unstructured data, the preferences may be used to inform the transformation process of the unstructured data into structured data for use in the one or more machine learning models.

[0028] In some examples, the modeling computing devices may be configured to only convert unstructured data relating to the user preferences into structured data. In some embodiments, unrelated data may be flagged so that the one or more machine learning model does not make use of it for recommendations or related data may be flagged to ensure that the one or more machine learning models does make use of it. In other embodiments, the modeling computing device may be configured such that the normalization process with regards to statistical analysis and the removal of anomalies will only be performed on unstructured datarelated to the user preferences, to again “filter’' the data to reflect what the user may be interested in or focused on with regards to their career goals.

[0029] In some examples, the user preferences may include characteristics such as location, cost, and duration in relation to vocational training programs.

[0030] In some examples, the unstructured data may be converted according to characteristics and / or criteria that are in addition to and / or alternative to user-provided preferences. Such characteristics and / or criteria may be predetermined and / or pre-defined or otherwise provided to the modeling computing device before a user interacts with the modeling computing device. In some examples, these characteristics and / or criteria may include one or more of: (i) location of a program; (ii) cost of a program; (iii) industry of a program; (iv) format of a program (e.g. in-person, hybrid, or online); (v) the time it takes to complete the program (e.g., months, hours); (vii) age of the average attendee and / or graduate of a program; (viii) educational or other requirements for participation in the vocational program; (ix) expected salary for those who complete a program; and / or (x) the type of vocational program (e g. apprenticeship, community college program, certification program, etc.).

[0031] The next step is to make use of the structured data to produce vocational recommendations. The structured data may in some examples include organization name, program information, affiliated careers, data sources, and support services.

[0032] Using one or more machine learning models, the structured data may be used to generate and train a vocation recommendation model that takes into account the various information that is related to the vocational programs as part of the data collected and centralized in the previous steps. In some examples, the vocation recommendation model may be generated using data related to the user preferences as described above.

[0033] The advantage of training the vocation recommendation model on the structured data rather than the raw unstructured data directly from the sources is that, as statedpreviously, some of the raw data is unsuited for use by machine learning models. By structuring the training data, the accuracy and performance of the generated vocation recommendation model is thus improved.

[0034] Once the vocation recommendation model is generated and trained, it may in some examples identify one or more vocation recommendations based on the data. For instance, a parameter may be set for the model to produce the top ten recommended vocational programs once the generation process has been completed.

[0035] These recommendations may then be presented to the user by way of transmitting a graphical representation of the one or more vocation recommendations to the client computing device which then displays the graphical representation via the user interface. In some examples, the graphical representation may be an image related to the recommendation. The graphical representation may include information that applicants may find important. In some examples, this information may include other characteristics and / or criteria that may include one or more of: (i) location of a program; (ii) cost of a program; (iii) industry of a program; (iv) format of a program (e.g. in-person, hybrid, or online); (v) the time it takes to complete the program (e.g., months, hours); (vii) age of the average attendee and / or graduate of a program; (viii) educational or other requirements for participation in the vocational program; (ix) expected salary forthose who complete a program; and / or (x) the type of vocational program (e.g. apprenticeship, community college program, certification program, etc.).

[0036] Once the user is presented with the information and the recommendation the system outputs, they can make use of these to make better informed decisions about their careers and possible opportunities for more meaningful and better-paid work.

[0037] In some examples, however, the system's process need not end at this point. As stated previously, the current labor market is dynamic and faced-paced, with changesoccurring rapidly and certain skills may be in demand one year and far less in another due to technological progress or other forces. Thus, it is desirable that, for prospective applicants to vocational training programs, they are able to respond to these shifting forces rapidly.

[0038] Following this, it is desirable that the computing system also be able to be dynamic and adapt to the shifting dynamics of the labor market to better connect applicants with relevant vocational programs. Updating the model to incorporate newly-available programs is desirable for the above reasons, as programs themselves and their availability may also change rapidly. The present disclosure implements a technical solution to this problem by allowing for the recommendation model to be updated based on newly available information and verity the existing information if necessary.

[0039] In some examples, the modeling computing device may be configured to retrieve further unstructured data from a variety of sources. For instance, this could be an updated listing of vocational training programs at a community college, as the previous listing is now outdated and the community college has added or removed programs since the last time the modeling computing device obtained unstructured data.

[0040] In order to determine if the further unstructured data differs from the previous data, the modeling computing device may be configured to compare the further unstructured data to the unstructured data obtained previously. If no difference is detected, then no update to the recommendation model is necessary. However, if a difference is detected, then further action may be taken.

[0041] In some examples, this further action may include transforming the further unstructured data into structured data, according to the processes described above with regards to converting and normalization, etc. In some further examples, the transformation process may include converting attributes of the unstructured data or identifying and removing an anomaly in the data.

[0042] Once the further data has been transformed for use with machine learning models, the vocation recommendation model may be updated with the further structured data by using the one or more machine learning models, which may generate and train a new vocation recommendation model based on the newly-acquired data.

[0043] By incorporating new data into the models, the system ensures that users are able to be informed of the latest information regarding training programs that they may be interested in.

[0044] However, it is also desirable that the system be able to take into account the changes in profile information related to the user. For example, a program may only admit applicants with a high school diploma. If an applicant without a high school diploma uses the system as described above, they may not be informed of programs that require one as they do not have the requisite qualifications. However, if the user does obtain their high school diploma, then the system should display updated information and recommendations responsive to the change in qualifications of the user. As described above, the system make take into account user preferences when obtaining or transforming the data, but it is also desirable that the system takes into account profile information related to the user for the above reasons.

[0045] The present disclosure implements a technical solution to the above problem by allowing for the recommendation model to provide personalized vocation recommendations based in part on profile information related to the user.

[0046] In some examples, the user may use the user interface of the client computing device to provide to the system information about themselves, which in other examples may include their skills, certifications, and personally identifying information. Such information may then transmitted to the modeling computing device.

[0047] Once received, this information may then be provided to the generated vocation recommendation model and used to identify vocation recommendations that fit profileinformation related to the user (for instance, their qualifications or obtained certification). After the personalized vocation recommendations are identified, instructions may be transmitted to the client computing device to display, via the user interface, a graphical representation of the personalized vocation recommendations.

[0048] In some examples, the graphical representation may be an image related to the recommendation. The graphical representation may include information that applicants may find important, such as the location, cost, duration, and expected salary for those who complete the program. In some further examples, the personalized vocation recommendations are distinguished from the other vocation recommendations by the personalized vocation recommendations being displayed with a colored border on the user interface of the client computing device.

[0049] These systems, methods, and devices may provide technical advantages of processing unstructured data and generating vocational recommendations through the use of one or more machine learning models. Other features of the systems, methods, and device are described in further detail in the example embodiments provided below.II. Example Architecture and OperationsA. Computing Device

[0050] Figure 1 is a simplified block diagram of an example computing device 100. The computing device 100 can be configured to perform and / or can perform one or more acts and / or functions, such as those described in this disclosure. The computing device 100 can include various components, such as a processor 102, a data storage unit 104, a communication interface 106. and / or a user interface 108. Each of these components can be connected to each other via a connection mechanism 110.

[0051] In this disclosure, the term “connection mechanism” means a mechanism that facilitates communication between two or more components, devices, systems,or other entities. A connection mechanism can be a relatively simple mechanism, such as a cable or system bus, or a relatively complex mechanism, such as a packet-based communication network (e.g., the Internet). In some instances, a connection mechanism can include anon-tangible medium (e.g., in the case where the connection is wireless).

[0052] The processor 102 can include a general-purpose processor (e.g., a microprocessor) and / or a special-purpose processor (e.g., a digital signal processor (DSP)). The processor 102 can execute program instructions included in the data storage unit 104 as discussed below.

[0053] The data storage unit 104 can include one or more volatile, non-volatile, removable, and / or non -removable storage components, such as magnetic, optical, and / or flash storage, and / or can be integrated in whole or in part with the processor 102. Further, the data storage unit 106 can take the form of a non-transitory computer-readable storage medium, having stored thereon program instructions (e.g., compiled or non-compiled program logic and / or machine code) that, upon execution by the processor 102, cause the computing device 100 to perform one or more acts and / or functions, such as those described in this disclosure. These program instructions can define, and / or be part of, a discrete software application. In some instances, the computing device 100 can execute program instructions in response to receiving an input, such as an input received via the communication interface 106 and / or the user interface 108. The data storage unit 104 can also store other types of data, such as those A pes described in this disclosure.

[0054] The communication interface 106 can allow the computing device 100 to connect with and / or communicate with another entity, such as another computing device, according to one or more protocols. In one example, the communication interface 108 can be a wired interface, such as an Ethernet interface. In another example, the communication interface 108 can be a wireless interface, such as a cellular or WI-FI interface. In thisdisclosure, a connection can be a direct connection or an indirect connection, the latter being a connection that passes through and / or traverses one or more entities, such as a router, switch, or other network device. Likewise, in this disclosure, a transmission can be a direct transmission or an indirect transmission.

[0055] The user interface 108 can include hardware and / or software components that facilitate interaction between the computing device 100 and a user of the computing device 100, if applicable. As such, the user interface 108 can include input components such as a keyboard, a keypad, a mouse, a touch-sensitive panel, and / or a microphone, and / or output components such as a display device (which, for example, can be combined with a touch-sensitive panel), a sound speaker, and / or a haptic feedback system.

[0056] The computing device 100 can take various forms, such as a workstation terminal, a desktop computer, a laptop, a tablet, and / or a mobile smartphone. Additionally, as used herein, ’‘mobile computing device’’ describes computing devices that are highly mobile (including a laptop, a tablet, and / or a mobile phone), as well as computing devices that are not as mobile (including a desktop computer, etc.). In a further aspect, the features described herein may involve some or all of these components arranged in different ways, including additional or fewer components and / or different types of components, among other possibilities.B. EXAMPLE VOCATION RECOMMENDATION COMPUTING SYSTEM

[0057] Figure 2 is a vocation recommendation system 200. The vocation recommendation system 200 can perform various acts and / or functions related to collecting unstructured data associated with vocational training and structuring the data in one or more ways that improve the performance and operation of a modeling computing device executing one or more machine learning models. The structured data may be used to generate one or more models and recommend one or more responsive actions to improve vocational training. In this disclosure the term “computing system” means a system that includes at least one computingdevice, such as computing device 100. In some instances, a computing system can include one or more other computing systems.

[0058] It should be readily understood that computing device 100, vocation recommendation system 200, and any of the components thereof, can be physical systems made up of physical devices, cloud-based systems made up of cloud-based devices that store program logic and / or data of cloud based applications and / or services (e.g., for performing at least one function of a software application or an application platform for computing systems and devices detailed herein), or some combination of the two.

[0059] In any event, the vocation recommendation system 200 can include various components, such as a modeling computing device 202 (shown here as a cloud-based computing device), a first data set computing device 204, a second data set computing device 206, a training program computing device 208, and a client computing device 210, each of which can be implemented as a computing system or part of a computing system.

[0060] The vocation recommendation system 200 can also include connection mechanisms (shown here as lines with arrows at each end (i.e., “double arrows”), which connect modeling computing device 202, first data set computing device 204, second data set computing device 206, a training program computing device 208, and a client computing device 210, and may do so in anumber of ways (e.g., a wired mechanism, wireless mechanisms and communications protocols, etc.).

[0061] In practice, the vocation recommendation system 200 is likely to include many of some or all of the example components described above, such as the modeling computing device 202. first data set computing device 204. second data set computing device 206, a training program computing device 208, and a client computing device 210, which can allow many users to communicate and interact with the vocational training program coordinators, potential employers, government agencies, and so on. as well as allow thevocational training program coordinators, potential employers, government agencies to communicate and interact with the user, and so on.

[0062] Other computational actions, displayed messages, audible alerts, visual alerts, and configurations are possible.

[0063] The vocation recommendation system 200 and / or components thereof can perform various acts and / or functions (many of which are described above). Examples of these related features will now be described in further detail.

[0064] The vocation recommendation system 200 may consist of several different components, some examples of which are illustrated in Figure 2. The vocation recommendation system 200 may perform operations in accordance with the techniques described above.

[0065] Specifically, operations may be performed by the modeling computing device 202 relating to the retrieving of unstructured data, transforming of unstructured data into structured data, generating a vocation recommendation model, identifying one or more vocation recommendations based on the generated vocation recommendation model, and transmitting graphical representations of the one or more vocation recommendations to other computing devices.

[0066] In one example, as stated previously, the modeling computing device 202 may be implemented as a cloud-based computing device which connects with other devices within the vocation recommendation system 200 through a number of ways, such as wired or wireless mechanisms and communications protocols.

[0067] In some examples, the modeling computing device 202 may be configured to retrieve unstructured training set data relating to vocational activities. This data may be retrieved from a first data set computing device 204 and / or a second data set computing device 206, depending on the number and sourcing of the data sets retrieved. Further data setcomputing devices are also possible for the modeling computing device 202 to retrieve unstructured data from.

[0068] The modeling computing device 202 may communicate with the data set computing devices 204 and 206 over the Internet, for example making use of an Internetbased communications protocol such as File Transfer Protocol (FTP) or Hypertext Transfer Protocol (HTTP). In some further examples, the data set computing devices 204 and 206 may be web servers. For instance, the modeling computing device 202 may retrieve unstructured data (which may be in the form of text data, picture data, or other data format) from the data set computing devices 204 and 206 using a web crawler.

[0069] The data set computing devices 204 and 206 may host unstructured set data from a variety of sources. For example, they may host a government database relating to vocational training.

[0070] User preferences may also be implemented into the data retrieval process. For instance, in some embodiments, the vocation recommendation system 200 may include a client computing device 210, which may communicate with the modeling computing device 202 through a number of ways as described previously.

[0071] The client computing device 210 may include a user interface that may be configured to collect user preferences relating to vocational training from the user. Such preferences may be transmitted to the modeling computing device 202. In some examples, the modeling computing device 202 may use this information to only retrieve unstructured training set data based on or related to the user preferences. As noted above, however, the modeling computing device 202 may also retrieve unstructured data according to characteristics or criteria that are unrelated to user-provided preferences.

[0072] For instance, the user preferences may include characteristics and / or criteria relating to vocational training that may include one or more of (i) location of aprogram; (ii) cost of a program; (iii) industry of a program; (iv) format of a program (e.g. in- person, hybrid, or online); (v) the time it takes to complete the program (e.g., months, hours); (vii) age of the average attendee and / or graduate of a program; (viii) educational or other requirements for participation in the vocational program; (ix) expected salary for those who complete a program; and / or (x) the ty pe of vocational program (e.g. apprenticeship, community college program, certification program, etc.). In some examples, the modeling computing device 202 may be configured to transform the unstructured training set data into structured training set data. This may involve converting attributes of the unstructured training data and / or identifying anomalies within the unstructured training data.

[0073] For instance, converting attributes of the unstructured training data could involve, if the unstructured data is numeric, converting each numeric value in the unstructured data set such that they each share a common characteristic. This could be as simple as ensuring that each number is on the same scale or accounting for regional differences in number punctuation (e.g., the usage of commas versus periods) or a more complex operation such as normalizing the numerical values such that they remaining useful for statistical analysis.

[0074] One an anomaly value is identified, such as an outlier value that skews averages or other important statistical values of the data set, the normalization process may occur as part of the overall transformation of the unstructured training set data into structured training set data. The normalization process as part of the overall transformation from unstructured data into structured data could thus include, in some examples, removing outher values and / or other anomalies in order to ensure statistical accuracy within the dataset.

[0075] Other anomalies identified and removed in some examples could include a redundant word in unstructured text data, such as an accidentally repeated word or otherwise superfluous text.

[0076] In some examples, user preferences could also be incorporated into the transformation of the unstructured training set data into structured training set data by the modeling computing device 202. For instance, user preferences could be obtained through the same method described above, through the user interface of the client computing device 210 and then transmitted to the modeling computing device 202. As also noted above, the unstructured data may be transformed according to characteristics or criteria that are unrelated to user-provided preferences, for example one or more of: (i) location of a program; (ii) cost of a program; (iii) industry' of a program; (iv) format of a program (e.g. in-person, hybrid, or online); (v) the time it takes to complete the program (e.g., months, hours); (vii) age of the average attendee and / or graduate of a program; (viii) educational or other requirements for participation in the vocational program; (ix) expected salary for those who complete a program; and / or (x) the type of vocational program (e.g. apprenticeship, community college program, certification program, etc.).

[0077] Then, the modeling computing device 202 may use the preferences to selectively convert one or more attributes of the unstructured training set data into structured training set data related to the user preferences. As previously stated, the user preferences may include characteristics relating to vocational training, such as the location, cost, or duration of the specific vocational training program in question.

[0078] In some examples, the modeling computing device 202 may be configured to generate a vocation recommendation model using the one or more machine learning models. The machine learning models may use the structured training data as a basis for generating and training the vocation recommendation model. In some examples, the training of the model may be performed by a training program computing device 208, and the resulting trained model transmitted to the modeling computing device 202.

[0079] In some examples, the modeling computing device 202 may be configured to identify one or more vocation recommendations based on the generated and trained vocation recommendation model. In some examples, the identification of the recommendations may be performed by a training program computing device 208.

[0080] Once the vocation recommendations have been identified, they may be transmitted by the modeling computing device 202 or training program computing device 208 to the client computing device 210 for presentation to the user via a user interface. In some examples, the vocation recommendations may be presented to the user in the form of a graphical representation of the recommendations. In some further examples, the graphical representation may be an image related to the recommendation. The graphical representation may include information that applicants may find important, such as the location, cost, duration, and expected salary for those who complete the program.

[0081] Additionally, the vocation recommendation system 200 may be further configured to update the models based upon new data relating to vocational training. In some examples, the modeling computing device 202 may be configured to retrieve further unstructured data from a variety of sources.

[0082] In order to determine if the further unstructured data differs from the previous data, the modeling computing device 202 may be configured to compare the further unstructured data to the unstructured data obtained previously. If no difference is detected, then no update to the recommendation model is necessary . However, if a difference is detected, then further action may be taken.

[0083] In some examples, this further action may include transforming the further unstructured data into structured data, according to the processes described above with regards to converting and normalization, etc. In some further examples, the transformationprocess may include converting attributes of the unstructured data or identifying and removing an anomaly in the data.

[0084] In further examples, the vocation recommendation system 200 may be configured to provide personalized vocation recommendations based in part on profile information related to the user. In some examples, the user may use the user interface of the client computing device 210 to provide to the system information about themselves, which in other examples may include their skills, certifications, and personally identifying information. Such information may then transmitted to the modeling computing device 202 and / or the training program computing device 208.

[0085] The modeling computing device 202 and / or the training program computing device 208 may provide the profile information related to the user to the generated vocation recommendation model and used to identify vocation recommendations that fit the profile information related to the user. After the personalized vocation recommendations are identified, instructions may be transmitted by the modeling computing device 202 and / or the training program computing device 208 to the client computing device to display, via the user interface, a graphical representation of the personalized vocation recommendations.

[0086] In some examples, the graphical representation may be an image related to the recommendation. The graphical representation may include information that applicants may find important, such as the location, cost, duration, and expected salary for those who complete the program. In some further examples, the personalized vocation recommendations are distinguished from the other vocation recommendations by the personalized vocation recommendations being displayed with a colored border on the user interface of the client computing device 210.

[0087] Figure 3 illustrates a high-level overview of the workflow 300 performed by the vocation recommendation system 200.

[0088] First, unstructured data is retrieved from a variety of sources. Given as examples in Figure 3 are several, including the Integrated Postsecondary Education Data System (IPEDS) administered by the United States Department of Education, the CareerOneStop database, Eligible Training Provider Lists (ETPLs), often administered by state and local public entities, and other harvested data collected by the vocation recommendation system 200.

[0089] Next, one or more machine learning models are used by the modeling computing device 202 to normalize the data, whether for statistical cleanup by removing anomalies or simply scaling the data to ensure better comparisons. Other attributes of the unstructured data may be converted as well.

[0090] Following the normalization, the data is transformed from unstructured into structured data, and may incorporate user preferences to better tailor the data to a specific use case.

[0091] Next, the one or more machine learning models are used to generate a vocation recommendation model, and then training the model on the structured data. The structured data may in some examples include organization name, program information, affiliated careers, data sources, and support services.

[0092] The vocation recommendation model may provide recommendations for a variety of use cases, including the creation of local training databases for regional vocational training providers. The recommendations and assembled data may be used by research organizations, industry organizations, and companies to analyze the direction the market may be heading in terms of the supply of skilled workers. This may be particularly useful for small businesses, for whom labor is a major factor in their success or failure. Lastly, tools directed for use by consumers may also implement these recommendations and data for their own improvement as well.

[0093] Other examples are possible in other embodiments.C. EXAMPLE GRAPHICAL USER INTERFACES

[0094] To further illustrate the above-described concepts and others, Figure 4 depicts a graphical user interface, in accordance with example embodiments. Although illustrated in Figure 4 as being displayed via a user interface of a client computing device (e.g., a mobile computing device), this graphical user interface may be provided for display by one or more components described in connection with vocation recommendation system 200 (e.g., via a user interface of client computing device 210), among other possibilities.

[0095] The information displayed by the graphical user interfaces may also be derived, at least in part, from data stored and processed by the components described in connection with the vocation recommendation system 200, and / or other computing devices or systems configured to generate such graphical user interfaces and / or receive input from one or more users (e.g., those described in connection with vocation recommendation system 200, as well as the components of Figures 1 and 3). This graphical user interface is merely for the purpose of illustration. The features described herein may involve graphical user interfaces that format information differently, include more or less information, include different types of information, and relate to one another in different ways.

[0096] Figure 4 depicts an example graphical user interface 400 in an example state. The graphical user interface 400 may allow a user to interact with the vocation recommendation system 200. In some embodiments, the graphical user interface 400 may be displayed to a user via the client computing device 208.

[0097] A user may perform a variety of operations while interacting with the graphical user interface 400. For example, the aforementioned user preferences may be used to selectively display only those vocation recommendations which meet specific criteria. A user may accomplish this by way of the filter drop-downs 402. As depicted, a user may filtervocation recommendations to meet their preferences regarding which industry they wish to be trained in, the format of the vocational program (e.g. in-person, hybrid, or online), the time it takes to complete (hours), age, educational or other requirements for participation in the vocational program, and the type of vocational program (e.g. apprenticeship, community college program, certification program, etc.). Other filters are also possible in some embodiments.

[0098] A user may also sort the displayed vocational recommendations according to their preferences. For example, using the sorting drop-dow n 404, a user may cause the graphical user interface 400 to display the vocation recommendations in order from the highest-earning to the lowest-earning. In other embodiments, the sorting drop-down 404 may allow the user to sort the vocation recommendations in order from lowest-cost to highest-cost. Other sorting criteria are also possible in some embodiments.

[0099] Additionally, if a user has a specific other preference in mind, the user may use the search bar 406 to enter their own filtering criteria and thus cause the graphical user interface 400 to display only the vocation recommendations that contain the entered criterion (e.g. a keyword).

[0100] The aforementioned interactive elements of the graphical user interface 400 allow for a better and more straightforward user experience for making use of the vocation recommendation system 200 to determine which vocational programs are the best fit for them.

[0101] Further, as described in further detail above, the graphical indications in Figure 4 may vary in real time based, for example, on updated unstructured and / or structured training set data and / or recursively regenerated vocation recommendation models, among other possibilities. Other examples and / or additional information and prompts for display via graphical user interface 400 are possible.

[0102] These example graphical user interfaces are merely for purposes of illustration. The features described herein may involve graphical user interfaces that are configured or formatted differently, include more or less information and / or additional or fewer instructions, include different types of information and / or instructions, and relate to one another in different ways.C. Example Method

[0103] Figure 5 is a flow chart illustrating an example method 500.

[0104] At block 502, the method 500 can include, retrieving unstructured training set data, wherein the unstructured training set data comprises data indicative of one or more vocational activities. In some examples, the unstructured training set data may be retrieved from one or more websites associated with vocational training via a web crawler. In some examples, such data may be in the form of unstructured text data or unstructured picture data. In some further examples, the unstructured training set data may be retrieved from a government database.

[0105] In some examples, retrieving unstructured training set data may involve obtaining user preferences relating to vocational training via a user interface of the client computing device 210, and accordingly retrieving unstructured training set data related to the user preferences. In some examples, the user preferences relating to vocational training may involve characteristics associated with the vocational training such as location, cost, and duration.At block 504, the method 500 can include, transforming the unstructured training set data into structured training set data, wherein transforming the unstructured training set data comprises one or more of: (i) converting one or more attributes of the unstructured training set data and (ii) identifying an anomaly in the unstructured training set data. In some examples, converting one or more attributes of the unstructured training set may involveconverting a numerical data set of the unstructured training set so that all numerical values of the numerical data set share a common characteristic. In some examples, identifying an anomaly in the unstructured training set data may involve identifying a redundancy of a text value of the unstructured training set. In some further examples, transforming the unstructured training set data into structured training set data may involve removing the anomaly from the unstructured training set data.

[0106] In some examples, transforming the unstructured training set data into structured training set data may involve obtaining user preferences relating to vocational training via a user interface of the client computing device 210, and accordingly converting one or more attributes of the unstructured training set data into structured training set data related to the user preferences. In some examples, the user preferences relating to vocational training may involve characteristics associated with the vocational training such as location, cost, and duration.

[0107] At block 506, the method 500 can include generating a vocation recommendation model using the one or more machine learning models, wherein the one or more machine learning models are configured to generate the vocation recommendation model using the structured training set data. In some examples, the one or more machine learning models are configured to generate the vocation recommendation model using the converted structured training set data related to the user preferences.

[0108] At block 508, the method 500 can also include, identifying one or more vocation recommendations, wherein the one or more vocation recommendations are based on at least the generated vocation recommendation model.

[0109] At block 510. the method 500 can also include transmitting, to a client computing device, instructions that cause the client computing device to display, via the user interface of the client computing device, a graphical indication of the one or more vocationrecommendations. In some examples the graphical representation of the one or more vocation recommendations comprises an image related to the one or more vocation recommendations.

[0110] In other example embodiments, the method 500 includes obtaining, from a user via the user interface of the client computing device 210, profile information related to the user, providing, to the vocation recommendation model, the obtained profile information data related to the user. In some examples, the method 500 includes identifying, based on at least the vocation recommendation model, one or more personalized vocation recommendations.[OHl] In some examples, the method 500 includes transmitting, to the client computing device, instructions that cause the computing device to display, via the user interface of the client computing device 210, a graphical representation of the one or more personalized vocation recommendations. In some examples, the profile information related to the user may include skills, certifications, and personally identifying information. In some examples, the personalized vocation recommendations may be displayed via the user interface of the client computing device 210 with a colored border.

[0112] In other example embodiments, the method 500 includes retrieving further unstructured training set data, wherein the unstructured training set data comprises data indicative of one or more vocational activities.

[0113] In some examples, the method 500 includes comparing the further unstructured training set data to the unstructured training set data and, in response to the comparing, transforming the further unstructured training set data into further structured training set data. In some examples, transforming the further unstructured training set data may include converting one or more attributes of the unstructured training set data and / or identifying an anomaly in the further unstructured training set data.

[0114] In some examples, the method 500 includes updating the vocation recommendation model using the one or more machine learning models, wherein the one or more machine learning models are configured to update the vocation recommendation model using the further structured training set data.III. Example Variations

[0115] Although some of the acts and / or functions described in this disclosure have been described as being performed by a particular entity, the acts and / or functions can be performed by any entity, such as those entities described in this disclosure. Further, although the acts and / or functions have been recited in a particular order, the acts and / or functions need not be performed in the order recited. However, in some instances, it can be desired to perform the acts and / or functions in the order recited. Further, each of the acts and / or functions can be performed responsive to one or more of the other acts and / or functions. Also, not all of the acts and / or functions need to be performed to achieve one or more of the benefits provided by this disclosure, and therefore not all of the acts and / or functions are required.

[0116] Although certain variations have been discussed in connection with one or more examples of this disclosure, these variations can also be applied to all of the other examples of this disclosure as well.

[0117] Although select examples of this disclosure have been described, alterations and permutations of these examples will be apparent to those of ordinary skill in the art. Other changes, substitutions, and / or alterations are also possible without departing from the invention in its broader aspects as set forth in the following claims.

Claims

CLAIMSWhat is claimed is:

1. A computing system for structuring training set data for one or more machine learning models, wherein the computing system comprises: a client computing device; and a modeling computing device comprising: one or more processors; and a non-transitory computer-readable storage medium comprising instructions that cause the one or more processors to perform a set of operations comprising: retrieving unstructured training set data, wherein the unstructured training set data comprises data indicative of one or more vocational activities; transforming the unstructured training set data into structured training set data, wherein transforming the unstructured training set data comprises one or more of: (i) converting one or more attributes of the unstructured training set data and (ii) identifying an anomaly in the unstructured training set data; generating a vocation recommendation model using the one or more machine learning models, wherein the one or more machine learning models are configured to generate the vocation recommendation model using the structured training set data; identifying one or more vocation recommendations, wherein the one or more vocation recommendations are based on at least the generated vocation recommendation model; and transmitting, to the client computing device, instructions that cause the client computing device to display, via a user interface of the client computingdevice, a graphical representation of the one or more vocation recommendations .

2. The computing system of claim 1, wherein retrieving the unstructured training set data comprises retrieving, via a web crawler, unstructured text data from one or more websites associated with vocational training.

3. The computing system of claim 1, wherein retrieving the unstructured training set data comprises retrieving, via a web crawler, unstructured picture data from one or more websites associated with vocational training.

4. The computing system of claim 1, wherein retrieving the unstructured training set data comprises retrieving, from a government database, unstructured text data associated with vocational training.

5. The computing system of claim 1 , wherein converting one or more attributes of the unstructured training set comprises converting a numerical data set of the unstructured training set so that all numerical values of the numerical data set share a common characteristic.

6. The computing system of claim 1, wherein identifying an anomaly in the unstructured training set data comprises identifying a redundancy of a text value of the unstructured training set.

7. The computing system of claim 6, wherein the set of operations further comprise, prior to generating the vocation recommendation model using the one or more machine learning models, removing the anomaly from the unstructured training set data.

8. The computing system of claim 1, wherein retrieving unstructured training set data comprises: obtaining, via a user interface of the client computing device, user preferences relating to vocational training; and retrieving unstructured training set data related to the user preferences.

9. The computing system of claim 8, wherein the user preferences relating to vocational training comprise at least one of the following characteristics associated with the vocational training: (i) location; (ii) cost; and (iii) duration.

10. The computing system of claim 1, wherein transforming the unstructured training set data into structured training set data further comprises: obtaining, via a user interface of the client computing device, user preferences relating to vocational training; and converting one or more attributes of the unstructured training set data into structured training set data related to the user preferences.

11. The computing system of claim 10, wherein the user preferences relating to vocational training comprise at least one of the following characteristics associated with the vocational training: (i) location; (ii) cost; and (iii) duration.

12. The computing system of claim 10, wherein the one or more machine learning models are configured to generate the vocation recommendation model using the converted structured training set data related to the user preferences.

13. The computing system of claim 1, wherein the graphical representation of the one or more vocation recommendations comprises an image related to the one or more vocation recommendations.

14. The computing system of claim 1, wherein the set of operations further comprise: obtaining, from a user via the user interface of the client computing device, profile information related to the user; providing, to the vocation recommendation model, the obtained profile information data related to the user; identifying, based on at least the vocation recommendation model, one or more personalized vocation recommendations; and transmitting, to the client computing device, instructions that cause the client computing device to display, via the user interface of the client computing device, a graphical representation of the one or more personalized vocation recommendations.

15. The computing system of claim 14, wherein the profile information related to the user comprises at least one of: skills, certifications, and personally identifying information.

16. The computing system of claim 14, wherein the personalized vocation recommendations are displayed via the user interface of the client computing device with a colored border.

17. The computing system of claim 1, wherein the set of operations further comprise: retrieving further unstructured training set data, wherein the unstructured training set data comprises data indicative of one or more vocational activities; comparing the further unstructured training set data to the unstructured training set data; in response to the comparing, transforming the further unstructured training set data into further structured training set data; and updating the vocation recommendation model using the one or more machine learning models, wherein the one or more machine learning models are configured to update the vocation recommendation model using the further structured training set data.

18. The computing system of claim 17, wherein transforming the further unstructured training set data comprises one or more of: (i) converting one or more attributes of the unstructured training set data and (ii) identifying an anomaly in the further unstructured training set data.

19. A non-transitory computer-readable storage medium comprising instructions that cause one or more processors to perform a set of operations comprising:retrieving unstructured training set data, wherein the unstructured training set data comprises data indicative of one or more vocational activities; transforming the unstructured training set data into structured training set data, wherein transforming the unstructured training set data comprises one or more of: (i) converting one or more attributes of the unstructured training set data and (ii) identifying an anomaly in the unstructured training set data; generating a vocation recommendation model using one or more machine learning models, wherein the one or more machine learning models are configured to generate the vocation recommendation model using the structured training set data; identifying one or more vocation recommendations, wherein the one or more vocation recommendations are based on at least the generated vocation recommendation model; and transmitting instructions that cause a computing device to display, via a user interface of a client computing device, a graphical representation of the one or more vocation recommendations.

20. A computer implemented method comprising: retrieving unstructured training set data, wherein the unstructured training set data comprises data indicative of one or more vocational activities; transforming the unstructured training set data into structured training set data, wherein transforming the unstructured training set data comprises one or more of: (i) converting one or more attributes of the unstructured training set data and (ii) identifying an anomaly in the unstructured training set data; generating a vocation recommendation model using one or more machine learning models, wherein the one or more machine learning models are configured to generate the vocation recommendation model using the structured training set data;identifying one or more vocation recommendations, wherein the one or more vocation recommendations are based on at least the generated vocation recommendation model; and transmitting, to a client computing device, instructions that cause the client computing device to display, via a user interface of the client computing device, a graphical representation of the one or more vocation recommendations.

Images