Techniques for determining product certification eligibility using machine learning

Abstract

The present disclosure relates to systems and methods for using machine learning to evaluate product certification eligibility. According to certain aspects, a server computer may use a set of training data to train a set of machine learning models, where the set of machine learning models may be specific to a product and product certification. The server computer may access product specifications associated with a set of products to be checked for certification and may use an appropriate machine learning model to analyze the product specifications, and the output of the machine learning model may indicate that the set of products is eligible for certification, that the set of products is not eligible for certification, or that the product specifications require further review.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims priority to U.S. Patent Application No. 62 / 933,175, filed on November 8, 2019, the entire disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure relates to determining product certification eligibility. More specifically, this disclosure relates to platforms and techniques for using machine learning to determine whether a given product is eligible for a product certification. Background Art

[0004] Depending on the applicable jurisdiction, type of use, or other factors, consumer - available or commercially - used or commercially - available products may be subject to certain standards, regulations, etc. If a product meets the standards or regulations, the product can be certified. Accordingly, a given product may have one or more certification eligibilities. After a product is certified, it is considered that the product has passed certain performance tests, quality assurance tests, or security tests, and / or meets the eligibility criteria specified in the applicable contracts, regulations, or specifications. Conventionally, to certify a product, an individual associated with an authorized organization must manually review and inspect the product and / or must conduct certain tests on the product before the individual determines whether the product should be certified.

[0005] However, these reviews, inspections, and tests of products are time - consuming and resource - consuming. In addition, when new standards are issued or updated, products subject to these standards may need to be retested or certain steps may need to be taken to determine whether these products are eligible for certification. This increases the research and development costs as well as the costs associated with ensuring that the products meet the standards. In addition, this affects certain components of the local and global supply chains.

[0006] Entities such as certification organizations have an opportunity to adopt various technologies to more accurately and effectively evaluate whether a product is eligible for certification, and entities associated with the product have an opportunity to more efficiently and effectively submit product specifications to be considered when determining whether a product is eligible for certification. Summary of the Invention

[0007] One embodiment of the present invention provides a computer-implemented method for determining product certification eligibility using machine learning. The method may include: training, by a computer processor, a set of machine learning models using a set of training data associated with a set of products, the set of training data including text content and visual content corresponding to the set of products; storing the set of machine learning models in a memory; accessing, by the computer processor, a specification associated with a product, the specification indicating a set of product features and identifying a certification; analyzing, by the computer processor, the specification using a machine learning model from the set of machine learning models applicable to the specification, including determining a set of keywords; and outputting, based on the analysis, an indication of whether the product has certification eligibility by the machine learning model.

[0008] Another embodiment of the present invention provides a system for determining product certification eligibility using machine learning. The system may include a transceiver, a memory storing instructions and data associated with a machine learning model, and a processor interfacing with the transceiver and the memory. The processor may be configured to execute the instructions to cause the processor to: train a set of machine learning models using a set of training data associated with a set of products, the set of training data including text content and visual content corresponding to the set of products; store the set of machine learning models in the memory; access a specification associated with a product, the specification indicating a set of product features and identifying a certification; analyze the specification using a machine learning model from the set of machine learning models applicable to the specification, including determining a set of keywords; and output, based on the analysis, an indication of whether the product has certification eligibility by the machine learning model.

[0009] The present invention also provides a non-transitory computer-readable storage medium having a set of instructions stored thereon, the instructions executable by a processor to determine product certification eligibility using machine learning. The instructions may include: instructions for training a set of machine learning models using a set of training data associated with a set of products, the set of training data including text content and visual content corresponding to the set of products; instructions for storing the set of machine learning models in a memory; instructions for accessing a specification associated with a product, the specification indicating a set of product features and identifying a certification; instructions for analyzing the specification using a machine learning model from the set of machine learning models applicable to the specification, including determining a set of keywords; and instructions for outputting, based on the analysis, an indication of whether the product has certification eligibility by the machine learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The following drawings depict various aspects of the system of the present disclosure. It should be understood that each drawing depicts an embodiment of a particular aspect of the system of the present disclosure, and each drawing is intended to conform to its possible embodiments. Additionally, where possible, the following will be described in conjunction with the reference numerals contained in the drawings, and features depicted in multiple drawings are labeled with consistent reference numerals.

[0011] Figure 1A A general view diagram showing components and entities associated with systems and methods according to some embodiments;

[0012] Figure 1B A general view diagram showing certain components configured to facilitate the systems and methods according to some embodiments.

[0013] Figure 2 An example signal diagram showing functions associated with using machine learning to determine product certification eligibility according to some embodiments;

[0014] Figure 3 An example flowchart showing using machine learning to determine product certification eligibility according to some embodiments. Detailed Description

[0015] If a product has passed certain performance and / or quality assurance tests and / or meets the eligibility criteria specified by standards, regulations, specifications (collectively referred to as "standards"), etc., the product can be certified. A product certification entity (sometimes also referred to as a certification body) can be authorized or compliant with domestic or international standards to ensure the ability to perform product, process, and service certifications. A certification scheme can be written to include performance test methods that must be performed on the product and a set of criteria that the product must meet to obtain certification. When the product certification entity certifies that the product has passed the test (and complies with or meets) the applicable standards, the product certification entity can certify a given product.

[0016] Embodiments of the present disclosure can particularly relate to platforms and techniques for using machine learning to determine the presence or absence of certain certifications for certain products. According to certain aspects, the systems and methods can train a set of machine learning models corresponding to certain products subject to certain standards. Entities associated with products to be searched for certification according to the standards (e.g., product manufacturers) can submit product specifications that include various text and visual contents associated with the product. For example, the specification can be an electronic document (e.g., a PDF document) including a schematic diagram and descriptive content of the product. The systems and methods can use various techniques to extract features associated with the product indicated in the specification from the specification.

[0017] The system and method can use an appropriate machine learning model to analyze the extracted specifications and features, where the appropriate machine learning model can output an indication of whether the product is eligible for certification. If the system and method determine that the product is eligible for certification, the system and method can facilitate product certification and can generate and send an output indicating product certification. Conversely, if the system and method determine that the product is not eligible for certification or the specifications require further review, the system and method can generate and transmit an output indicating that the product has failed certification, or can forward the relevant information to an electronic device for further review of the specifications.

[0018] Accordingly, the system and method have many advantages. Specifically, entities associated with the product can benefit from knowing whether a particular product is eligible for certification, such as during the research and design phase or before the product is introduced to the market, which reduces the costs incurred by the entity. This benefit also applies in cases where new product certifications and standards are issued or existing product certifications and standards are updated. In addition, entities such as safety standards organizations can experience a more effective and efficient (lower cost) product review when determining whether a product is eligible for certification. It should be appreciated that additional benefits can be expected.

[0019] The system and method discussed herein address challenges specific to product certification technology. The challenge involves the difficulty of efficiently determining product eligibility for certification according to established criteria and the difficulty of communicating relevant information regarding the determination of eligibility. This difficulty is particularly evident when a product in the later stages of development does not yet have a specific certification eligibility. Conventionally, authorized organizations manually review products and product specifications to determine eligibility. The system and method provide the ability to address these issues by employing trained machine learning models corresponding to a particular product and a particular certification. Additionally, the system and method use the trained machine learning models to analyze product specifications, and the output of these models indicates whether the product associated with the product specifications is eligible for entity certification. Additionally, the system and method communicate between and among multiple devices and components, and thus the system and method must be attributable to computer technology in order to overcome the above-described deficiencies specifically arising in the field of product certification technology.

[0020] Figure 1A A general overview of system 100 configured to facilitate the components of the system and method is shown. It should be appreciated that system 100 is provided only as an example, and alternative or additional components may be envisioned.

[0021] As Figure 1AAs shown, system 100 may include a set of electronic devices 101, 102, 103. Each of the electronic devices 101, 102, 103 may be any type of electronic device, such as a mobile device (e.g., smartphone), desktop computer, laptop computer, tablet computer, phablet, GPS (Global Positioning System) or GPS-enabled device, smartwatch, smart glasses, smart bracelet, wearable electronic device, PDA (Personal Digital Assistant), pager, computing device configured for wireless communication, etc. In an embodiment, any one of the electronic devices 101, 102, 103 may be an electronic device associated with a person or entity such as a company, enterprise, legal person, etc. (e.g., a server computer or machine).

[0022] The electronic devices 101, 102, 103 may communicate with a server computer 115 via one or more networks 110. In an embodiment, the network 110 may support communication via any standard or technology (e.g., GSM, CDMA, VoIP, TDMA, WCDMA, LTE, EDGE, OFDM, GPRS, EV-DO, UWB, Internet, IEEE 802, including Ethernet, WiMAX, Wi-Fi, Bluetooth, 4G / 5G / 6G, Edge, etc.). The server computer 115 may be associated with an entity such as a company, enterprise, legal person, etc., where the entity may be a certification or security company that can perform security analysis on various products, create product standards, certify products according to the standards, and / or perform other functions.

[0023] The server computer 115 may communicate with one or more data sources 116 via the network 110. In an embodiment, each data source 116 may be associated with a merchant, enterprise, legal person, etc. that can compile, generate, or otherwise access data or information associated with products. For example, one of the data sources 116 may be associated with a company that manufactures electrical wiring. According to an embodiment, each data source 116 may store data or information indicating or describing certain products, as well as the standards that the products may follow, the applicable geographical locations and jurisdictions associated with these standards, and other information.

[0024] The server computer 115 may access, retrieve, or generate a training data set 114 from, for example, one or more of the electronic devices 101, 102, 103, one or more of the data sources 116, and / or a combination of other data sources. According to an embodiment, the training data set 116 may indicate and describe products, including text content and / or visual content that describes and / or illustrates the products. Additionally, as described herein, the data in the training data set 114 may be labeled with certain categories.

[0025] Server computer 115 may employ various machine learning techniques, computations, algorithms, etc. to generate a set of machine learning models using the training data set 114. In particular, server computer 115 may first use the training data set 114 to train the set of machine learning models, and then apply or input a validation set into the generated set of machine learning models to determine which machine learning model is the most accurate or can be used as the final or selected machine learning model.

[0026] According to an embodiment, server computer 115 may input a set of input data (which may be a set of real-world product data) associated with one or more products that may require certification qualifications into the generated machine learning model. In an embodiment, the set of input data may include text content and / or visual content that describes and / or depicts one or more products. The machine learning model may output a result, which may include an indication that the product has certification qualifications, that the product does not have certification qualifications, or that the input set requires further review. A user of electronic devices 101, 102, 103 (e.g., an entity associated with the product) may review the result or output, make a decision, and act accordingly. In an embodiment, the user may access the result or output directly from server computer 115.

[0027] Server computer 115 may be configured to interface with or support a memory or storage 113 capable of storing various data in, for example, one or more databases or other forms of storage. According to an embodiment, memory 113 may store data or information associated with the machine learning models generated by server computer 115. Additionally, server computer 115 may access data associated with the stored machine learning models to input the input set into the machine learning models.

[0028] Although Figure 1A depicted as a single server computer 115, it should be appreciated that server computer 115 may be in the form of a distributed cluster of computers, servers, machines, cloud services, etc. In this embodiment, an entity may utilize the distributed server computer 115 as part of an on-demand cloud computing platform. Accordingly, when electronic devices 101, 102, 103 and data source 116 interface with server computer 115, electronic devices 101, 102, 103 and the data source may actually interface with one or more of several distributed computers, servers, machines, etc. to facilitate the described functionality.

[0029] Although Figure 1AThree (3) electronic devices 101, 102, and 103, one (1) data source 116, and one (1) server computer 115 are depicted, but it should be appreciated that greater or fewer numbers are contemplated. For example, there may be multiple server computers, each associated with a different entity. Figure 1B Depicts specific components associated with the system and method.

[0030] Figure 1B Is an example environment 150 according to an embodiment, in which an input data set 151 is processed into an output data 152 via an authentication analysis platform 155. In one implementation, the input data set 151 may be a training data set. As referred to Figure 1A As described, the authentication analysis platform 155 may be implemented on any computing device, including the server computer 115 (or in some embodiments, one or more of the electronic devices 101, 102, 103). Components of the computing device may include, but are not limited to, a processing unit (e.g., a processor 156), a system memory (e.g., a memory 157), and a system bus 158 that couples various system components including the memory 157 to the processor 156. The computing device may further include various communication components (e.g., transceivers and ports) that facilitate communication of data with one or more additional computing devices.

[0031] In some embodiments, the processor 156 may include one or more parallel processing units capable of processing data in parallel with each other. The system bus 158 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, or a local bus, and may use any suitable bus architecture. By way of example and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus (also known as a mezzanine bus).

[0032] The authentication analysis platform 155 may further include a user interface 153 configured to present content (such as the content of the input data 151 and / or the output data 152 and the information associated therewith). Additionally, the user may make selections on the content via the user interface 153, such as to browse different information, review certain input data, and / or other actions. The user interface 153 may be embodied as part of a touch screen configured to sense the user's touch interactions and gestures. Although not shown in the figure, other system components communicatively coupled to the system bus 158 may include input devices, such as cursor control devices (e.g., mouse, trackball, touchpad, etc.) and keyboards (not shown). A monitor or other type of display device may also be connected to the system bus 158 via an interface (such as a video interface). In addition to the monitor, the computer may also include other peripheral output devices that can be connected via a peripheral output interface (not shown), such as a printer.

[0033] The memory 157 may include various computer-readable media. Computer-readable media can be any available media that can be accessed by a computing device and can include volatile media and non-volatile media as well as removable media and non-removable media. By way of non-limiting example, computer-readable media may include computer storage media, which may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, routines, applications (such as the authentication analysis application 160), data structures, program modules, or other data.

[0034] Computer storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store the required information and can be accessed by the processor 156 of the computing device.

[0035] The authentication analysis platform 155 may operate in a networked environment and communicate with one or more remote platforms, such as the remote platform 165, via a network 162, such as a local area network (LAN), a wide area network (WAN), a telecommunications network, or other suitable network. The remote platform 165 may be implemented on any computing device, including one or more of the electronic devices 101, 102, 103 referred to Figure 1A above, and may include many or all of the elements described above with respect to the platform 155. In some embodiments, as an alternative or supplement to the platform 155, the authentication analysis application 160 may be stored and executed by the remote platform 165.

[0036] The authentication analysis application 160 can adopt machine learning techniques, such as regression analysis (e.g., logistic regression, linear regression, random forest regression, probabilistic regression, or polynomial regression), classification analysis, k-nearest neighbor, decision tree, random forest, boosting algorithms, neural network, support vector machine, deep learning, reinforcement learning, Bayesian network, etc. When the data 151 is a training data set, the authentication analysis application 160 can analyze / process the data 151 to generate a machine learning model for storage as part of the model data 163 that can be stored in the memory 157.

[0037] When the data 151 includes data associated with product specifications to be analyzed using the machine learning model, the authentication analysis application 160 can use the machine learning model to analyze or process the data 151 to generate output data 152, which can indicate various results obtained from the analysis using the machine learning model. The memory 157 can be configured to store various authentications, standards, and submission data 164 that the authentication analysis platform 155 can use to generate the machine learning model or to perform analysis using the machine learning model.

[0038] The authentication analysis application 160 (or another component) can cause the output data 152 (in some cases, cause the training or input data 151) to be displayed on the user interface 153 for review by the user of the authentication analysis platform 155. The user can select to review and / or modify the displayed data. For example, the user can review the output data 152 to evaluate the results of the product submission.

[0039] Generally, a computer program product according to an embodiment can include a computer-usable storage medium (e.g., a standard random access memory (RAM), an optical disc, a universal serial bus (USB) drive, etc.) embedded with computer-readable program code, where the computer-readable program code is adapted to be executed by the processor 156 (e.g., work in cooperation with an operating system) to facilitate the functions described herein. In this regard, the program code can be implemented in any desired language and can be implemented as machine code, assembly code, byte code, interpretable source code, etc. (e.g., through Golang, Python, Scala, C, C++, Java, Actionscript, Objective-C, Javascript, CSS, XML, R, Stata, AI libraries). In some embodiments, the computer program product can be part of a resource cloud network.

[0040] In some embodiments, a computer program product can be part of a resource cloud network. Generally, each of Data 151 and Data 152 can be embodied as any type of electronic document, file, template, etc., which can include various text contents and can be stored in a memory as program data in a hard disk drive, a magnetic disk, and / or an optical disk drive in the authentication analysis platform 155 and / or the remote platform 165.

[0041] Figure 2 is a signal diagram 200 depicting various functions associated with the system and method. The signal diagram 200 includes a submission device 220 associated with a person or entity (such as one of the electronic devices 101, 102, 103 referred to Figure 1A as described), a server computer 215 (such as the server computer 115 referred to Figure 1A as described), and a data source 216 (such as one of the data sources 116 referred to Figure 1A as described).

[0042] The signal diagram 200 can begin with the data source 216 transmitting a training data set and product certification / standard data to the server computer 215. In an embodiment, the training data set can be associated with a set of products, and each product can meet the standards and / or be eligible for certification. For each product in the set of products, the training data set can include text information or content and / or visual information or content, and their combination can be referred to as product specifications. It should be appreciated that the server computer 215 can access at least a portion of the training data and the product certification data set locally or from one or more other data sources.

[0043] Specifically, the text information can include a product description, including materials, dimensions, construction or assembly, installation, intended use, and / or other similar information. In an embodiment, the text information can include information associated with applicable standards or certifications. The resulting standards and certifications vary based on location and / or jurisdiction, so the text data can alternatively or additionally include the location and / or jurisdiction that the server computer 215 can use to identify the standards or certifications applicable to the underlying product.

[0044] Additionally, the visual information can include a set of diagrams, photos, images, schematic diagrams, floor plans, blueprints, and / or similar drawings that can visually depict the product. Any of the visual information can also include the text information contained therein. For example, a schematic diagram of a product can include dimensions, descriptions, keywords, and / or similar information. It should be appreciated that the training data set can be embodied in one or more formats of electronic files that can be read by the server computer 215.

[0045] Additionally or alternatively, the data transmitted by the data source 216 can include information associated with the standards that the base product may follow. In particular, the data can identify any applicable standards, the language or script of the standard, the applicable geographical location or jurisdiction of the standard, and / or other information.

[0046] The specifications of a given product can also include one or more labels indicating whether the product is eligible for certification. According to the embodiments described herein, the labels can include "eligible", "not eligible", and "under review". For example, a schematic diagram of a razor can include the label "eligible". It should be appreciated that alternative and additional labels can be envisioned and the labels can be modified, added, or reduced. It should be appreciated that different information or documents contained in a given product specification can have different labels. For example, a first schematic diagram of an LED lamp can include the label "eligible", while a second schematic diagram of the same LED lamp can include the label "not eligible".

[0047] The server computer 215 can use the training data and the product standard / certification data set to train a set of machine learning models (224). In an embodiment, the server computer 215 can train different machine learning models corresponding to different products with different certification qualifications. For example, the server computer 215 can train a first machine learning model for an LED lamp with a first certification qualification in the United States and can train a second machine learning model for the same LED lamp with a second certification qualification in Europe. It should be appreciated that the server computer 215 can use various types of techniques, algorithms, computations, etc. to train the set of machine learning models (such as regression analysis (e.g., logistic regression, linear regression, random forest regression, probabilistic regression, polynomial regression), classification analysis, k-nearest neighbor, decision tree, random forest, boosting algorithms, neural network, support vector machine, deep learning, reinforcement learning, Bayesian network, etc.).

[0048] In addition to training the set of machine learning models, the server computer 215 can also additionally access (e.g., from the data source 216), generate, or compile a list of keywords associated with the products for which the machine learning models have been associated, where each machine learning model can have an associated list. According to the embodiments, each list can include keywords of various categories. For example, the categories can be "eligible", "not eligible", or "under review", where each category can have zero, one, or more keywords. It should be appreciated that the list of keywords can be manually compiled by a security standards organization or can be automatically generated by the server computer 215. In particular, the server computer 215 can use the training data of a given product and certification to automatically generate a list of keywords and their categories.

[0049] According to an embodiment, an entity associated with the submission device 220 may desire to determine whether a particular product complies with or is eligible for a certain certification associated with standards to which the particular product may be subject in the market. Accordingly, the submission device 220 may submit a product specification (226) to the server computer 215, where the product specification may be associated with the particular product. In an embodiment, the submission device 220 may enable a user to input information associated with the product specification via a form, questionnaire, upload, or other form of submission.

[0050] The product specification may include or identify various information associated with the product, such as including an identification and description of the product, a set of illustrations or diagrams (usually visual content) depicting the product, a geographical location corresponding to the intended market of the product, and / or jurisdiction and / or other information. At least a portion of the information (such as geographical location and / or jurisdiction) may be embodied as metadata, such as metadata captured via an entry on the submission device 220.

[0051] The server computer 215 may analyze the specification (228) using a machine learning model applicable to the product. The server computer 215 may first identify which machine learning model in a set of trained machine learning models is applicable to the product. In particular, the applicable machine learning model may match the product identification and the applicable geographical location or jurisdiction specified in the product specification. For example, if the product specification identifies electrical wiring for an electrical appliance in a UK jurisdiction, the server computer 215 may identify the machine learning model corresponding to the electrical wiring for the electrical appliance in the UK jurisdiction.

[0052] When analyzing the specification using the applicable machine learning model, the server computer 215 may perform one or more analyses. In particular, the server computer 215 may perform an optical character recognition (OCR) analysis on the information contained in the specification to identify a set of words, phrases, and / or terms that may be included in the information. It should be appreciated that the server computer 215 may perform OCR on any text content or visual content contained in the information.

[0053] Additionally or alternatively, the server computer 215 may perform a visual analysis technique on any visual content contained in the information to determine or identify terms, keywords, dimensions, materials, or other aspects associated with the product depicted in the visual content. For example, when performing a visual analysis technique on a diagram depicting electrical wiring, the server computer 215 may determine the composition, material, and length of the electrical wiring. It should be appreciated that when analyzing the information, the server computer 215 may employ various image analysis and OCR techniques, computations, algorithms, and the like.

[0054] Generally, using a machine learning model to analyze specifications in conjunction with the server computer 215 can result in a set of keywords associated with the product specifications. Additionally, the server computer 215 can access a keyword list associated with authentication and can compare the determined set of keywords with the keyword list, where the comparison can be performed step by step.

[0055] First, the server computer 215 can determine (230) whether any ineligible keywords are included in the determined set of keywords. In particular, the server computer 215 can determine whether any keyword in the determined set of keywords matches any keyword specified as belonging to the "ineligible" category in the keyword list associated with authentication. For example, one of the ineligible keywords for the authentication associated with electrical wiring can be "extra". It should be appreciated that if any (i.e., one or more) of the determined keywords are ineligible, or if a threshold number (e.g., at least five (5)) of the determined keywords are ineligible, the determination can be affirmative (i.e., "yes"). If the server computer 215 determines that there is one or more ineligible keywords ("yes"), the server computer 215 can consider the product described and depicted in the specification as ineligible for authentication, and the process can continue to (240), or other functions.

[0056] If the server computer 215 determines that there are no ineligible keywords (or if the number of ineligible keywords does not meet or exceed the threshold number) ("no"), the server computer can determine (232) whether the specification requires review. In particular, the server computer 215 can determine whether any keyword in the determined set of keywords is a "trigger" keyword specified in the corresponding "trigger" list category. For example, one of the trigger keywords for the authentication associated with electrical wiring can be "harness". It should be appreciated that if any (i.e., one or more) of the determined keywords are trigger keywords, or if a threshold number (e.g., at least five (5)) of the determined keywords are trigger keywords, the determination can be affirmative (i.e., "yes"). If the server computer 215 determines that there is one or more trigger keywords ("yes"), the server computer 215 can consider the product described and depicted in the specification as requiring further review, and the process can continue to (238), or other functions.

[0057] If the server computer 215 determines that there are no trigger keywords (or if the number of trigger keywords does not meet or exceeds a threshold number) ("No"), then the server computer can determine (234) whether the product is eligible for certification. In particular, the server computer 215 can determine whether any of the keywords in the determined set of keywords is an "eligible" keyword as specified in the corresponding "eligible" list category. For example, one of the eligible keywords for certification related to electrical wiring can be "grounding". It should be appreciated that if any (i.e., one or more) of the determined keywords is an eligible keyword, or if a threshold number (e.g., at least five (5)) of the determined keywords is an eligible keyword, then the determination can be affirmative (i.e., "Yes"). If the server computer 215 determines that there is one or more eligible keywords ("Yes"), then the server computer 215 can consider the product described and depicted in the specification to be eligible for certification, and the process can continue to (240), or other functions.

[0058] If the server computer 215 determines that there are no eligible keywords (or if the number of eligible keywords does not meet or exceeds a threshold number) ("No"), then the server computer can determine (236) whether there are any keywords included in the determined set of keywords (i.e., whether the determined set of keywords is an empty set). If the server computer 215 determines that a keyword is found ("Yes"), then the process can continue to (238), or other functions. If the server computer 215 determines that no keyword is found ("No"), then the process can continue to (240), or other functions.

[0059] At (238), the server computer 215 can perform a review of the product specification. In some embodiments, the server computer 215 can automatically perform the review without user intervention to attempt to reconcile or clarify any detected trigger words and / or determine the reason for the lack of eligible keywords. In other embodiments, the user can access the server computer 215 to review the product specification and optionally determine whether the product is eligible for certification. Based on the review, the server computer 215 can determine whether the product is eligible or ineligible for certification.

[0060] At (240), the server computer 215 can notify the submission device 220 of the analysis result. In particular, the server computer 215 can generate a notification or other type of electronic communication that indicates the results of the eligibility determination functions performed at (230), (232), (234), and (236), and transmit the notification to the submission device 220. It should be appreciated that the server computer 215 can facilitate various functions that may lead to the product passing certification (e.g., communicating with a safety standards organization).

[0061] Additionally, if a product is deemed ineligible for certification, the server computer 215 can determine, based on the analysis, a set of changes that may need to be applied to or implemented on the product to render it eligible for certification. In such a scenario, the server computer 215 can communicate with the manufacturer or other entity associated with the product, indicating the set of changes that may need to be made.

[0062] Figure 3 A block diagram depicting an example method 300 for determining product certification eligibility using machine learning. Method 300 can be facilitated by an electronic device (such as server computer 115 or components associated with the certification analysis platform 155 as described with reference to Figure 1B and capable of communicating with additional devices and / or data sources).

[0063] Method 300 can begin with the electronic device training a set of machine learning models using a set of training data associated with a set of products (block 305). In an implementation, the set of training data can include text content and visual content corresponding to the set of products. The set of training data can further explain the set of certifications for which the set of products may be eligible. The electronic device can store the set of machine learning models in a memory (block 310), where the set of machine learning models corresponds to the set of products and the set of certifications.

[0064] The electronic device can access the specifications associated with the product (block 315), where the specifications can indicate a set of product features and where the product is eligible for certification. According to an implementation, the specifications can indicate a region or jurisdiction, and the electronic device can access the machine learning model in the set of machine learning models that is applicable to the product and the region or jurisdiction.

[0065] The electronic device can use the machine learning models in the set of machine learning models to extract at least one of a set of text content or a set of visual content from the specifications (block 320). Additionally, the electronic device can use the machine learning models to determine a set of keywords based on at least one of the set of text content or the set of visual content (block 325). In an implementation, the electronic device can use the machine learning models to perform OCR techniques on the set of text content to determine the set of keywords and / or can perform visualization analysis techniques on the set of visual content to determine the set of keywords.

[0066] The electronic device can access a keyword list associated with the certification, where the keyword list can indicate various keyword categories (such as "ineligible", "under review", or "trigger" and "eligible"). Additionally, the electronic device can determine whether any ineligible keywords are included in the determined set of keywords (diamond 330). In an implementation, the determination can be affirmative (i.e., "yes") if the determined set of keywords includes at least one or at least a threshold number of ineligible keywords.

[0067] If the electronic device determines that there is an ineligible keyword ("yes"), the process can continue to block 345. If the electronic device determines that there is no ineligible keyword ("no"), the electronic device can determine whether the specification needs to be further reviewed to make the product eligible for certification (block 335). In particular, if at least one keyword in the keyword set is a trigger keyword for certification, the specification may need to be further reviewed. If the electronic device determines that the specification needs to be further reviewed ("yes"), the process can continue to block 345.

[0068] If the electronic device determines that the specification does not need to be further reviewed ("no"), the electronic device can determine whether the product is eligible for certification (block 340). In particular, if the keyword set (i) does not include any ineligible keyword or trigger keyword, and (ii) includes at least one eligible keyword, the electronic device can determine that the product is eligible for certification. Regardless of the result of block 340 ("yes" or "no"), the process can continue to block 345.

[0069] At block 345, the electronic device can output an indication of whether the product is eligible for certification through a machine learning model. In particular, based on the determinations in blocks 330, 335, and 340, the electronic device can output an indication that the product is not eligible for certification, is eligible for certification, or the specification needs to be further reviewed to make the product eligible for certification. In an implementation, the electronic device can facilitate the review of the specification.

[0070] Although specific descriptions of many different implementations are set forth herein, it should be understood that the scope of protection of the present invention can be defined by the terms in the claims appended to this patent. This specific description is to be construed as exemplary only and not as describing every possible implementation, because it would be impractical to describe every possible implementation even if it were possible. Many alternative implementations can be achieved using current technology or technology developed after the filing date of this patent application, and these still fall within the scope of protection of the claims.

[0071] In this specification, multiple instances can implement components, operations, or structures described as a single instance. Although the individual operations in one or more methods are illustrated and described as separate operations, one or more of the individual operations can be performed simultaneously without requiring the operations to be performed in the order shown. Structures and functions presented as separate components in an example configuration can be implemented as a combined structure or component. Similarly, structures and functions presented as a single component can be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of protection of the subject matter of this disclosure.

[0072] In addition, certain embodiments are described herein as including logic or several routines, sub-routines, applications, or instructions. They can constitute software (e.g., code contained on a non-transitory computer-readable medium) or hardware. The hardware, routines, etc. are tangible units capable of performing certain operations and can be configured or arranged in a certain manner. In an example embodiment, one or more computer systems (e.g., stand-alone client or server computer systems) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) can be configured by software (e.g., an application or a part of an application) to operate as a hardware module for performing certain operations as described herein.

[0073] In various embodiments, the hardware module can be implemented mechanically or electronically. For example, the hardware module can include dedicated circuitry or logic (e.g., a dedicated processor, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC)) permanently configured to perform certain operations. The hardware module can also include programmable logic or circuitry (e.g., contained within a general-purpose processor or other programmable processor) temporarily configured by software to perform certain operations. It should be appreciated that the decision to implement the hardware module mechanically in permanently configured dedicated circuitry or in temporarily configured circuitry may be influenced by cost and time considerations.

[0074] Accordingly, the term "hardware module" should be understood to include a physical entity that is constructed, permanently configured (e.g., hardwired) or temporarily configured (e.g., programmed) to operate or perform certain operations as described herein in a certain manner. Considering embodiments where the hardware module is temporarily configured (e.g., programmed), each hardware module need not be configured or instantiated at any given time period. For example, in the case where the hardware module includes a general-purpose processor configured by software, the general-purpose processor can be configured into different hardware modules at different times. The software can accordingly configure the processor, for example, to construct a particular hardware module at one time period and different hardware modules at different time periods.

[0075] The hardware module can provide information to and receive information from other hardware modules. Accordingly, the hardware modules can be regarded as communicatively coupled. In the case where multiple such hardware or software modules coexist, communication between the software modules can be achieved through signal transmission (e.g., via appropriate circuitry and buses). In embodiments where multiple hardware modules are configured or instantiated at different times, communication between such hardware modules can be achieved, for example, by storing and retrieving information in a storage structure accessible to the multiple hardware modules. For example, one hardware module can perform an operation and store the output of the operation in a storage device communicatively coupled thereto. Another hardware module can then access the storage device at a later time to retrieve and process the stored output. The hardware module can also initiate communication with an input or output device and can operate on resources (e.g., collect information).

[0076] The various operations in the example methods described herein can be performed, at least in part, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can be constructed to implement processor-implemented modules that perform one or more operations or functions. The modules referred to herein can include processor-implemented modules in some example implementations.

[0077] Similarly, the methods or routines described herein can be implemented, at least in part, by a processor. For example, at least some of the operations in a method can be performed by one or more processors or processor-implemented hardware modules. Performing certain operations can be assigned to one or more processors that are not only within a single machine but are distributed across several machines. In some example implementations, one or more processors can be located in a single location (e.g., within a home environment, an office environment, or a server farm), while in other implementations, multiple processors can be distributed across several locations.

[0078] Performing certain operations can be assigned to one or more processors that are not only within a single machine but are distributed across several machines. In some example implementations, one or more processors or processor-implemented modules can be located in a single geographical location (e.g., within a home environment, an office environment, or a server farm). In other example implementations, one or more processors or processor-implemented modules can be distributed across several geographical locations.

[0079] Discussions herein using terms such as "processing," "computing," "calculating," "determining," "presenting," "displaying," etc., can refer to actions or processes of a machine, such as a computer, that manipulates or transforms data represented as physical quantities (e.g., electronic, electromagnetic, or optical quantities) within one or more memories (e.g., volatile memory, non-volatile memory, or a combination thereof), registers, or other machine components that receive, store, transmit, or display information, unless otherwise specified.

[0080] As used herein, any reference to "an implementation" or "implementations" means that the particular elements, features, structures, or characteristics described in connection with the implementation can be included in at least one implementation. The phrase "in an implementation" that appears throughout this specification is not necessarily all referring to the same implementation.

[0081] As used herein, the terms "comprises," "comprising," "has," or any other variation is intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Further, the term "or" as used herein refers to an inclusive "or" and not an exclusive "or," unless stated otherwise explicitly. For example, the condition A or B is satisfied in any of the following relationships: A is true (or present) and B is false (or absent); A is false (or absent) and B is true (or present); and both A and B are true (or present).

[0082] Furthermore, the indefinite article "a" is used to describe elements and components of the embodiments described herein. This is done for convenience only and to give a general meaning to the description. This description should be read to include one or at least one, and the singular also includes the plural, unless expressly stated otherwise.

[0083] This detailed description should be construed as illustrative, and not as describing every possible embodiment, as it would be impractical to describe every possible embodiment.

Claims

1. A computer-implemented method for using machine learning to determine product certification eligibility, the method comprising: Using a computer processor to train multiple machine learning models using a training data set associated with a product set, the training data set including (i) text content and visual content corresponding to the product set, and (ii) multiple keywords associated with the product set, wherein each of the multiple keywords is labeled as one of multiple categories; Storing the set of machine learning models in a memory; Accessing, by the computer processor, specifications identifying electrical wiring, the specifications (i) indicating a set of characteristics of the electrical wiring, (ii) identifying a certification, (iii) including visual content, and (iv) indicating a region or jurisdiction corresponding to the intended market for the electrical wiring, wherein the certification applies to the region or jurisdiction; Identifying, by the computer processor, one machine learning model from the multiple machine learning models that corresponds to (i) the set of characteristics of the electrical wiring and (ii) the region or jurisdiction corresponding to the intended market for the electrical wiring; Performing, by the computer processor, optical character recognition (OCR) technology on the visual content included in the specifications to determine a set of keywords associated with the specifications; Analyzing, by the computer processor, the set of keywords using the machine learning model, wherein each keyword in the set of keywords has a type of ineligible keyword, trigger keyword, or eligible keyword; Based on the analysis: Outputting, by the machine learning model, an indication that the electrical wiring is not eligible for certification, wherein The indication is at least based on the type of each keyword in the set of keywords, and Determining a set of changes to be implemented in the electrical wiring that result in the electrical wiring being eligible for certification; Automatically performing, by the computer processor, a review without user intervention to reconcile or clarify any set of keywords of the trigger keyword type; and Determining, by the computer processor, the reason that there is no keyword of the eligible keyword type in the set of keywords.

2. The computer-implemented method according to claim 1, further comprising: Determining that at least one keyword in the set of keywords is an ineligible certification keyword.

3. A system for using machine learning to determine product certification eligibility, comprising: A transceiver; A memory that stores instructions and data associated with a machine learning model; And A processor that interfaces with the transceiver and the memory and is configured to execute the instructions and cause the processor to: Train multiple machine learning models using a training data set associated with a product set, the training data set including (i) text content and visual content corresponding to the product set, and (ii) multiple keywords associated with the product set, wherein each of the multiple keywords is labeled as one of multiple categories, Store the set of machine learning models in the memory, Access the specification identifying the electrical wiring, where the specification (i) indicates a set of characteristics of the electrical wiring, (ii) identifies the certification, (iii) includes visual content, and (iv) indicates the region or jurisdiction corresponding to the intended market of the electrical wiring, where the certification applies to the region or jurisdiction, Identify one machine learning model among the multiple machine learning models, where the machine learning model corresponds to (i) the set of characteristics of the electrical wiring, and (ii) the region or jurisdiction corresponding to the intended market of the electrical wiring, Perform optical character recognition (OCR) technology on the visual content included in the specification to determine a set of keywords associated with the specification, Analyze the set of keywords using the machine learning model, where each keyword in the set of keywords has a type of ineligible keyword, trigger keyword, or eligible keyword, Based on the analysis: Output, through the machine learning model, an indication that the electrical wiring is not eligible for certification, where The indication is at least based on the type of each keyword in the set of keywords, and Determine a set of changes to be implemented in the electrical wiring, where the set of changes results in the electrical wiring being eligible for certification, Automatically perform a review without user intervention to reconcile or clarify any set of keywords of the trigger keyword type, and Determine the reason that there is no keyword of the eligible keyword type in the set of keywords.

4. The system according to claim 3, wherein, The processor is further configured to: Determine that at least one keyword in the set of keywords is an ineligible keyword.

5. A non-transitory computer-readable storage medium storing a set of instructions executable by a processor to determine product certification eligibility using machine learning, the instructions including: Instructions for training multiple machine learning models using a set of training data associated with a set of products, the set of training data including (i) text content and visual content corresponding to the set of products, and (ii) a set of multiple keywords associated with the set of products, where each of the multiple keywords is labeled as one of multiple categories; Instructions for storing the set of machine learning models in a memory; Instructions for accessing the specification identifying the electrical wiring, where the specification (i) indicates a set of characteristics of the electrical wiring, (ii) identifies the certification, (iii) includes visual content, and (iv) indicates the region or jurisdiction corresponding to the intended market of the electrical wiring, where the certification applies to the region or jurisdiction; Instructions for identifying one machine learning model among the multiple machine learning models, where the machine learning model corresponds to (i) the set of characteristics of the electrical wiring, and (ii) the region or jurisdiction corresponding to the intended market of the electrical wiring; Instructions for performing optical character recognition (OCR) technology on the visual content included in the specification to determine a set of keywords associated with the specification; Instructions for analyzing the set of keywords using the machine learning model, where each keyword in the set of keywords has a type of ineligible keyword, trigger keyword, or eligible keyword; Instructions for performing the following operations based on the analysis: Output an indication that the electrical wiring is not certified by the machine learning model, where the indication is at least based on the type of each keyword in the set of keywords, and Determine a set of changes to be implemented in the electrical wiring, the set of changes causing the electrical wiring to be certified; Instructions for automatically performing a review without user intervention to reconcile or clarify any set of keywords of the trigger keyword type; And Instructions for determining the reason that there is no keyword of the eligible keyword type in the set of keywords.

Images