Process for creating a fixed length representation of a variable length input

Abstract

A computer system identifies that a first markup language portion extracted from a markup language document of a website corresponds to a first actionable element, where the first markup language portion is a variable length representation. In response to the identification that the first markup language portion corresponds to the first actionable element, the computer system creates a first code representation corresponding to the first markup language portion using a recurrent neural network (RNN) encoder. The computer system identifies first additional information corresponding to one or more predefined targets. The computer system creates a final fixed length markup language representation that includes the first code representation and the first additional information. The computer system inputs the final fixed length markup language representation into a model.

Description

Technical Field

[0001] This disclosure relates to autoencoders, and more specifically to training and utilizing autoencoders to create fixed-length representations of variable-length inputs. Background Technology Background Technology

[0003] The Web represents a vast data source for many companies developing meaningful insights for risk assessment, marketing, and other business purposes. In many cases, companies rely on machine learning algorithms to extract these insights from the collected data. However, machine learning algorithms typically require structured input data; therefore, using data from the Web as input for machine learning algorithms can be problematic because website content is represented in HTML, a text-based syntax that is unstructured and variable in length, making it poorly evaluated. Creating a solution that conveniently and easily utilizes Web data as input for machine learning algorithms would be beneficial. Attached Figure Description

[0004] Figure 1 The figure illustrates an automatic encoder system according to one embodiment.

[0005] Figure 2 and Figure 3 The illustration is based on one embodiment. Figure 1 The conversion process during training Figure 1 The flowchart shows the operation of the automatic encoder.

[0006] Figure 4 The illustration is for training according to one embodiment. Figure 1 The flowchart of the automatic encoder process.

[0007] Figure 5 The illustration is for training according to one embodiment. Figure 1 The description of the process of an autoencoder through multiple specific iterations.

[0008] Figure 6 The illustration is based on one embodiment. Figure 1 The conversion process utilizes the training data. Figure 1 The flowchart shows the process of creating a fixed-length representation from a variable-length markup language fragment in an autoencoder.

[0009] Figure 7 The illustration is based on one embodiment. Figure 1 The flowchart shows the process of creating a fixed-length representation from variable-length markup language fragments to be input into the model.

[0010] Figure 8 This describes an embodiment. Figure 1 A block diagram of the hardware components of an automatic encoder. Detailed Implementation

[0011] Embodiments of this disclosure provide a system, method, and program product. A computer system identifies a first markup language portion extracted from a markup language document of a website corresponding to a first operable element, wherein the first markup language portion is a variable-length representation. In response to identifying the first markup language portion corresponding to the first operable element, the computer system utilizes a recurrent neural network (RNN) encoder to create a first code representation corresponding to the first markup language portion. The computer system identifies first additional information corresponding to one or more predefined targets. The computer system creates a final fixed-length markup language representation that includes the first code representation and the first additional information. The computer system inputs the final fixed-length markup language representation into a model.

[0012] Furthermore, in response to recognizing that the first markup language portion corresponds to the first operable element, the computer system creates a first embedded token sequence corresponding to the first markup language portion. In response to creating the first embedded token sequence, the computer system uses a recurrent neural network (RNN) encoder to create a first code representation corresponding to the first embedded token sequence. The computer system inputs the first code representation into an RNN decoder and receives a first output. The computer system determines a loss value by comparing the probability vector (or the corresponding output token sequence) output by the RNN decoder with the first embedded token sequence. Based on the determined loss value, the computer system adjusts one or more weight values ​​associated with the RNN encoder.

[0013] In an example embodiment, this disclosure describes a solution according to one embodiment, which describes the process for training a recurrent neural network (RNN) autoencoder to output a fixed-length markup language representation based on input of a variable-length markup language fragment. This disclosure describes a solution that includes utilizing a web crawler to identify operable elements within a markup language (e.g., Hypertext Markup Language (HTML) or Extensible Markup Language (XML)), and further creating a token sequence corresponding to the operable elements. This disclosure describes creating embeddings for the token sequence, and additionally, inputting the embedded tokens through an RNN encoder to create a code representation of the operable elements. This disclosure then describes inputting the code representation of the operable elements into the RNN decoder and determining a series of probability vectors (probability vectors corresponding to each token in the desired token sequence). Furthermore, this disclosure describes determining an output token sequence based on the series of probability vectors, and additionally determining whether there is convergence between the output embedded token sequence and the input embedded token sequence. This disclosure also describes comparing a probability vector (or a corresponding output embedded token sequence) with a desired embedded token sequence to identify a loss value (when compared with the desired output), and additionally, updating the weights associated with the RNN autoencoder. This process can be repeated, and the weights can be continuously adjusted accordingly until convergence exists between the output of the RNN decoder and the input of the RNN encoder.

[0014] Furthermore, this disclosure describes a process according to one embodiment for using a trained RNN encoder to output a fixed-length markup language representation based on an input of a variable-length markup language fragment. In an example embodiment, this disclosure identifies operable elements within a markup language (e.g., HTML or XML), and further preprocesses the markup language (as described above) and utilizes the RNN encoder to create a fixed-length markup language representation of the operable elements. Furthermore, this disclosure describes identifying additional information corresponding to the operable elements and utilizing said additional information (if any) to create a final fixed-length markup language representation of the operable elements. Furthermore, this disclosure describes inputting the fixed-length markup language representation into a model, such as a machine learning model, and obtaining an output.

[0015] As stated above, machine learning algorithms typically require structured input data, and therefore, using data obtained from the Web as input for machine learning is problematic. This disclosure describes a process that leverages the power of RNNs to process sequential data and the powerful ability of an autoencoder to reproduce input from short codes to produce an RNN encoder for sequences. As described below, in conjunction with the accompanying drawings, a trained RNN encoder can be used to create fixed-length representations of HTML fragments (thus creating a mapping between variable-length HTML code and fixed-length vector representations). Embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0016] Figure 1 The figure illustrates an automatic encoder system 100 according to one embodiment. In an example embodiment, the automatic encoder system 100 includes a server 110, a web server 120, a server 140, and a server 150 interconnected via a network 130.

[0017] In the example embodiment, network 130 is the Internet, representing a global collection of networks and gateways used to support communication between devices connected to the Internet. Network 130 may include, for example, wired, wireless, or fiber optic connections. In other embodiments, network 130 may be implemented as an intranet, Bluetooth network, local area network (LAN), or wide area network (WAN). Generally, network 130 can be any combination of connections and protocols that support communication between computing devices, such as server 110 and server 140.

[0018] In an example embodiment, web server 120 includes website 122. In an example embodiment, web server 120 may be a desktop computer, laptop computer, tablet computer, mobile device, handheld device, thin client, or any other electronic device or computing system capable of receiving and sending data to and from other computing devices, such as server 110, via network 130. Although not shown, alternatively, web server 120 may include a cluster of servers executing the same software to uniformly process requests distributed across front-end servers and load balancers. In an example embodiment, web server 120 is a computing device optimized for supporting websites hosted on web server 120, such as website 122, and for supporting network requests associated with websites hosted on web server 120. Web server 120 is described in more detail with reference to the accompanying drawings.

[0019] In the example embodiment, website 122 is a collection of files, including, for example, HTML files, CSS files, image files, and JavaScript files. Website 122 may also include other resource files, such as audio files and video files. Website 122 is described in more detail with reference to the accompanying drawings.

[0020] In an example embodiment, server 120 includes model 142. In an example embodiment, server 140 may be a desktop computer, laptop computer, tablet computer, mobile device, handheld device, thin client, or any other electronic device or computing system capable of receiving and sending data to and from other computing devices, such as server 110, via network 130. Furthermore, in an example embodiment, server 140 is a computing device optimized for supporting programs hosted on server 140, such as model 142. Although not shown, alternatively, server 140 may include a cluster of servers executing the same software to uniformly process requests distributed across front-end servers and load balancers. Website 140 is described in more detail with reference to the accompanying drawings.

[0021] In the example embodiment, model 142 is a model capable of receiving input and providing corresponding output, such as a machine learning model. For example, in one or more embodiments, model 142 is capable of receiving input corresponding to a purpose and providing a predicted output corresponding to the next action to be taken (by a web crawler or other application) to achieve that purpose. Furthermore, in the example embodiment, model 142 can operate in a reinforcement learning environment and is also capable of observing the environment, such as activities performed by web crawler 112, and using the observed activities to determine predictions. Additionally, in one or more embodiments, model 142 may require the input to be a fixed-length input or a fixed-structure input. Model 142 is described in more detail with reference to the accompanying drawings.

[0022] In an example embodiment, server 150 includes database 154. In an example embodiment, web server 150 may be a desktop computer, laptop computer, tablet computer, mobile device, handheld device, thin client, or any other electronic device or computing system capable of receiving and sending data to and from other computing devices, such as server 110, via network 130. Furthermore, in an example embodiment, server 150 is a computing device optimized for supporting database requests corresponding to database 154. Although not shown, alternatively, server 150 may include a server cluster executing the same software to uniformly process requests distributed across front-end servers and load balancers. Website 150 is described in more detail with reference to the accompanying drawings.

[0023] In an example embodiment, database 154 is a database containing information corresponding to one or more web pages. For example, database 154 may contain information corresponding to web pages accessed by web crawler 112, such as HTML source code, one or more operable elements extracted from the HTML source code, additional information corresponding to the web page (e.g., whether the digital shopping cart is empty or contains items), and previous web pages accessed by web crawler 112 (and previous actions taken by web crawler 112). In other embodiments, database 154 may include user information or other types of information. Database 154 is described in more detail with reference to the accompanying drawings.

[0024] In an example embodiment, server 110 includes a web crawler 112, a browser 114, an autoencoder 116, and a conversion program 118. In an example embodiment, web server 110 may be a desktop computer, laptop computer, tablet computer, mobile device, handheld device, thin client, or any other electronic device or computing system capable of receiving and sending data to and from other computing devices, such as web server 120, via network 130. Furthermore, in an example embodiment, server 110 is a computing device optimized to support programs hosted on server 110, such as web crawler 112, autoencoder 116, and conversion program 118. Although not shown, alternatively, server 110 may include a cluster of servers executing the same software to uniformly process requests distributed across front-end servers and load balancers. Website 110 is described in more detail with reference to the accompanying drawings.

[0025] In the example embodiment, browser 114 is an application capable of communicating with other computing devices to send requests and receive information.

[0026] Furthermore, browser 114 can display the received information to the user of server 110. In an example embodiment, browser 114 can send a request to website 122 and further receive web page information from website 122. Browser 114 is described in more detail with reference to the accompanying drawings.

[0027] Web crawler 112 is a software application capable of browsing the Internet to identify information corresponding to one or more web pages, such as identifying identifying elements of a web page. In an example embodiment, web crawler 112 can access one or more databases to identify one or more websites to be analyzed (and can also store information associated with one or more web pages or websites in one or more databases). Additionally, in an example embodiment, web crawler 112 can extract information and content from web pages, such as source code corresponding to one or more elements of a web page. Furthermore, in one or more embodiments, web crawler 112 can utilize the functionality of browser 114 to access one or more websites, such as website 122. Web crawler 112 is described in more detail with reference to the accompanying drawings.

[0028] Furthermore, in one or more embodiments, the web crawler 112 may utilize an application programming interface (API) when communicating with other programs and also when communicating with the database 154.

[0029] The autoencoder 116 includes an encoder 116a and a decoder 116b. In an example embodiment, the autoencoder 116 is an RNN autoencoder capable of utilizing the capabilities of an RNN for processing sequential data. In other embodiments, the autoencoder 116 may utilize other neural network capabilities. In an example embodiment, the autoencoder 116 comprises an encoder 116a capable of converting an input token sequence (corresponding to a variable-length input, such as HTML) into a fixed-length representation that can be used in a machine learning model. The autoencoder 116 also comprises a decoder 116b capable of converting the fixed-length representation back into the token sequence provided to the encoder 116a. The autoencoder 116, encoder 116a, and decoder 116b are described in more detail with reference to the accompanying drawings.

[0030] The conversion program 118 is a program capable of recognizing operable elements from markup language (e.g., extracted from HTML by web crawler 112). Furthermore, in one or more embodiments, the conversion program 118 is capable of training an autoencoder 116 using the markup language corresponding to one or more operable elements and configuring the autoencoder by adjusting the weight values ​​associated with the autoencoder 116. Additionally, in an example embodiment, the conversion program 118 is capable of converting the markup language corresponding to the recognized operable elements into one or more embedded tokens, which can then be input into encoder 116a to create a fixed-length vector representation. Furthermore, the conversion program 118 is capable of inputting the fixed-length vector representation into a model such as model 142 and recognizing appropriate actions or outcomes based on the output of that model. The operation of the conversion program 118 is described in more detail with reference to the accompanying drawings.

[0031] Furthermore, although in the example embodiment, model 142 and database 154 are described as residing on server 140 and server 150, respectively, in other embodiments, model 142 and / or database 154 may reside on a single server or may reside on server 110.

[0032] Figure 2 and 3 This is a flowchart illustrating the operation of the conversion procedure 118 during the training (or calibration) of the autoencoder 116 according to an embodiment. In an example embodiment, a web crawler 112 extracts markup language (e.g., HTML, XML, etc.) from web pages on a website 122. In an example embodiment, the web crawler 112 may utilize the capabilities of a browser 114 to access the web pages.

[0033] In an example embodiment, conversion program 118 analyzes the extracted markup language and identifies the markup language corresponding to the operable element (step 202). In an example embodiment, the markup language corresponding to the operable element may include markup language associated with: selected links and / or buttons, fields that can accept input (e.g., address fields on a form), drop-down menus, sliding areas, draggable elements, or any other element corresponding to an action. In an example embodiment, conversion program 118 identifies the markup language corresponding to the operable element by analyzing tags (such as HTML tags) within the markup language and further identifying tags and corresponding markup languages ​​corresponding to the operable element. For example, conversion program 118 may analyze the extracted markup language and identify the markup language typically corresponding to a hyperlink. "Tags, and based on recognition" "Tags can identify the corresponding markup language, such as..." The attributes associated with the tag (such as the "href" attribute, which includes hyperlinks to web pages) correspond to the operable elements.

[0034] In an example embodiment, conversion procedure 118 creates a token sequence corresponding to the markup language associated with the operable element (step 204). In an example embodiment, conversion procedure 118 can create a token sequence of markup language corresponding to the operable element by, for example, modifying or replacing certain parts of the markup language using special predefined tokens. For example, " <start>"It can be a token used to mark the beginning of a sequence that constitutes a token sequence, and" <end>"It could be a token used to mark the end of a sequence. Furthermore," <pad>"It can be a token used as a filler item," <digit>"It could be a token used to replace all numbers, and" <site> "This can be used to replace the protocol and host portions of a Uniform Resource Locator (URL)." (See reference)< / site> < / digit> < / pad> < / end> < / start> Figure 5 Two examples are shown illustrating how the conversion procedure 118 creates token sequences based on the markup language associated with the operable element. Specifically, column 502 depicts two examples of markup languages ​​associated with the operable element, and column 504 depicts token sequences of these two parts of the markup language. In creating the token sequence shown in column 504, the conversion procedure 118 uses the token mapping described above; however, in other embodiments, different token mappings may be used. Furthermore, in the example embodiments, the token sequences may have a predetermined set sequence length, and therefore, if a particular token sequence is greater than the predetermined set sequence length, tokens at the end of the token sequence may be discarded until the set sequence length is reached. Similarly, if a particular token sequence contains tokens less than the predetermined set sequence length, padding tokens (e.g., "...") may be added. <pad>A token is added to the end of the token sequence until the set sequence length is reached.

[0035] In the example embodiment, conversion procedure 118 creates an embedding for each token within the token sequence (step 206). In the example embodiment, conversion procedure 118 utilizes a one-hot encoding method, but in other embodiments, different embedding methods may be used. In the example embodiment, conversion procedure 118 may create and maintain a mapping between one or more tokens and one or more assigned integers. For example, conversion procedure 118 may maintain the following mapping: "a" = 1, "b" = 2, "<" = 3, and ">" = 4. Therefore, conversion procedure 118 will embed the tokens... The sequence is transformed into [0,0,1,0...],[1,0,0....],[0,0,0,1...].

[0036] The conversion process 118 sequentially inputs the embedded tokens into encoder 116a (step 208). In the example embodiment, as described above, encoder 116a is an RNN encoder capable of processing sequential data using an RNN; however, in other embodiments, encoder 116a may be an encoder utilizing other neural network capabilities. In the example embodiment, encoder 116a generates a code representation of the embedded token sequence.

[0037] Then, the conversion process 118 can input the code representation into the decoder 116b (step 210). In the example embodiment, as stated above, the decoder 116b is an RNN decoder capable of utilizing the capabilities of an RNN. In the example embodiment, the decoder 116b receives the input code representation and generates a series of probability vectors, where each probability vector includes one or more token probabilities. Furthermore, in the example embodiment, the conversion process 118 can analyze each probability vector and determine the token corresponding to the highest probability or likelihood value from the one or more token probabilities. Using this process, the conversion process 118 can select tokens from each probability vector to determine an output embedded token sequence.

[0038] Furthermore, based on the configuration of the weight values ​​associated with the autoencoder 116, the output embedded token sequence may not be equal to the input embedded token sequence derived from the markup language associated with the operable element. To determine what adjustments need to be made to the weight values ​​associated with the autoencoder 116, the conversion procedure 118 compares the embedded tokens output by the decoder 116b with the embedded token sequence input to the encoder 116a to determine whether there is convergence (determination 304) between the output of the decoder 116b and the input provided to the encoder 116a. In other words, the conversion procedure 118 determines whether the output of the decoder 116b is substantially equal to the input of the encoder 116a. In an example embodiment, this may include determining whether the output embedded token sequence is equal to or belongs to a threshold percentage of equality with the input embedded token sequence.

[0039] If the conversion procedure 118 determines whether convergence exists between the output vector of decoder 116b and the input vector provided to encoder 116a (judgment 304, "Yes" branch), then the conversion procedure 118 determines that training is complete and no adjustment is needed to the weight values ​​associated with autoencoder 116. If the conversion procedure 118 determines that convergence does not exist between the output vector of decoder 116b and the input vector provided to encoder 116a (judgment 304, "No" branch), then the conversion procedure 118 updates the weight values ​​associated with autoencoder 116 based on the determined loss value (step 306). In an example embodiment, the conversion procedure 118 determines the loss value based on comparing the output token sequence (or the corresponding probability vector) with the desired output token sequence, and updates the weight values ​​associated with autoencoder 116 based on the determined loss value. In an example embodiment, the conversion procedure 118 uses a backpropagation algorithm to adjust the weight values; however, in other embodiments, other neural network training algorithms or other types of algorithms may be used. Once the weight values ​​have been adjusted, the conversion procedure 118 can be repeated. Figure 2 and 3 The process described herein continues until there is convergence between the output vector of decoder 116b and the input vector provided to encoder 116a.

[0040] In one or more embodiments, the mapping maintained by the conversion procedure 118 can be used to further convert the output embedded token sequence into a token sequence. For example, refer again to Figure 5 Column 506 depicts the token sequence reconstructed using decoder 116b. Furthermore, in one or more of these embodiments, instead of comparing the output embedded token sequence with the input embedded token sequence, the conversion procedure 118 can compare the output token sequence with the input token sequence.

[0041] Figure 4 This is a flowchart illustrating a process for training an autoencoder 116 according to one embodiment. In the example embodiment, as described above, the conversion process 118 inputs an embedded token sequence corresponding to a markup language associated with an operable meta-element (i.e., input 402) into encoder 116a to obtain a code representation, code 404. The conversion process 118 can then input this code representation into decoder 116b to obtain an embedded token sequence, referred to as output 406. The conversion process 118 can then compare output 406 with input 402 in the manner described above to determine whether convergence exists between output 406 and input 402. The conversion process 118 can iteratively repeat this process with additional inputs until convergence between output 406 and input 402 has been achieved.

[0042] Figure 5 This is an illustration of multiple specific iterations of a process for training an autoencoder 116 according to one embodiment. In the example embodiment, Figure 5 The illustrated markup language 502 may include a markup language corresponding to an operable element, a token sequence 504 corresponding to the markup language, and an output token sequence 506 corresponding to the token sequence output by the decoder 116b. As shown, the output token sequence 506 may not be equal to the input token sequence 504, and therefore, the conversion procedure 118 may adjust the weight values ​​associated with the autoencoder 116 until convergence between the output and the input is achieved.

[0043] Figure 6 The illustration shows, according to an embodiment, after training, Figure 1 The conversion program utilizes Figure 1 The flowchart describes the operation of an autoencoder to create a fixed-length representation from variable-length markup language fragments. As stated above, web crawler 112 can use the capabilities of browser 114 to extract markup language (such as HTML, XML, etc.) from web pages of website 122.

[0044] In an example embodiment, conversion procedure 118 analyzes the extracted markup language and identifies the markup language corresponding to the operable element (step 602). As stated above, in an example embodiment, the markup language corresponding to the operable element may include markup language associated with: selected links and / or buttons, fields that can accept input (e.g., address fields on a form), drop-down menus, sliding areas, draggable elements, or any other element corresponding to an action. In an example embodiment, conversion procedure 118 identifies the markup language corresponding to the operable element by analyzing tags (such as HTML tags) within the markup language and further identifying tags and corresponding markup languages ​​corresponding to the operable element.

[0045] In an example embodiment, as stated above, the conversion procedure 118 creates a token sequence corresponding to the markup language associated with the operable element. In an example embodiment, the conversion procedure 118 can create a token sequence of the markup language corresponding to the operable element by, for example, modifying or replacing certain parts of the markup language using special predefined tokens. Furthermore, in an example embodiment, the token sequence may have a predetermined set sequence length, and therefore, if a particular token sequence is greater than the predetermined set sequence length, tokens at the end of the token sequence can be discarded until the set sequence length is reached. Similarly, if a particular token sequence contains tokens less than the predetermined set sequence length, padding tokens (e.g., "...") can be used. <pad>A token is added to the end of the token sequence until the set sequence length is reached. Furthermore, as stated above, the conversion procedure 118 may further create an embedding for each token within the token sequence. In the example embodiment, the conversion procedure 118 utilizes a one-hot encoding method, but in other embodiments, different embedding methods may be used.

[0046] The conversion procedure 118 sequentially inputs the embedded tokens into encoder 116a (step 604). In the example embodiment, as described above, encoder 116a is an RNN encoder capable of processing sequential data using an RNN; however, in other embodiments, encoder 116a may be an encoder utilizing other neural network capabilities. In the example embodiment, encoder 116a generates a code representation of the embedded token sequence.

[0047] The conversion process 118 can determine additional information (judgment 606) corresponding to the overall goal of the web crawler 112. In an example embodiment, the web crawler 112 may have a specific goal, such as accessing the checkout page of website 122. Therefore, in order to achieve the goal of accessing the checkout page, the administrator can define specific information or determinations that the model can use when achieving the goal of accessing the checkout page of website 122. For example, "Has the item been added to the digital cart?" or "Have we identified the cart element on the visible webpage?" can be specific judgments defined for the purpose of accessing the checkout page of website 122. The conversion process 118 can analyze the source code corresponding to the current webpage (or the webpage accessed during the current session) and / or the access database 154 to analyze the previous crawling activities performed by the web crawler 112 in order to determine the relevant information / answers for these specific judgments. For example, the conversion process 118 can analyze the previous crawling activities and determine that the web crawler 112 has taken steps to add the item to the digital cart of website 122 during the current session. Additional information may also include other types of information, such as the number of web pages visited in the current session, and other characteristics related to the process and / or goal associated with web crawler 112. Additional information corresponding to a specific judgment can be used by model 142 to determine the next appropriate step to take. For example, if conversion procedure 118 determines that an item has not yet been added to the digital shopping cart (the goal is to access the checkout page), model 142 can instruct web crawler 112 to take appropriate steps to add the item to the digital shopping cart.

[0048] If the conversion process 118 determines that no additional information is available (judgment 606, "No" branch), then the conversion process 118 uses the code representation as the final fixed-length markup language representation corresponding to the operable element, and further inputs the code representation into model 142 (step 610). If the conversion process 118 determines that additional information is available (judgment 606, "Yes" branch), then the conversion process 118 creates a final fixed-length markup language representation based on the code representation and the identified additional information (step 608), and further inputs the created final fixed-length markup language representation into model 142 (step 610). In an example embodiment, the identified additional information can be concatenated to the code representation of the created final fixed-length markup language. Furthermore, in an example embodiment, the model can be a machine learning model, and can further process the input information and identify the next step of the web crawler 112 based on the provided input and a preset goal. For example, if the preset goal is to access the checkout page, model 142 can output suggestions for taking actions to encourage adding items to the digital shopping cart.

[0049] As stated above, Model 142 can be a machine learning model running in a reinforcement learning environment.

[0050] Figure 7 This is a flowchart illustrating a process, according to one embodiment, of creating a fixed-length representation from variable-length markup language fragments for input into model 142. In an exemplary embodiment, Figure 7 This represents a reinforcement learning system, which includes an environment: environment 702 and a model observing environment 702: model 142, and provides predictions / suggestions on what actions to take to efficiently achieve a preset goal based on received input. In an example embodiment, as stated above, a conversion process 118 analyzes markup language extracted from Web content 704 and further identifies markup language 706 corresponding to operable elements. Then, as described above, the conversion process 118 inputs the processed markup language to create a corresponding embedded token sequence, which is input into encoder 116a to produce code representation 708. The conversion process 118 then determines whether there is additional information corresponding to the preset goal, and if additional information such as additional information 710 is identified, combines the additional information with code representation 708 to create a final fixed-length markup language representation 712. The conversion process 118 inputs the final fixed-length markup language representation 712 into model 142, where model 142 processes the input and provides information on the next steps to be taken to efficiently achieve the preset goal.

[0051] For example, if the preset goal is to access the checkout page of a website, web crawler 112 can use the capabilities of browser 114 to extract markup language (e.g., HTML) from the web pages of website 122. Then, conversion process 118 can identify actionable items corresponding to the extracted markup language and further utilize one or more techniques (as described above) to provide a representation corresponding to each actionable item, which may include additional information such as previous actions taken by web crawler 112. Conversion process 118 can then provide the representation of each actionable item to model 142, which can analyze the representation and determine, based on the goal to be achieved, whether an action is necessary for that actionable item, and further determine what action web crawler 112 should take. For example, if the actionable item corresponds to adding an item to a digital shopping cart, model 142 can analyze additional information corresponding to whether the item already exists in the digital cart and, based on this analysis, determine whether web crawler 112 should select the actionable item (and thus add the item to the digital cart). Furthermore, if model 142 determines, through analysis of additional information, that an item already exists in the digital shopping cart, then model 142 can determine that no action is needed for the actionable item corresponding to adding the item to the digital shopping cart, because the predefined goal of accessing the checkout page does not require this action. In other words, since model 142 operates in a reinforcement learning environment, rewards can be provided based on the model's efficiency in achieving the predefined goal. Therefore, adding an item to the digital shopping cart when it already exists might be considered an unnecessary and inefficient task by model 142. However, if, through analysis of additional information, model 142 determines that the item does not exist in the digital cart, then model 142 can determine / recommend the web crawler 112 to select the aforementioned actionable item corresponding to adding the item to the digital shopping cart.

[0052] Furthermore, the predefined goal of reaching the checkout page is provided as an example, and other goals can be achieved using the process described above, such as the customer due diligence (KYC) process, where the goal could be to identify and / or verify information corresponding to a customer.

[0053] In one or more embodiments, Figure 7 The reinforcement learning system described herein can receive input corresponding to a representation of the environment, and based on the representation of the environment, can determine the actions to be taken to efficiently achieve the objective. Although in the example embodiment, the representation of the environment can be presented as a final fixed-length markup language representation 712 formatted using the above processing, in other embodiments, other methods may be used to format the representation of the environment.

[0054] The foregoing description of various embodiments of this disclosure is provided for purposes of illustration and description. It is not intended to be exhaustive, nor is it intended to limit this disclosure to the specific forms disclosed. Many modifications and variations are possible. Such modifications and variations that will be apparent to those skilled in the art should be included within the scope of the disclosure as defined by the appended claims.

[0055] Figure 8 The illustration is based on an embodiment. Figure 1 A block diagram of the components of the computing device included in the automatic encoder system 100. It should be recognized that... Figure 8 This illustration is provided only as an example of an implementation and does not imply any limitation on the environment in which different embodiments may be implemented. Many modifications can be made to the depicted environment.

[0056] The computing device may include one or more processors 802, one or more computer-readable RAMs 804, one or more computer-readable ROMs 806, one or more computer-readable storage media 808, device drivers 812, read / write drivers or interfaces 814, and network adapters or interfaces 816, all of which are interconnected via a communication structure 818. The communication structure 818 may be implemented using any architecture designed to transfer data and / or control information between processors (e.g., microprocessors, communication and network processors, etc.), system memory, peripheral devices, and any other hardware components within the system.

[0057] One or more operating systems 810 and one or more applications 811, such as a web crawler 112, are stored on one or more computer-readable storage media 808 for execution by one or more processors 802 and utilize one or more corresponding RAMs 804 (typically including cache memory). In the illustrated embodiment, each of the computer-readable storage media 808 may be a disk storage device such as an internal hard disk drive, CD-ROM, DVD, Memory Stick, magnetic tape, disk, optical disk, semiconductor storage device such as RAM, ROM, EPROM, flash memory, or any other computer-readable tangible storage device capable of storing computer programs and digital information.

[0058] The computing device may also include a R / W drive or interface 814 for reading and writing to one or more portable computer-readable storage media 826. An application program 811 on the computing device may be stored on one or more portable computer-readable storage media 826, read via the corresponding R / W drive or interface 814, and loaded into the corresponding computer-readable storage medium 808.

[0059] The computing device may also include a network adapter or interface 816, such as a TCP / IP adapter card or a wireless communication adapter (e.g., a 4G wireless communication adapter using OFDMA technology). An application 811 on the computing device can be downloaded from an external computer or external storage device via a network (e.g., the Internet, a local area network, or other wide area networks or wireless networks) and the network adapter or interface 816. From the network adapter or interface 816, the program can be loaded onto a computer-readable storage medium 808. The network may include copper wire, fiber optic, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers.

[0060] The computing device may also include a display screen 820 and an external device 822, which may include, for example, a keyboard, a computer mouse, and / or a touchpad. Device driver 812 implements an interface to display screen 820 for displaying images to external device 822, and / or implements an interface to display screen 820 for pressure sensing of alphanumeric character input and user selection. Device driver 812, R / W driver or interface 814, and network adapter or interface 816 may include hardware and software (stored on computer-readable storage medium 808 and / or ROM 806).

[0061] The programs described herein are identified based on the applications for which they are implemented in specific embodiments. However, it should be recognized that any particular program naming conventions herein are for convenience only, and therefore this disclosure should not be limited to use in any specific application identified and / or implied by such naming conventions.

[0062] Based on the foregoing, a computer system, method, and computer program product have been disclosed. However, many modifications and substitutions can be made without departing from the scope of this disclosure. Therefore, these various embodiments are provided as examples and not as limiting the scope of the disclosure.

[0063] Various embodiments of this disclosure can be a system, method, and computer program product. A computer program product may include one or more computer-readable storage media having computer-readable program instructions thereon for causing a processor to implement various aspects of this disclosure.

[0064] A computer-readable storage medium can be a tangible device capable of retaining and storing instructions for use by an instruction execution device. A computer-readable storage medium can be, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or raised structures in grooves on which instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses transmitted through fiber optic cables), or electrical signals transmitted through wires.

[0065] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device or via a network, such as the Internet, a local area network (LAN), a wide area network (WAN), and / or a wireless network, to an external computer or external storage device. This network may include copper transmission cables, fiber optic cables, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them for storage in a computer-readable storage medium within the respective computing / processing device.

[0066] Computer-readable program instructions used to perform the operations of this disclosure may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, partially on the user's computer as a standalone software package, partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using the Internet provided by an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may utilize the status information of the computer-readable program instructions to execute the computer-readable program instructions to customize the electronic circuitry to perform various aspects of this disclosure.

[0067] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of these flowchart illustrations and / or block diagrams, and combinations of blocks in these flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0068] These computer-readable program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to create a machine, thereby creating means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram, through instructions that execute via the processor of the computer or other programmable data processing apparatus. These computer-readable program instructions can also be stored in a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, and / or other apparatus to operate in a particular manner, such that the computer-readable storage medium containing the instructions includes an article of manufacture comprising instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0069] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus or other device, perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0070] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, instruction segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the functions mentioned in multiple blocks may not occur in the order shown in the drawings. For example, two blocks shown sequentially may actually execute substantially simultaneously, or these blocks may sometimes execute in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of multiple blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or performs a combination of special-purpose hardware and computer instructions.< / pad> < / pad>

Claims

1. A computer system, comprising: One or more computer-readable storage devices, wherein the one or more computer-readable storage devices store program instructions; as well as One or more processors, the one or more processors being configured to execute the program instructions to cause the system to perform operations including: Identify whether a first markup language portion extracted from a markup language document of a website corresponds to a first operable element, wherein the first markup language portion is a variable-length representation; In response to recognizing that the first markup language portion corresponds to the first operable element, a first code representation corresponding to the first markup language portion is created using a recurrent neural network (RNN) encoder; Identify the first additional information corresponding to one or more predefined targets; Create a final fixed-length markup language representation, which includes the first code representation and the first additional information; and The final fixed-length markup language representation is then input into the model.

2. The computer system according to claim 1, further comprising: In response to recognizing that the first markup language portion corresponds to the first operable element, a first embedded token sequence corresponding to the first markup language portion is created.

3. The computer system according to claim 1, wherein, The first additional information includes information associated with the activity of web crawlers on the website, or information corresponding to one or more elements in the markup language document.

4. The computer system according to claim 3, wherein, The information corresponding to one or more elements in the markup language document includes instructions to add items to a digital shopping cart, and wherein the one or more predefined targets include the checkout page of the website accessed by the web crawler.

5. The computer system according to claim 1, further comprising: The model receives an output that provides an indication of whether the first operable element should be selected, wherein the output is determined based on an analysis of the first additional information and the one or more predefined targets.

6. The computer system according to claim 1, further comprising: Before recognizing that the first markup language portion corresponds to the first operable element, the RNN autoencoder is calibrated, wherein the RNN autoencoder includes an RNN encoder and an RNN decoder, and wherein calibrating the RNN autoencoder includes: In response to identifying a second markup language portion corresponding to a second operable element, a second embedded token sequence corresponding to the second markup language portion is created; In response to the creation of the second embedded token sequence, a second code representation corresponding to the second embedded token sequence is created using the RNN encoder; In response to inputting the second code representation into the RNN decoder, a first set of one or more probability vectors is received; The first output is determined based on one or more probability vectors from the first group; The loss value is determined by comparing the first output of the RNN decoder with the second embedded token sequence; and Based on the determined loss value, adjust one or more weight values ​​associated with the RNN autoencoder.

7. The computer system according to claim 6, wherein, Calibrling the RNN autoencoder also includes: In response to identifying a third markup language portion corresponding to a third operable element, a third embedded token sequence corresponding to the third markup language portion is created; In response to the creation of the third embedded token sequence, a third code representation corresponding to the third embedded token sequence is created using the RNN encoder; The third code representation is input into the RNN decoder, and convergence between the second output and the third embedded token sequence is determined by comparing the second output with the third embedded token sequence, wherein the second output is created based on a second set of one or more probability vectors output by the RNN decoder; and Based on the determination that there is convergence between the second output and the third embedded token sequence, it is determined that no adjustment is needed for the one or more weight values ​​associated with the RNN autoencoder.

8. A non-transitory computer-readable medium storing computer-executable instructions that, in response to execution by one or more hardware processors, cause the one or more hardware processors to perform operations including: The first markup language portion extracted from the markup language document of the website is identified as corresponding to the first operable element, wherein, The first markup language part is a variable-length representation; In response to the identification that the first markup language portion corresponds to the first operable element, a first code representation corresponding to the first markup language portion is created using a recurrent neural network (RNN) encoder; Identify the first additional information corresponding to one or more predefined targets; Create a final fixed-length markup language representation that includes the first code representation and the first additional information; as well as The final fixed-length markup language representation is input into the model, wherein the model can only accept fixed-length input.

9. The non-transitory computer-readable medium of claim 8, further comprising: In response to recognizing that the first markup language portion corresponds to the first operable element, a first embedded token sequence corresponding to the first markup language portion is created.

10. The non-transitory computer-readable medium according to claim 8, wherein, The first additional information includes information associated with the activity of web crawlers on the website, or information corresponding to one or more elements in the markup language document.

11. The non-transitory computer-readable medium according to claim 8, wherein, The first markup language part is Hypertext Markup Language (HTML) or Extensible Markup Language (XML).

12. The non-transitory computer-readable medium of claim 8, further comprising: Feedback is received from the model, including suggested next actions for achieving the one or more predefined objectives.

13. The non-transitory computer-readable medium of claim 8, further comprising: Before recognizing that the first markup language portion corresponds to the first operable element, the RNN autoencoder is calibrated, wherein the RNN autoencoder includes an RNN encoder and an RNN decoder, and wherein calibrating the RNN autoencoder includes: In response to identifying a second markup language portion corresponding to a second operable element, a second embedded token sequence corresponding to the second markup language portion is created; In response to the creation of the second embedded token sequence, a second code representation corresponding to the second embedded token sequence is created using the RNN encoder; In response to inputting the second code representation into the RNN decoder, a first set of one or more probability vectors is received; The first output is determined based on one or more probability vectors from the first group; Based on the determination that no convergence exists between the first output and the second embedded token sequence, a loss value is determined by comparing the first set of one or more probability vectors with the second embedded token sequence; and Based on the determined loss value, adjust one or more weight values ​​associated with the RNN autoencoder.

14. The non-transitory computer-readable medium according to claim 13, wherein, Calibrling the RNN encoder also includes: In response to identifying a third markup language portion corresponding to a third operable element, a third embedded token sequence corresponding to the third markup language portion is created; In response to the creation of the third embedded token sequence, a third code representation corresponding to the third embedded token sequence is created using the RNN encoder; The third code representation is input into the RNN decoder, and convergence between the second output and the third embedded token sequence is determined by comparing the second output with the third embedded token sequence, wherein the second output is created based on a second set of one or more probability vectors output by the RNN decoder; and Based on the determination that there is convergence between the second output and the third embedded token sequence, it is determined that no adjustment is needed for the one or more weight values ​​associated with the RNN autoencoder.

15. A method comprising: In response to recognizing that the first markup language portion corresponds to the first operable element, the computer system creates a first embedded token sequence corresponding to the first markup language portion; In response to the creation of the first embedded token sequence, the computer system uses a recurrent neural network (RNN) encoder to create a first code representation corresponding to the first embedded token sequence; The computer system inputs the first code representation into the RNN decoder and receives a first set of one or more probability vectors; The computer system determines the first output based on the first set of one or more probability vectors; Based on the determination that there is no convergence between the first output and the first embedded token sequence, the computer system determines the loss value by comparing the first output of the RNN decoder with the first embedded token sequence; as well as Based on the determined loss value, the computer system adjusts one or more weight values ​​associated with the RNN encoder.

16. The method according to claim 15, wherein, Calibrling the RNN encoder also includes: In response to recognizing that the second markup language portion corresponds to the second operable element, the computer system creates a second embedded token sequence corresponding to the second markup language portion; In response to the creation of the second embedded token sequence, the computer system uses the RNN encoder to create a second code representation corresponding to the second embedded token sequence; The computer system inputs the second code representation into the RNN decoder, and determines, based on a comparison of the second output with the second embedded token sequence, that there is convergence between the second output and the second embedded token sequence, wherein the second output is created based on a second set of one or more probability vectors received from the RNN decoder; and Based on the determination that there is convergence between the second output and the second embedded token sequence, the computer system determines that no adjustment is needed to the RNN encoder.

17. The method of claim 16, further comprising: The computer system identifies that the third markup language portion extracted from the markup language document of the website corresponds to a third operable element; In response to recognizing that the third markup language portion corresponds to the third operable element, the RNN encoder is used to create a third code representation corresponding to the third markup language portion; Identify the first additional information corresponding to one or more predefined targets; Create a final fixed-length markup language representation that includes the third code representation and the first additional information; as well as The final fixed-length markup language representation is then input into the model.

18. The method according to claim 17, wherein, The first additional information includes information associated with the activity of web crawlers on the website, or information corresponding to one or more elements in the markup language document.

19. The method according to claim 18, wherein, The information corresponding to one or more elements in the markup language document includes instructions to add items to a digital shopping cart, and wherein the one or more predefined targets include the checkout page of the website accessed by the web crawler.

20. The method of claim 17, further comprising: Feedback is received from the model, including suggested next actions for achieving the one or more predefined objectives.

Images