Training methods, apparatus, equipment, and storage media for feature extraction networks

Abstract

This disclosure relates to a training method, apparatus, device, and storage medium for a feature extraction network. The training method includes: performing vector encoding on a first operation to obtain a feature vector corresponding to the first operation; extracting features from a first sample page and a second sample page based on the feature extraction network to obtain feature vectors for the first and second sample pages; determining a first similarity between the real operation and the first operation based on the first operation, the first sample page, and the feature vectors of the second sample page; and adjusting the parameters of the feature extraction network based on the first similarity and first labeling information to make the feature extraction network converge. The feature extraction network provided in this disclosure can extract features related to the operation that triggers a page switch to another page, thereby accurately identifying the real operation that triggers the page switch to another specific page based on these features, achieving matching of upstream extracted page features with downstream tasks.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a training method, apparatus, device, and storage medium for a feature extraction network. Background Technology

[0002] In current technologies, the analysis and processing of page features typically involves an upstream model (e.g., a pre-trained language model, also known as a BERT model) extracting and processing these features, which are then fed into a downstream model for analysis and processing. However, the page features obtained by the upstream model are currently quite broad in dimensions, while the downstream model may only be interested in one or a few dimensions of the page's features. This leads to a mismatch between the extracted page features and the downstream task. Summary of the Invention

[0003] To address the aforementioned technical problems, or at least partially address them, this disclosure provides a training method, apparatus, device, and storage medium for a feature extraction network.

[0004] In a first aspect, embodiments of this disclosure provide a method for training a feature extraction network, comprising:

[0005] Obtain a sample array, which includes information about a first sample page, a second sample page, a first operation, and first marker information. The first marker information is used to mark whether the first operation is a real operation that triggers the switch from the first sample page to the second sample page.

[0006] The first operation is vector encoded to obtain the feature vector corresponding to the first operation;

[0007] Based on the feature extraction network, feature vectors of the first sample page and the second sample page are extracted to obtain the feature vectors of the first sample page and the second sample page.

[0008] Based on the feature vectors of the first operation, the first sample page, and the second sample page, a first similarity between the real operation and the first operation is determined.

[0009] Based on the first similarity and the first labeling information, the parameters of the feature extraction network are adjusted to make the feature extraction network converge.

[0010] Optionally, determining the first similarity between the real operation and the first operation based on the feature vectors of the first operation, the first sample page, and the second sample page includes:

[0011] Based on a preset first similarity model, the feature vectors of the first operation, the first sample page, and the second sample page are processed to obtain the first similarity between the actual operation that triggers the switch from the first sample page to the second sample page and the first operation.

[0012] Optionally, after processing the feature vectors of the first operation, the first sample page, and the second sample page based on a preset first similarity model to obtain a first similarity between the actual operation that triggers the switch from the first sample page to the second sample page and the first operation, the method further includes:

[0013] Based on the first similarity and the first label information, the parameters of the first similarity model are adjusted.

[0014] Optionally, the feature extraction network includes a preprocessing network and a Siamese neural network;

[0015] The step of extracting features from the first sample page and the second sample page using a feature extraction network to obtain feature vectors for the first sample page and the second sample page includes:

[0016] The first sample page and the second sample page are processed based on the preprocessing network to obtain the feature vectors of the first sample page and the second sample page;

[0017] Based on the twin neural network, feature vectors related to page switching operations are extracted from the feature vectors of the first sample page and the second sample page.

[0018] Optionally, adjusting the parameters of the feature extraction network based on the first similarity and the first labeling information includes:

[0019] Based on the first labeling information, a second similarity between the first operation and the actual operation is determined;

[0020] Determine the mean square error between the first similarity and the second similarity;

[0021] The parameters of the Siamese neural network are adjusted based on the mean square error.

[0022] Optionally, after obtaining the converged feature extraction network, the method further includes:

[0023] Obtain a verification array, which includes two pages to be processed and a second marker information, the second marker information marking the actual operation that triggers the switching of the two pages to be processed;

[0024] Based on the feature extraction network, feature extraction processing is performed on the two pages to be processed to obtain the feature vectors of the two pages to be processed.

[0025] Based on the feature vectors of the two pages to be processed, the similarity between the actual operation that triggers the switching of the two pages to be processed and a number of preset operations is determined, including the actual operation that triggers the switching of the two pages to be processed.

[0026] The preset operations are sorted based on their similarity.

[0027] Based on the second marker information, determine the position of the actual operation that triggered the switching between the two pending pages in the sorting;

[0028] Based on the location, the accuracy of the feature extraction network is determined.

[0029] Secondly, embodiments of this disclosure provide a method for identifying page operations, including:

[0030] Get the first page, and the second page obtained by switching from the first page;

[0031] Feature vectors of the first page and the second page are extracted based on a preset feature extraction network;

[0032] Based on the feature vector, determine the page operation that triggers the switch from the first page to the second page;

[0033] The feature extraction network is trained based on a first sample page, a second sample page, a first operation, and first labeling information. The first labeling information is used to mark whether the first operation is a real operation that triggers the switch from the first sample page to the second sample page.

[0034] Thirdly, embodiments of this disclosure provide a training apparatus for a feature extraction network, comprising:

[0035] A sample data acquisition unit is used to acquire a sample array, which includes information about a first sample page, a second sample page, a first operation, and first marker information. The first marker information is used to mark whether the first operation is a real operation that triggers the switching of the first sample page to the second sample page.

[0036] The first operation encoding unit is used to perform vector encoding processing on the first operation to obtain the feature vector corresponding to the first operation;

[0037] The sample page feature extraction unit is used to extract features from the first sample page and the second sample page based on the feature extraction network to obtain feature vectors of the first sample page and the second sample page.

[0038] The first similarity calculation unit is used to determine the first similarity between the real operation and the first operation based on the feature vectors of the first operation, the first sample page, and the second sample page.

[0039] The model training unit is used to adjust the parameters of the feature extraction network based on the first similarity and the first label information, so that the feature extraction network converges.

[0040] Optionally, the similarity calculation unit processes the feature vectors of the first operation, the first sample page, and the second sample page based on a preset first similarity model to obtain a first similarity between the actual operation that triggers the switch from the first sample page to the second sample page and the first operation.

[0041] Optionally, the similarity calculation unit is further configured to adjust the parameters of the first similarity model based on the first similarity and the first label information.

[0042] Optionally, the feature extraction network includes a preprocessing network and a Siamese neural network; the sample page feature extraction unit includes:

[0043] The first feature extraction subunit is used to process the first sample page and the second sample page based on the preprocessing network to obtain the feature vectors of the first sample page and the second sample page;

[0044] The second feature extraction subunit is used to extract feature vectors related to page switching operations from the feature vectors of the first sample page and the second sample page based on the Siamese neural network.

[0045] Optionally, the model training unit includes:

[0046] The similarity determination subunit determines a second similarity between the first operation and the actual operation based on the first tag information;

[0047] The mean square error calculation subunit is used to determine the mean square error between the first similarity and the second similarity.

[0048] The parameter adjustment subunit is used to adjust the parameters of the Siamese neural network based on the mean square error.

[0049] Optionally, the feature extraction network training device further includes:

[0050] A verification data acquisition unit is used to acquire a verification array, which includes two pages to be processed and a second marker information, the second marker information marking the actual operation that triggers the switching of the two pages to be processed;

[0051] The feature extraction network usage unit is used to perform feature extraction processing on the two pages to be processed based on the feature extraction network, so as to obtain the feature vectors of the two pages to be processed.

[0052] The second similarity calculation unit is used to determine the similarity between the real operation that triggers the switching of the two pages to be processed and a plurality of preset operations based on the feature vectors of the two pages to be processed. The plurality of preset operations include the real operation that triggers the switching of the two pages to be processed.

[0053] The sorting unit is used to sort the multiple preset operations based on the similarity corresponding to each preset operation;

[0054] The position determination unit is used to determine, based on the second marker information, the position of the actual operation that triggers the switching between the two pending pages in the sorting;

[0055] An accuracy determination unit is used to determine the accuracy of the feature extraction network based on the location.

[0056] Fourthly, embodiments of this disclosure provide a page operation recognition device, including:

[0057] A page acquisition unit is used to acquire a first page and a second page obtained by switching from the first page;

[0058] The feature vector acquisition unit is used to extract feature vectors of the first page and the second page based on a preset feature extraction network.

[0059] A page operation determination unit is used to determine, based on the feature vector, the page operation that triggers the switching from the first page to the second page;

[0060] The feature extraction network is trained based on a first sample page, a second sample page, a first operation, and first labeling information. The first labeling information is used to mark whether the first operation is a real operation that triggers the switch from the first sample page to the second sample page.

[0061] Fifthly, embodiments of this disclosure provide a computing device including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, it can implement the above-mentioned training method for the feature extraction network or the page operation recognition method.

[0062] Sixthly, embodiments of this disclosure provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-described training method for the feature extraction network or the method for recognizing page operations.

[0063] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0064] The technical solution of this disclosure is used to train a feature extraction network. Based on a first operation, vector encoding is performed to obtain the corresponding feature vector. Then, feature extraction is performed on a first sample page and a second sample page based on the feature extraction network to obtain feature vectors for the first and second sample pages. Subsequently, a first similarity is determined based on the first operation, the feature vectors of the first and second sample pages, and the parameters of the feature extraction network are adjusted based on the first similarity and first labeling information to achieve the training of the feature extraction network. The feature extraction network provided by this disclosure can extract features related to the operation that triggers the page to switch to another page. Based on these features, the actual operation that triggers the page to switch to another specific page can be accurately identified, thereby achieving matching of the page features extracted upstream with the downstream task. Attached Figure Description

[0065] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0066] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 This is a flowchart of a training method for a feature extraction network provided in an embodiment of this disclosure;

[0068] Figure 2 This is a flowchart of a method for verifying the accuracy of a feature extraction network provided in an embodiment of this disclosure;

[0069] Figure 3 This is a flowchart of the page operation recognition method provided in the embodiments of this disclosure;

[0070] Figure 4 This is a schematic diagram of the structure of a training device for a feature extraction network provided in an embodiment of this disclosure;

[0071] Figure 5 This is a schematic diagram of the structure of a page operation recognition device provided in an embodiment of this disclosure;

[0072] Figure 6 This is a schematic diagram of the structure of a computing device according to an embodiment of this disclosure. Detailed Implementation

[0073] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0074] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0075] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0076] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0077] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0078] Figure 1 This is a flowchart illustrating a training method for a feature extraction network provided in an embodiment of this disclosure. In some embodiments of this disclosure, the training method for the feature extraction network can be executed by a computing device. Exemplarily, the computing device can be an electronic device such as a desktop computer or an all-in-one computer, or it can be a server.

[0079] like Figure 1 As shown, the training method for the feature extraction network provided in this embodiment includes steps S101-S105.

[0080] Step S101: Obtain a sample array, which includes information about the first sample page, the second sample page, the first operation, and the first tag information.

[0081] In this embodiment of the disclosure, the sample array is a data set used to train the feature extraction network, and there are multiple sample arrays. Each sample array includes a first sample page, a second sample page, information about a first operation, and first labeling information.

[0082] The first sample page is the page before a certain real operation is performed, and the second sample page is the page to which the user is redirected after performing the aforementioned real operation. The first and second sample pages can be pages represented using Extensible Markup Language (XML), or pages represented using other languages ​​such as Hyper Text Markup Language (HTML). This disclosure does not impose any particular limitation on these pages.

[0083] The first marker information is used to mark whether the first operation is a real operation that triggers the switch from the first sample page to the second sample page. If the first marker information is true, it indicates that the information of the first operation represents the real operation that causes the first sample page to switch to the second sample page; while if the first marker information is false, it indicates that the information of the first operation represents other possible operations besides the aforementioned real operation.

[0084] In this embodiment of the disclosure, the sample array can be constructed based on data tuples generated during daily operations. During daily operations, a large number of data tuples containing (first page, second page, actual operation) can be collected. The data tuple (first page, second page, specific operation) indicates that an actual operation is performed on the first page, navigating to the second page. The actual operation can be clicking a location on the page, clicking a control on the page, or swiping, etc.

[0085] After obtaining the aforementioned data tuple, a sample array can be determined based on it. For example, for the data tuple (first page, second page, actual operation), the first page can be used as the first sample page, the second page as the second sample page, and "actual operation" as the first operation. Based on "actual operation," the first flag information is determined to be "true," thus obtaining the sample array (first sample page, second sample page, first operation, true). Of course, in practical applications, "true" in the aforementioned sample array can also be represented by "1."

[0086] For example, for a data tuple (first page, second page, operation action), the first page can be used as the first sample page, the second page as the second sample page, and other operations different from the "real operation" can be randomly used as the first operation. Based on the aforementioned other operations, the first marker information is determined to be "false", thus obtaining a sample array (first sample page, second sample page, first operation, false). Of course, in practical applications, "false" in the aforementioned sample array can also be represented by "0".

[0087] Step S102: Perform vector encoding processing on the first operation to obtain the feature vector corresponding to the first operation.

[0088] In this embodiment of the disclosure, the vector encoding process for obtaining the feature vector corresponding to the first operation includes: determining all possible real operations, determining the vector dimension of the feature vector corresponding to the first operation based on the number of all possible real operations, and determining the elements in the feature vector corresponding to each real operation. Subsequently, vector encoding processing is performed according to the type of the first operation to obtain the corresponding feature vector.

[0089] For example, in one specific embodiment, all possible real operations include clicking a location on the page, clicking a control on the page, and swiping the screen. Therefore, the feature vector can be determined to be three-dimensional and can be represented as (x1, x2, x3). Here, x1 represents the element corresponding to clicking a location on the page, x2 represents the element corresponding to clicking a control on the page, and x3 represents the element corresponding to swiping the screen. Furthermore, when the first operation is a real operation, the value of the element at the corresponding position in the feature vector is 1, and the value of the elements at other positions is 0. Assuming the first operation is "clicking a location on the page," its corresponding feature vector is (1, 0, 0).

[0090] Step S103: Extract features from the first sample page and the second sample page based on the feature extraction network to obtain the feature vectors of the first sample page and the second sample page.

[0091] In this embodiment of the disclosure, feature extraction of the first sample page and the second sample page based on the feature extraction network can be performed by inputting the first sample page and the second sample page into the feature extraction network respectively to obtain feature vectors representing the attribute features of the first sample page and the second sample page respectively.

[0092] In some embodiments of this disclosure, the feature extraction network may include a preprocessing network and a Siamese neural network.

[0093] The preprocessing network is used to preprocess the first sample page and the second sample page to obtain a preprocessed network that represents the preprocessed feature vectors of the two sample pages. After processing by the preprocessing network, the first sample page and the second sample page can obtain corresponding first feature vectors and second feature vectors. In some embodiments of this disclosure, the preprocessing network can be a pre-trained processing network.

[0094] When the preprocessing network is a pre-trained processing network, the preprocessing network can be a BERT model, a TF-IDF model, or a Word2vec model, and this disclosure does not impose any particular limitation. When the preprocessing network is a pre-trained processing network, the training of the feature extraction network is the training of the Siamese neural network within it.

[0095] In this embodiment of the disclosure, the Siamese neural network is a network used to extract feature vectors related to page switching operations. The Siamese neural network includes a first sub-network and a second sub-network that share a structure and weights. The first sub-network is used to process the first feature vector obtained by the preprocessing network processing the first sample page, and the second sub-network is used to process the second feature vector obtained by the preprocessing network processing the second sample page.

[0096] In this embodiment of the disclosure, when the feature extraction network includes a preprocessing network and a Siamese neural network, step S103 may include steps S1031-S1032.

[0097] Step S1031: Process the first sample page and the second sample page based on the preprocessing network to obtain the feature vectors of the first sample page and the second sample page.

[0098] Step S1032: Extract the feature vector related to the page switching operation from the feature vectors of the first sample page and the second sample page based on the Siamese neural network.

[0099] In this embodiment of the present disclosure, feature vectors related to page switching operations are extracted from the feature vectors of the first sample page and the second sample page. This is achieved by processing the feature vector of the first sample page using a first sub-network to obtain a feature vector representing the features of the first sample page related to page switching operations, and by processing the feature vector of the second sample page using a second sub-network to obtain a feature vector representing the features of the second sample page related to page switching operations.

[0100] In the foregoing embodiments, the feature extraction network includes a preprocessing network and a Siamese neural network. In other embodiments of this disclosure, the feature extraction network may also include other types of networks.

[0101] Step S104: Based on the feature vectors of the first operation, the first sample page, and the second sample page, determine the first similarity between the real operation and the first operation.

[0102] In this embodiment of the disclosure, after obtaining the feature vector of the first operation, the feature vector of the first sample page, and the feature vector of the second sample page, the feature vector of the first operation, the feature vector of the first sample, and the feature vector of the second sample can be concatenated to obtain a comprehensive vector. Then, the comprehensive vector is input into a preset first similarity model to determine the first similarity between the actual operation that switches from the first sample page to the second sample page and the first operation.

[0103] The aforementioned first similarity model can be a determination model with known parameters or a determination model with unknown internal parameters. This disclosure does not impose any particular limitation on the model.

[0104] Step S105: Based on the first similarity and the first label information, adjust the parameters of the feature extraction network to make the feature extraction network converge.

[0105] In this embodiment of the disclosure, the first marker information is used to mark whether the first operation is a real operation that triggers the switch from the first sample page to the second sample page. The first marker information can determine the true similarity between the first operation and the real operation. The true similarity can only be 0 or 1, meaning the first marker information indicates whether the first operation is the same as or different from the real operation. For example, if the first marker information is true, the true similarity is 1; if the first marker information is false, the true similarity is 0.

[0106] Because the first label information represents the true similarity between the first operation and the real operation, the difference between the first similarity determined by the feature extraction network and the true similarity can be determined by using the first similarity and the second label information. Based on the aforementioned difference, error backpropagation can be performed to adjust the parameters of the feature extraction network through error backpropagation. After multiple iterations, the feature extraction network converges, resulting in the trained feature extraction network.

[0107] In some embodiments of this disclosure, step S105, which adjusts the parameters of the feature extraction network based on the first similarity and the first labeling information to make the feature extraction network converge, may include steps S1051-S1053.

[0108] Step S1051: Based on the first label information, determine the second similarity between the first operation and the real operation.

[0109] In this embodiment of the disclosure, the second similarity represents the true similarity between the first operation and the actual operation. As before, if the first label information is true, the second similarity is 1; and if the first label information is false, the second similarity is 0. Similarly, if the first label information is 1, the second similarity is 1; and if the first label information is 0, the second similarity is 0.

[0110] Step S1052: Determine the mean square error between the first similarity and the second similarity.

[0111] Step S1053: Adjust the parameters of the feature extraction network based on the mean square error.

[0112] In this embodiment of the disclosure, in order to use the backpropagation algorithm to perform error reflection propagation and thus train the feature extraction network until it converges, the mean square error of the first similarity and the second similarity is first calculated, and then the error is backpropagated based on the calculated mean square error to adjust the parameters in the feature extraction network.

[0113] It should be noted that in the embodiments of this disclosure, the feature extraction network includes a preprocessing network and a Siamese neural network, and when the preprocessing network is a pre-trained network, adjusting the parameters of the feature extraction network according to the mean square error in step S1053 specifically means adjusting the parameters of the Siamese neural network.

[0114] The feature extraction network training method provided in this disclosure uses a sample array for training the feature extraction network, excluding information from a first sample page, a second sample page, a first operation, and a first label. During feature extraction network model training, vector encoding is performed based on the first operation to obtain the corresponding feature vector. Then, feature extraction is performed on the first and second sample pages based on the feature extraction network to obtain feature vectors for the first and second sample pages. Subsequently, a first similarity is determined based on the first operation, the feature vectors of the first and second sample pages, and the parameters of the feature extraction network are adjusted based on the first similarity and the first label information to achieve feature extraction network training.

[0115] The feature extraction network provided in this embodiment can extract features related to the operation that triggers the page to switch to another page. Based on these features, it can accurately identify the actual operation that triggers the page to switch to another specific page, thus achieving matching of the page features extracted upstream with the downstream task.

[0116] As previously stated, in some embodiments of this disclosure, the first similarity model used to determine the first similarity between the real operation and the first operation based on the feature vectors of the first operation, the first sample page, and the second sample page can be a determination model with unknown internal parameters. In this case, while performing the aforementioned step S105, step S106 also needs to be performed.

[0117] Step S106: Adjust the parameters of the first similarity model based on the first similarity and the first label information.

[0118] In a specific embodiment, steps S105 and S106 can be executed simultaneously to achieve simultaneous adjustment of the parameters of the feature extraction network and the first similarity model.

[0119] In some embodiments of this disclosure, after the feature extraction network is trained, it is necessary to verify the accuracy of the features extracted by the feature extraction network in order to determine whether the feature extraction network needs to be retrained.

[0120] Figure 2 This is a flowchart of a method for verifying the accuracy of a feature extraction network provided in an embodiment of this disclosure. Figure 2 As shown, the method for verifying the accuracy of the feature extraction network includes steps S201-S206.

[0121] Step S201: Obtain the verification array, which includes two pages to be processed and a second marker information. The second marker information marks the actual operation that triggers the switching between the two pages to be processed.

[0122] In this embodiment, the verification array is used to determine the accuracy of the trained feature extraction network, and there are multiple verification arrays. Each verification array includes two pages to be processed and second labeling information. One of the two pages to be processed is the page before a certain real operation is performed, and the other is the page to which the user jumps after performing the aforementioned real operation. The second labeling information marks the real operation that triggered the switching between the two pages to be processed.

[0123] In this embodiment of the disclosure, the verification array can also be constructed based on data tuples generated from daily operations. As mentioned before, a large number of data tuples containing (first page, second page, actual operation) can be collected during daily operations. After obtaining the aforementioned data tuples, the first page in the data tuple can be used as the page before the actual operation is performed in the page to be processed, the second page can be used as the page after the actual operation is performed in the page to be processed, and the aforementioned actual operation can be used as the second tag information.

[0124] Step S202: Perform feature extraction processing on the two pages to be processed based on the feature extraction network to obtain the feature vectors of the two pages to be processed.

[0125] In this embodiment of the disclosure, the feature extraction network is used to extract features from the two pages to be processed by inputting the two pages to be processed into the feature extraction network respectively, so as to obtain feature vectors representing the two pages to be processed respectively.

[0126] Step S203: Based on the feature vectors of the two pages to be processed, determine the similarity between the actual operation that triggers the switching between the two pages and multiple preset operations.

[0127] In this embodiment of the disclosure, the multiple preset operations represent all possible operations in actual applications. To achieve step S203, which determines the similarity between the actual operation and the multiple preset operations, it is first necessary to obtain the feature vector corresponding to each preset operation.

[0128] In this embodiment of the disclosure, obtaining the feature vectors of each preset operation can be performed based on the steps described in step S102 above. Specifically, the dimension of the feature vectors and the elements in the feature vectors corresponding to each preset operation can be determined first based on all possible preset operations. Subsequently, vector encoding processing is performed according to the type of each preset operation to obtain the corresponding feature vector.

[0129] After obtaining the feature vectors of each preset operation, the feature vectors of each preset operation can be concatenated with the feature vectors of the two pages to be processed to obtain a comprehensive vector. Subsequently, the comprehensive vector is processed to obtain the corresponding similarity. In a specific embodiment, the comprehensive vector can be input into a first similarity model to obtain the corresponding similarity.

[0130] Step S204: Sort the multiple preset operations based on the similarity corresponding to each preset operation.

[0131] Step S205: Based on the second marker information, determine the position of the actual operation that triggers the switching between the two pending pages in the sorting.

[0132] Step S206: Determine the accuracy of the feature extraction network based on location.

[0133] In this embodiment, the preset operations are sorted based on their similarity. The sorting can be done in descending order of similarity to obtain a sorted sequence. After obtaining the sorted sequence, the second marker information and multiple preset operations in the sorted sequence can be compared to determine the position of the actual operation within the sorted sequence. Based on the position of the actual operation in the sorted sequence, the accuracy of the feature extraction network is determined.

[0134] The earlier the actual operation appears in the sorted sequence, the higher the accuracy of the feature extraction network. In practice, after obtaining the sorted queue from largest to smallest, the first few pre-operations in the queue can be extracted, and the actual operation can be compared with these first few pre-operations to determine the accuracy of the feature extraction network.

[0135] For example, in one specific embodiment, only the first pre-operation in the sorting queue can be extracted. The actual operation is then compared with the first-ranked pre-operation; if they are the same, the result is set to 1; otherwise, the result is set to 0. Finally, the number of results with a result of 1 is counted, and the accuracy of the feature extraction network is determined by comparing this number with the total number of results.

[0136] In specific applications of this disclosure, if the aforementioned steps determine that the accuracy of the feature extraction network is greater than a preset accuracy, then the feature extraction network is deemed to have passed verification and can be applied to page feature extraction. If the aforementioned steps determine that the accuracy of the feature extraction network is less than a preset accuracy, then the feature extraction network is deemed to have failed verification and requires further training using the aforementioned steps S101-S106.

[0137] In addition to providing the aforementioned training method for the feature extraction network, this disclosure also provides a method for identifying page operations. The page operation identification method uses the aforementioned trained feature extraction network to extract page features and determine the type of page operation.

[0138] Figure 3 This is a flowchart of a page operation recognition method provided in an embodiment of this disclosure. Figure 3As shown, the page operation identification method provided in this embodiment includes steps S301-S303.

[0139] Step S301: Obtain the first page and the second page obtained by switching from the first page.

[0140] Step S302: Extract feature vectors of the first page and the second page based on the preset feature extraction network.

[0141] In practice, we can obtain data tuples such as (first page, second page). These data tuples indicate that after performing an unknown page operation on the first page, we switch to the second page.

[0142] After obtaining the first page and the second page, the embodiments of this disclosure can input the first page and the second page into the feature extraction network trained using the method described above, and obtain the feature vectors of the first page and the second page.

[0143] Step S303: Based on the feature vector, determine the page operation that triggers the switch from the first page to the second page.

[0144] After obtaining the feature vectors of the first page and the second page, the aforementioned feature vectors can be input into the first similarity model mentioned above to obtain the probability of various preset operations for page operations, and the preset operation corresponding to the highest probability is selected as the page operation that triggers the switch from the first page to the second page.

[0145] The page operation recognition method provided in this embodiment extracts feature vectors of the first page and the second page based on a preset feature extraction network, and determines the page operation that triggers the switch from the first page to the second page based on the feature vectors of the first page and the second page, thus realizing the recognition of page operations.

[0146] Figure 4 This is a schematic diagram of the structure of a training device for a feature extraction network provided in an embodiment of this disclosure. This processing device can be understood as the aforementioned computing device or a functional module within the aforementioned computing device. Figure 4 As shown, the training device 400 includes a sample data acquisition unit 401, a first operation encoding unit 402, a sample page feature extraction unit 403, a first similarity calculation unit 404, and a model training unit 405.

[0147] The sample data acquisition unit 401 is used to acquire a sample array, which includes information about a first sample page, a second sample page, a first operation, and first marker information. The first marker information is used to mark whether the first operation is a real operation that triggers the switch from the first sample page to the second sample page.

[0148] The first operation encoding unit 402 is used to perform vector encoding processing on the first operation to obtain the feature vector corresponding to the first operation.

[0149] The sample page feature extraction unit 403 is used to extract features from the first sample page and the second sample page based on the feature extraction network, so as to obtain the feature vectors of the first sample page and the second sample page.

[0150] The first similarity calculation unit 404 is used to determine the first similarity between the real operation and the first operation based on the feature vectors of the first operation, the first sample page, and the second sample page.

[0151] The model training unit 405 is used to adjust the parameters of the feature extraction network based on the first similarity and the first label information so that the feature extraction network converges.

[0152] In some embodiments of this disclosure, the similarity calculation unit processes the feature vectors of the first operation, the first sample page, and the second sample page based on a preset first similarity model to obtain the first similarity between the actual operation that triggers the switch from the first sample page to the second sample page and the first operation.

[0153] In some embodiments of this disclosure, the similarity calculation unit is further configured to adjust the parameters of the first similarity model based on the first similarity and the first label information.

[0154] In some embodiments of this disclosure, the feature extraction network includes a preprocessing network and a Siamese neural network; the sample page feature extraction unit 403 includes a first feature extraction subunit and a second feature extraction subunit.

[0155] The first feature extraction subunit is used to process the first sample page and the second sample page based on the preprocessing network to obtain the feature vectors of the first sample page and the second sample page.

[0156] The second feature extraction subunit is used to extract feature vectors related to page switching operations from the feature vectors of the first sample page and the second sample page based on a Siamese neural network.

[0157] In some embodiments of this disclosure, the model training unit 405 includes a similarity determination subunit, a mean squared error calculation subunit, and a parameter adjustment subunit.

[0158] The similarity determination subunit determines the second similarity between the first operation and the actual operation based on the first label information. The mean squared error calculation subunit is used to determine the mean squared error between the first similarity and the second similarity; the parameter adjustment subunit is used to adjust the parameters of the Siamese neural network based on the mean squared error.

[0159] In some embodiments of this disclosure, the training apparatus 400 for the feature extraction network may further include a verification data acquisition unit, a feature extraction network usage unit, a second similarity calculation unit, a sorting unit, a position determination unit, and an accuracy judgment unit.

[0160] The verification data acquisition unit is used to acquire a verification array, which includes two pages to be processed and a second marker information. The second marker information marks the actual operation that triggers the switching between the two pages to be processed.

[0161] The feature extraction network uses a unit to perform feature extraction on two pages to be processed, and obtain the feature vectors of the two pages to be processed.

[0162] The second similarity calculation unit is used to determine the similarity between the real operation that triggers the switching of the two pages and multiple preset operations based on the feature vectors of the two pages to be processed. The multiple preset operations include the real operation that triggers the switching of the two pages.

[0163] The sorting unit is used to sort multiple preset operations based on the similarity corresponding to each preset operation;

[0164] The position determination unit is used to determine the position of the actual operation that triggers the switching between the two pending pages in the sorting, based on the second marker information;

[0165] The accuracy assessment unit is used to determine the accuracy of the feature extraction network based on location.

[0166] Figure 5 This is a schematic diagram of the structure of a page operation recognition device provided in an embodiment of this disclosure. This processing device can be understood as the aforementioned computing device or a functional module within the aforementioned computing device. Figure 5 As shown, the page operation recognition device 500 provided in this embodiment includes a page acquisition unit 501, a feature vector acquisition unit 502, and a page operation determination unit 503.

[0167] The page acquisition unit 501 is used to acquire the first page and the second page obtained by switching from the first page.

[0168] The feature vector acquisition unit 502 is used to extract feature vectors of the first page and the second page based on a preset feature extraction network.

[0169] The page operation determination unit 503 is used to determine the page operation that triggers the switch from the first page to the second page based on the feature vector.

[0170] The feature extraction network is trained based on the first sample page, the second sample page, the first operation, and the first label information. The first label information is used to mark whether the first operation is a real operation that triggers the switch from the first sample page to the second sample page.

[0171] The apparatus provided in this embodiment is capable of performing the above-described... Figures 1-3 The methods in any of the embodiments are similar in execution and beneficial effects, and will not be described again here.

[0172] This disclosure also provides a computing device, which includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it can perform the above-described functions. Figures 1-3 The method of any of the embodiments.

[0173] Example, Figure 6 This is a schematic diagram of the structure of a computing device according to an embodiment of this disclosure. See below for details. Figure 6 The diagram illustrates a structural schematic suitable for implementing the computing device 600 in the embodiments of this disclosure. The computing device 600 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The computing device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0174] like Figure 6 As shown, computing device 600 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of computing device 600. Processing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0175] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 608 including, for example, magnetic tape, hard disk, etc.; and communication devices 609. Communication device 609 allows computing device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 A computing device 600 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0176] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0177] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0178] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0179] The aforementioned computer-readable medium may be included in the aforementioned computing device; or it may exist independently and not assembled into the computing device.

[0180] The aforementioned computer-readable medium carries one or more programs. When the one or more programs are executed by the computing device, the computing device causes the following: to acquire a sample array, which includes information about a first sample page, a second sample page, a first operation, and first labeling information, wherein the first labeling information is used to mark whether the first operation is a real operation that triggers the switch from the first sample page to the second sample page; to perform vector encoding processing on the first operation to obtain a feature vector corresponding to the first operation; to perform feature extraction on the first sample page and the second sample page based on a feature extraction network to obtain feature vectors for the first sample page and the second sample page; to determine a first similarity between the real operation and the first operation based on the first operation, the first sample page, and the feature vectors of the second sample page; and to adjust the parameters of the feature extraction network based on the first similarity and the first labeling information to make the feature extraction network converge.

[0181] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0182] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0183] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0184] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0185] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0186] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can perform the above-described functions. Figures 1-4 The methods in any of the embodiments are similar in execution and beneficial effects, and will not be described again here.

[0187] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0188] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for training a feature extraction network, characterized in that, include: Obtain a sample array, which includes information about a first sample page, a second sample page, a first operation, and first marker information. The first marker information is used to mark whether the first operation is a real operation that triggers the switch from the first sample page to the second sample page. The first operation is subjected to vector encoding processing to obtain the feature vector corresponding to the first operation; wherein, the vector encoding processing is performed according to the type of the first operation; Based on the feature extraction network, feature vectors of the first sample page and the second sample page are extracted to obtain the feature vectors of the first sample page and the second sample page. Based on the feature vectors of the first operation, the first sample page, and the second sample page, a first similarity between the real operation and the first operation is determined. Based on the first similarity and the first labeling information, the parameters of the feature extraction network are adjusted to make the feature extraction network converge.

2. The method according to claim 1, characterized in that, Determining the first similarity between the real operation and the first operation based on the feature vectors of the first operation, the first sample page, and the second sample page includes: Based on a preset first similarity model, the feature vectors of the first operation, the first sample page, and the second sample page are processed to obtain the first similarity between the actual operation that triggers the switch from the first sample page to the second sample page and the first operation.

3. The method according to claim 2, characterized in that, After processing the feature vectors of the first operation, the first sample page, and the second sample page based on a preset first similarity model to obtain the first similarity between the actual operation that triggered the switch from the first sample page to the second sample page and the first operation, the method further includes: Based on the first similarity and the first label information, the parameters of the first similarity model are adjusted.

4. The method according to claim 1, characterized in that, The feature extraction network includes a preprocessing network and a Siamese neural network; The step of extracting features from the first sample page and the second sample page using a feature extraction network to obtain feature vectors for the first sample page and the second sample page includes: The first sample page and the second sample page are processed based on the preprocessing network to obtain the feature vectors of the first sample page and the second sample page; Based on the twin neural network, feature vectors related to page switching operations are extracted from the feature vectors of the first sample page and the second sample page.

5. The method according to claim 4, characterized in that, The step of adjusting the parameters of the feature extraction network based on the first similarity and the first labeling information includes: Based on the first labeling information, a second similarity between the first operation and the actual operation is determined; Determine the mean square error between the first similarity and the second similarity; The parameters of the Siamese neural network are adjusted based on the mean square error.

6. The method according to any one of claims 1-5, characterized in that, After obtaining the converged feature extraction network, the method further includes: Obtain a verification array, which includes two pages to be processed and a second marker information, the second marker information marking the actual operation that triggers the switching of the two pages to be processed; Based on the feature extraction network, feature extraction processing is performed on the two pages to be processed to obtain the feature vectors of the two pages to be processed. Based on the feature vectors of the two pages to be processed, the similarity between the actual operation that triggers the switching of the two pages to be processed and a number of preset operations is determined, including the actual operation that triggers the switching of the two pages to be processed. The preset operations are sorted based on their similarity. Based on the second marker information, determine the position of the actual operation that triggered the switching between the two pending pages in the sorting; Based on the location, the accuracy of the feature extraction network is determined.

7. A method for recognizing page operations, characterized in that, include: Get the first page, and the second page obtained by switching from the first page; Feature vectors of the first page and the second page are extracted based on a preset feature extraction network; Based on the feature vector, determine the page operation that triggers the switch from the first page to the second page; The feature extraction network is trained based on the training method of the feature extraction network according to any one of claims 1-6.

8. A training apparatus for a feature extraction network, characterized in that, include: A sample data acquisition unit is used to acquire a sample array, which includes information about a first sample page, a second sample page, a first operation, and first marker information. The first marker information is used to mark whether the first operation is a real operation that triggers the switching of the first sample page to the second sample page. A first operation encoding unit is configured to perform vector encoding processing on the first operation to obtain a feature vector corresponding to the first operation; wherein the vector encoding processing is performed according to the type of the first operation; The sample page feature extraction unit is used to extract features from the first sample page and the second sample page based on the feature extraction network to obtain feature vectors of the first sample page and the second sample page. The first similarity calculation unit is used to determine the first similarity between the real operation and the first operation based on the feature vectors of the first operation, the first sample page, and the second sample page. The model training unit is used to adjust the parameters of the feature extraction network based on the first similarity and the first label information, so that the feature extraction network converges.

9. The apparatus according to claim 8, characterized in that, The similarity calculation unit processes the feature vectors of the first operation, the first sample page, and the second sample page based on a preset first similarity model to obtain the first similarity between the actual operation that triggers the switch from the first sample page to the second sample page and the first operation.

10. The apparatus according to claim 9, characterized in that, The similarity calculation unit is also used to adjust the parameters of the first similarity model based on the first similarity and the first label information.

11. The apparatus according to claim 8, characterized in that, The feature extraction network includes a preprocessing network and a Siamese neural network; the sample page feature extraction unit includes: The first feature extraction subunit is used to process the first sample page and the second sample page based on the preprocessing network to obtain the feature vectors of the first sample page and the second sample page; The second feature extraction subunit is used to extract feature vectors related to page switching operations from the feature vectors of the first sample page and the second sample page based on the Siamese neural network.

12. The apparatus according to claim 11, characterized in that, The model training unit includes: The similarity determination subunit determines a second similarity between the first operation and the actual operation based on the first tag information; The mean square error calculation subunit is used to determine the mean square error between the first similarity and the second similarity. The parameter adjustment subunit is used to adjust the parameters of the Siamese neural network based on the mean square error.

13. The apparatus according to any one of claims 8-12, characterized in that, Also includes: A verification data acquisition unit is used to acquire a verification array, which includes two pages to be processed and a second marker information, the second marker information marking the actual operation that triggers the switching of the two pages to be processed; The feature extraction network usage unit is used to perform feature extraction processing on the two pages to be processed based on the feature extraction network, so as to obtain the feature vectors of the two pages to be processed. The second similarity calculation unit is used to determine the similarity between the real operation that triggers the switching of the two pages to be processed and a plurality of preset operations based on the feature vectors of the two pages to be processed. The plurality of preset operations include the real operation that triggers the switching of the two pages to be processed. The sorting unit is used to sort the multiple preset operations based on the similarity corresponding to each preset operation; The position determination unit is used to determine, based on the second marker information, the position of the actual operation that triggers the switching between the two pending pages in the sorting; An accuracy determination unit is used to determine the accuracy of the feature extraction network based on the location.

14. A page operation recognition device, characterized in that, include: A page acquisition unit is used to acquire a first page and a second page obtained by switching from the first page; The feature vector acquisition unit is used to extract feature vectors of the first page and the second page based on a preset feature extraction network. A page operation determination unit is used to determine, based on the feature vector, the page operation that triggers the switching from the first page to the second page; The feature extraction network is trained based on the training method of the feature extraction network according to any one of claims 1-6.

15. A computing device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the method as described in any one of claims 1-7.

16. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-7.

Images