Website Data Processing Method, Device and Equipment

Abstract

The disclosed embodiments provide a website data processing method, apparatus, device, and storage medium. The method comprises: obtaining website instance data, the website instance data including multiple website instances; evolving an initialized particle swarm using a competitive particle swarm algorithm and the website instance data, wherein the particles in the particle swarm represent an instance selection scheme for the website instance data; and performing instance selection in the website instance data based on the evolved particle swarm to obtain a target instance subset, which is used to train a classification model for phishing websites. Thus, the competitive particle swarm algorithm optimizes the instance selection scheme in the website instance data, and selects an instance subset from the website instance data according to the optimized instance selection scheme for training the classification model for phishing websites, thereby improving the quality of the training data for the classification model and the accuracy of phishing website detection based on the classification model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more particularly to a website data processing method, apparatus, device, and storage medium. Background Art

[0002] The existence of phishing websites brings certain security risks to users when surfing the Internet. With the development of artificial intelligence technology, artificial intelligence technology can be used to detect phishing websites.

[0003] In related technologies, a large amount of website data is obtained, an artificial intelligence model is trained based on the website data, a classification model for phishing websites is obtained, and then the classification model is used to detect phishing websites.

[0004] However, website data has shortcomings such as high noise and data redundancy, which leads to poor results of the trained classification model, thus affecting the accuracy of phishing website detection. Summary of the Invention

[0005] The embodiments of the present disclosure provide a website data processing method, apparatus, and device to address the problem that website data has shortcomings such as excessive noise data and data redundancy, which result in poor results of the trained classification model and thus affect the accuracy of phishing website detection.

[0006] In a first aspect, an embodiment of the present disclosure provides a website data processing method, comprising: obtaining website instance data, wherein the website instance data includes multiple website instances; evolving an initialized particle swarm through a competitive particle swarm algorithm and the website instance data, wherein the particles in the particle swarm represent an instance selection scheme for the website instance data; performing instance selection in the website instance data based on the evolved particle swarm to obtain a target instance subset, wherein the target instance subset is used for classification model training of phishing websites.

[0007] In some embodiments, an evolution process of the particle swarm includes: selecting instances from the website instance data according to the instance selection scheme represented by the particles in the particle swarm to obtain a subset of instances corresponding to the particles in the particle swarm; evaluating the subset of instances corresponding to the particles in the particle swarm on a target dimension related to instance selection to obtain a target value corresponding to the particles in the particle swarm on the target dimension; clustering the particles in the particle swarm according to the target value corresponding to the particles in the particle swarm on the target dimension to obtain multiple particle clusters; comparing the particles in the multiple particle clusters to determine a winning particle in the multiple particle clusters. The method comprises the steps of: performing an evolution operation on the winner particles in the plurality of particle clusters according to the evolution strategy corresponding to the winner particles, and obtaining evolved particles corresponding to the winner particles in the plurality of particle clusters; performing an evolution operation on the loser particles in the plurality of particle clusters according to the evolution strategy corresponding to the loser particles, and obtaining evolved particles corresponding to the loser particles in the plurality of particle clusters; performing an environmental selection operation on the particles in the particle swarm, the evolved particles corresponding to the winner particles in the plurality of particle clusters, and the evolved particles corresponding to the loser particles in the plurality of particle clusters, and determining particles in the particle swarm that have undergone the current evolution.

[0008] In some embodiments, the target dimension includes a first dimension representing a phishing network detection error and / or a second dimension representing a subset complexity, and the instance subset corresponding to the particles in the particle swarm is evaluated on the target dimension related to instance selection to obtain a target value corresponding to the particles in the particle swarm on the target dimension, including: using the instance subsets corresponding to the particles in the particle swarm as multiple independent training data sets, respectively training the classification model to obtain the model training error of the instance subset corresponding to the particles in the particle swarm; determining the target value corresponding to the particles in the particle swarm on the first dimension based on the model training error of the instance subset corresponding to the particles in the particle swarm; and / or determining the target value corresponding to the particles in the particle swarm on the second dimension based on complexity-related features of the instance subset corresponding to the particles in the particle swarm.

[0009] In some embodiments, the clustering operation is performed on the particles in the particle swarm according to the target values corresponding to the particles in the particle swarm on the target dimension to obtain multiple particle clusters, including: determining the angles between the particles in the particle swarm and multiple reference vectors according to the target values corresponding to the particles in the particle swarm on the target dimension, wherein the multiple reference vectors are uniformly distributed in the target space formed by the target dimension; clustering the multiple particles into multiple initial clusters according to the angles between the particles in the particle swarm and the multiple reference vectors, wherein the multiple initial clusters correspond one-to-one to the multiple reference vectors; and adjusting the particles in the initial clusters whose actual particle number is greater than the cluster capacity and the particles in the initial clusters whose actual particle number is less than the cluster capacity according to the multiple initial clusters and the multiple reference vectors to obtain the multiple particle clusters.

[0010] In some embodiments, the comparing of the particles in the multiple particle clusters to determine the winner particles in the multiple particle clusters and the loser particles in the multiple particle clusters includes: for any particle cluster in the multiple particle clusters, determining the target value corresponding to the ideal particle corresponding to the any particle cluster in the target dimension based on the target value corresponding to the particles in the any particle cluster in the target dimension; determining the Chebyshev value of the particles in the any particle cluster based on the target value corresponding to the particles in the any particle cluster in the target dimension, the target value corresponding to the ideal particle corresponding to the any particle cluster in the target dimension, and the multiple reference vectors; and determining the winner particle in the any particle cluster and the loser particle in the any particle cluster by comparing the Chebyshev values of the particles in the any particle cluster.

[0011] In some embodiments, the evolutionary strategy corresponding to the winner particle is an evolutionary strategy based on a Gaussian model, and the evolutionary operation is performed on the winner particles in the multiple particle clusters according to the evolutionary strategy corresponding to the winner particle to obtain evolved particles corresponding to the winner particles in the multiple particle clusters, including: constructing the Gaussian model, the mean matrix in the Gaussian model and the covariance matrix in the Gaussian model are determined according to the position matrix of the winner particles in the multiple particle clusters; performing random sampling according to the Gaussian model to obtain an updated particle position matrix; and guiding the evolution of the winner particles in the multiple particle clusters according to the updated particle position matrix to obtain evolved particles corresponding to the winner particles in the multiple particle clusters.

[0012] In some embodiments, the evolution strategy corresponding to the loser particles is an evolution strategy based on constraints, wherein the constraints include intra-domain learning conditions and inter-domain learning conditions. The evolution strategy corresponding to the loser particles is used to perform evolution operations on the loser particles in the multiple particle clusters to obtain evolved particles corresponding to the loser particles in the multiple particle clusters, including: when the current evolution satisfies the intra-domain learning conditions, for a same particle cluster among the multiple particle clusters, guiding the evolution of the loser particles in the same particle cluster according to the position matrix of the winner particles in the same particle cluster to obtain evolved particles corresponding to the loser particles in the same particle cluster; when the current evolution satisfies the inter-domain learning conditions, guiding the evolution of the loser particles in the same particle cluster according to the position matrix of the winner particles in the same particle cluster and the position matrix of the winner particles in adjacent clusters of the same particle cluster to obtain evolved particles corresponding to the loser particles in the same particle cluster; wherein the intra-domain learning condition includes that a conditional probability value in the current evolution is less than a probability threshold, and the inter-domain learning condition includes that the conditional probability value is greater than or equal to the probability threshold, and the conditional probability value is randomly generated in the current evolution.

[0013] In some embodiments, the performing of an environmental selection operation on the particles in the particle swarm, the evolved particles corresponding to the winner particles in the multiple particle clusters, and the evolved particles corresponding to the loser particles in the multiple particle clusters to determine the particles in the particle swarm that have undergone the current evolution includes: merging the particles in the particle swarm, the evolved particles corresponding to the winner particles in the multiple particle clusters, and the evolved particles corresponding to the loser particles in the multiple particle clusters into a temporary group; determining, in the temporary group, a non-dominated relationship of the particles in the temporary group based on the target values of the particles in the temporary group under the target dimension; stratifying the particles in the temporary group based on the non-dominated relationship of the particles in the temporary group to determine the non-dominated layer where the particles in the temporary group are located; and selecting the particles that have undergone the current evolution in the temporary group based on the non-dominated layer where the particles in the temporary group are located and the target values of the particles in the temporary group under the target dimension.

[0014] In a second aspect, an embodiment of the present disclosure provides a website data processing device, comprising: an acquisition unit for acquiring website instance data, wherein the website instance data includes multiple website instances; an evolution unit for evolving an initialized particle swarm through a competitive particle swarm algorithm and the website instance data, wherein the particles in the particle swarm represent an instance selection scheme for the website instance data; a selection unit for performing instance selection in the website instance data based on the evolved particle swarm to obtain a target instance subset, wherein the target instance subset is used for training a classification model for a phishing website.

[0015] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the website data processing method described in the first aspect above.

[0016] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the website data processing method described in the first aspect above is implemented.

[0017] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, which includes computer-executable instructions. When a processor executes the computer-executable instructions, the website data processing method described in the first aspect above is implemented.

[0018] In the website data processing method, apparatus, and device provided by the embodiments of the present disclosure, the particles in the particle swarm represent instance selection schemes for website instance data. The particle swarm is evolved through a competitive particle swarm algorithm to optimize the instance selection scheme for website instance data. Instance selection is performed in the website instance data according to the evolved particle swarm, which improves the accuracy of instance selection and enables representative instances to be selected from the website instance data. The instance subsets formed by these instances are significantly reduced in noise, redundant data, and other aspects compared to the website instance data. Using these instance subsets for training a classification model for phishing websites can effectively improve the classification model's accuracy in detecting phishing websites. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 A flowchart of a website data processing method provided by an embodiment of the present disclosure;

[0021] Figure 2 A schematic flow chart of a particle swarm optimization process in a website data processing method provided by an embodiment of the present disclosure;

[0022] Figure 3 This is an example diagram of the evolution process of the particle swarm;

[0023] Figure 4 A structural block diagram of a website data processing device provided by an embodiment of the present disclosure;

[0024] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0026] First, the related technologies involved in the embodiments of the present disclosure are introduced.

[0027] Phishing website detection refers to identifying whether a website is a phishing website based on the website's feature information. A classification model can be trained using a large amount of website data and used for phishing website detection.

[0028] Instance selection refers to selecting representative or discriminative subsets from large-scale data sets to reduce data redundancy, reduce noise interference, and speed up the computational efficiency of subsequent data processing.

[0029] The competitive particle swarm optimization (CPSO) algorithm is a stochastic optimization algorithm based on swarm intelligence. It combines the advantages of particle swarm optimization and competitive learning algorithms. Specifically, the algorithm maps the problem solution space into a search space for particles and solves the problem by evolving the particles in the search space using a competitive learning strategy.

[0030] In the relevant technologies for phishing website detection, the website data used for classification model training has shortcomings such as a lot of noisy data and data redundancy, which leads to a decrease in the training efficiency of the classification model and poor training results of the classification model, and the classification model has low detection accuracy for phishing websites.

[0031] In order to solve the above problems, the embodiment of the present disclosure combines instance selection and a competitive particle algorithm, optimizes the instance selection scheme through the competitive particle algorithm, and performs instance selection on the website instance data according to the optimized instance selection scheme to obtain a highly representative and less redundant instance subset. Such an instance subset is used for the classification model training of phishing websites, which can effectively improve the training efficiency and training effect of the classification model, and improve the detection accuracy of the classification model for phishing websites.

[0032] Figure 1 This is a flow chart of a website data processing method provided by an embodiment of the present disclosure. Figure 1 As shown, the website data processing method includes:

[0033] S101: Acquire website instance data, where the website instance data includes multiple website instances.

[0034] Among them, a website instance corresponds to a website. The website instance refers to the website sample data, which includes the characteristic information of the website, such as the Internet Protocol (IP) address, Uniform Resource Locator (URL) length, website traffic, etc.

[0035] Optionally, the website instance also includes a website tag of the website, which indicates whether the website is a phishing website, so that subsequent instance subsets can be used for supervised training of the classification model.

[0036] In this embodiment, the website instance data input by the user may be obtained; or the website instance may be read from a database; or the website instance data sent by other devices may be received; or the website instance data collected by a data collection service may be obtained.

[0037] S102, evolving the initialized particle swarm through the competitive particle swarm algorithm and the website instance data, where the particles in the particle swarm represent instance selection schemes for the website instance data.

[0038] Among them, the competitive particle swarm algorithm can refer to the above explanation and will not be repeated here.

[0039] The particle swarm includes multiple particles, one particle represents an instance selection scheme of the website instance data, and the instance selection scheme corresponding to the particle can be represented by the particle position matrix and the particle velocity matrix.

[0040] In this embodiment, a particle swarm is first randomly initialized, including the random initialization of the particle position matrix and the particle velocity matrix. Next, the particle swarm is evolved multiple times using a competitive particle swarm algorithm and website instance data to obtain a final evolved particle swarm. During each evolution of the particle swarm, instances are selected from the website instance data according to the instance selection scheme represented by the particles in the particle swarm. This yields a subset of instances corresponding to the particles in the particle swarm (i.e., the subset of instances corresponding to the instance selection scheme represented by the particles). Based on the performance of the schemes reflected by the instance subsets corresponding to the particles (i.e., the performance of the instance selection schemes represented by the particles), competition and evolution occur among the particles to optimize the instance selection scheme.

[0041] S103, based on the evolved particle swarm, performing instance selection in the website instance data to obtain a target instance subset, and the target instance subset is used for training a classification model for phishing websites.

[0042] In this embodiment, instance selection can be performed within the website instance data based on the instance selection scheme represented by the particles in the evolved particle swarm to obtain target instance subsets. Multiple particles can produce multiple target instance subsets. These target instance subsets can be used to train a classification model for phishing websites. Alternatively, a subset of these target instance subsets can be selected to train a classification model for phishing websites, thereby improving the training effect of the classification model.

[0043] In an embodiment of the present application, the instance selection scheme of the website instance data is represented by particles in a particle swarm, and the instance selection scheme of the website instance data is optimized and solved by evolving the particle swarm through a competitive particle swarm algorithm, thereby improving the accuracy of instance selection of the website instance data, and obtaining a highly representative and less redundant instance subset. By using the highly representative and less redundant instance subset, the training efficiency and training effect of the classification model are improved, and the detection accuracy of the classification model for phishing websites is improved.

[0044] In some embodiments, the example selection scheme of the website instance data includes selection schemes corresponding to multiple website instances in the website instance data. The selection schemes corresponding to the multiple website instances may include the probabilities of being selected corresponding to the multiple website instances and the selection labels corresponding to the multiple website instances, wherein the selection labels include participating in the selection or not participating in the selection.

[0045] In this embodiment, the position matrix of particles in the particle swarm can be used to represent the selection labels corresponding to multiple website instances, while the velocity matrix of particles in the particle swarm can be used to represent the selection probabilities corresponding to multiple website instances. Thus, combining the position matrix and velocity matrix of particles in the particle swarm accurately represents the instance selection scheme for website instance data.

[0046] Optionally, the number of elements in the particle position matrix is equal to the number of site instances in the site instance data, and the number of elements in the particle velocity matrix is equal to the number of site instances in the site instance data.

[0047] Furthermore, the element values in the particle position matrix are binary values, and the number of elements in the particle position matrix can be equal to the number of website instances in the website instance data. Multiple binary values in the particle position matrix represent selection labels corresponding to multiple website instances in the website instance data. The number of elements in the particle velocity matrix can be equal to the number of website instances in the website instance data.

[0048] For example: website instance data is represented as {I1, I2, I3, I4, I5, I6}, where I1, I2, I3, I4, I5, and I6 represent different website instances respectively; the position matrix of the first particle in the particle group is [1, 0, 1, 1, 0, 1], which indicates that I1, I3, I4, and I6 participate in the selection, and I2 and I5 do not participate in the selection; the position matrix of the second particle in the particle group is [0, 1, 1, 0, 1, 1], which indicates that I2, I3, I5, and I6 participate in the selection, and I1 and I4 do not participate in the selection.

[0049] The competitive particle swarm algorithm improves particle search efficiency (i.e., evolutionary efficiency) during the evolutionary process through a competitive mechanism. However, this can add additional computational overhead. This can lead to insufficient computing resources, particularly for instance selection optimization on large-scale instance datasets, resulting in limited optimization performance. To address this issue, the disclosed embodiments improve the evolutionary strategies for loser and winner particles during the particle evolution process. For details, please refer to the description of the subsequent embodiments.

[0050] Figure 2 This is a flow chart of a particle swarm optimization process in the website data processing method provided by the embodiment of the present disclosure. Figure 2 As shown in Figure 2, the optimization process of the particle swarm includes:

[0051] S201 , selecting instances from website instance data according to the instance selection scheme represented by the particles in the particle swarm, and obtaining instance subsets corresponding to the particles in the particle swarm.

[0052] In this embodiment, a particle swarm includes multiple particles, each representing an instance selection scheme. For each particle in the particle swarm, instances are selected from the website instance data according to the instance selection scheme represented by the particle. The selected instances constitute the instance subset corresponding to the particle. In this way, an instance subset corresponding to each particle in the particle swarm is obtained.

[0053] In one possible implementation, the position matrix of particles in a particle swarm represents the selection labels corresponding to multiple website instances, and the velocity matrix of particles in the particle swarm represents the probability of selection corresponding to multiple website instances. Based on the position matrix and velocity matrix of particles in the particle swarm, instances can be selected from the website instance data to obtain a subset of instances corresponding to the particles in the particle swarm.

[0054] In this implementation, for any particle in a particle swarm, an instance is selected from the website instance data based on the selection label represented by the value of the i-th element in the position matrix of that particle and the probability of selection represented by the value of the i-th element in the velocity matrix of that particle. This yields the i-th instance in the instance subset corresponding to that particle, where i is greater than or equal to 1. This process yields the instance subset corresponding to each particle in the particle swarm. Thus, the instance selection scheme is accurately represented by the velocity matrix and position matrix of the particles in the particle swarm, improving the accuracy of instance selection from website instances to obtain the instance subset.

[0055] S202 , evaluating the instance subset corresponding to the particles in the particle swarm on the target dimension related to the instance selection, and obtaining the target values corresponding to the particles in the particle swarm on the target dimension.

[0056] The target dimension related to the instance selection may be one or more dimensions. The target dimension is an evaluation dimension of the instance selection, and the instance subset of the instance selection is evaluated on the target dimension.

[0057] Among them, the target value corresponding to the particle in the target dimension reflects the quality of the instance subset corresponding to the particle in the target dimension, and further reflects the quality of the instance selection scheme represented by the particle.

[0058] In this embodiment, the instance subset corresponding to the particles in the particle swarm may be evaluated according to the subset evaluation method corresponding to the target dimension to obtain the target values corresponding to the particles in the particle swarm in the target dimension.

[0059] In one example, the subset evaluation method corresponding to the target dimension can include one or more of the following: a subset evaluation formula corresponding to the target dimension, a subset evaluation condition corresponding to the target dimension, and a subset evaluation model corresponding to the target dimension. These subset evaluation methods can be used to improve the accuracy and diversity of the evaluation of instance subsets corresponding to particles. These evaluation methods are described below.

[0060] When the subset evaluation method corresponding to the target dimension includes a subset evaluation formula corresponding to the target dimension, the subset evaluation formula corresponding to the target dimension can be calculated based on the instance subset corresponding to the particles in the particle swarm to obtain the target value corresponding to the particles in the particle swarm on the target dimension.

[0061] When the subset evaluation method corresponding to the target dimension includes a subset evaluation condition corresponding to the target dimension, the subset evaluation condition satisfied by the instance subset corresponding to the particle in the particle population can be determined within the subset evaluation condition corresponding to the target dimension. Based on the subset evaluation condition satisfied by the instance subset corresponding to the particle, the target value corresponding to the particle in the target dimension can be determined. For example, if the target dimension corresponds to multiple subset evaluation conditions, each of which corresponds to a different target value, the target value corresponding to the particle in the target dimension is the target value corresponding to the subset evaluation condition satisfied by the instance subset corresponding to the particle.

[0062] In the case where the subset evaluation method corresponding to the target dimension includes a subset evaluation model corresponding to the target dimension, the subset evaluation model corresponding to the target dimension can be a classification model of a phishing website. The classification model is trained using an instance subset corresponding to a particle in a particle swarm, and the target value corresponding to the particle in the target dimension is determined based on the training indicator of the classification model.

[0063] In one possible implementation, the target dimension includes a first dimension representing phishing network detection error and / or a second dimension representing subset complexity. S202 includes: using instance subsets corresponding to particles in a particle swarm as multiple independent training data sets, training the classification model separately, obtaining model training errors for the instance subsets corresponding to the particles in the particle swarm; determining target values corresponding to the particles in the particle swarm along the first dimension based on the model training errors for the instance subsets corresponding to the particles in the particle swarm; and / or determining target values corresponding to the particles in the particle swarm along the second dimension based on complexity-related features of the instance subsets corresponding to the particles in the particle swarm. The phishing network detection error can reflect the effectiveness of the instance subset in training the phishing network classification model, and the subset complexity can reflect the data volume of the instance subset. Evaluating the instance subset based on these two target dimensions can accurately reflect the quality of the instance selection scheme corresponding to the particle, thereby improving the accuracy of subsequent particle competition evolution.

[0064] In this implementation, when the target dimension includes a first dimension representing phishing network detection error, for any particle in the particle swarm, the instance subset corresponding to the particle is used as a training dataset, and a classification model is trained based on the training dataset to obtain the training error of the classification model during the training process. This training error is the model training error of the instance subset corresponding to the particle, and the target value corresponding to the particle in the first dimension is determined to be the model training error. When the target dimension includes a second dimension representing subset complexity, for any particle in the particle swarm, the target value corresponding to the particle in the second dimension is determined based on the complexity-related features of the instance subset corresponding to the particle. The complexity-related features of the instance subset may include one or more of the following: the number of website instances in the instance subset, the number of website instances in the instance subset with a website label of phishing websites, and the number of website instances in the instance subset with a website label of normal websites.

[0065] When the target dimension includes the above-mentioned first dimension and / or the above-mentioned second dimension, the objective function of the particle swarm evolution in the competitive particle swarm algorithm can be: minimizing the target value corresponding to the particles in the particle swarm in the first dimension and / or minimizing the target value of the particles in the particle swarm in the second dimension, so as to extract a representative instance subset with good model training effect, less noise, low redundancy and from the website instance data.

[0066] For example, the objective function can be expressed as:

[0067]

[0068] in, represents the instance subset corresponding to the particle, Indicates the target value corresponding to the particle in the first dimension, Indicates the target value corresponding to the particle in the second dimension.

[0069] S203 , performing a clustering operation on the particles in the particle swarm according to the target values corresponding to the particles in the particle swarm in the target dimension, to obtain a plurality of particle clusters.

[0070] In this embodiment, the target values corresponding to the particles in the particle swarm in the target dimension can be used as one of the reference data for clustering. The particles in the particle swarm can be clustered according to the set clustering rules or clustering algorithms to obtain multiple particle clusters.

[0071] In one possible implementation, S203 includes: determining the angles between the particles in the particle swarm and multiple reference vectors based on the target values corresponding to the particles in the particle swarm in the target dimension; clustering the multiple particles into multiple initial clusters based on the angles between the particles in the particle swarm and the multiple reference vectors, wherein the multiple initial clusters correspond one-to-one to the multiple reference vectors; and adjusting the particles in the initial clusters where the actual number of particles exceeds the cluster capacity and the particles in the initial clusters where the actual number of particles is less than the cluster capacity based on the multiple initial clusters and the multiple reference vectors, thereby obtaining multiple particle clusters. Thus, by introducing reference vectors uniformly distributed in the target space and combining the angles between the particles in the particle swarm and the multiple reference vectors, the particles in the particle swarm are clustered, thereby improving the accuracy of particle clustering. The multiple reference vectors are uniformly distributed in the target space formed by the target dimension.

[0072] Among them, the target space constituted by the target dimension may include the coordinate space constituted by the target dimension, and the coordinate space constituted by the target dimension refers to the coordinate system space with the target dimension as the coordinate axis, and the coordinate space includes at least one quadrant space in the coordinate system with the target dimension as the coordinate axis.

[0073] Optionally, when the target dimensions include a first dimension and a second dimension, the target space formed by the target dimensions includes at least one quadrant space in a two-dimensional coordinate system with the first dimension and the second dimension as coordinate axes. If the number of target dimensions is three, the target space may include at least one quadrant space in a three-dimensional coordinate system with the three target dimensions as coordinate axes, and so on.

[0074] The reference vector is a vector with its starting point as the origin and uniformly distributed in the target space.

[0075] In the process of determining the angles between the particles in the particle swarm and multiple reference vectors based on the target values corresponding to the particles in the particle swarm on the target dimension, the target values corresponding to the multiple reference vectors on the target dimension can be determined. For any particle in the particle swarm, the angles between the any particle and the multiple reference vectors can be calculated based on the target value corresponding to the any particle on the target dimension and the target values corresponding to the multiple reference vectors on the target dimension, thereby obtaining the angles between the any particle and the multiple reference vectors.

[0076] Optionally, the angles between particles in a particle swarm and multiple reference vectors are calculated as follows:

[0077]

[0078] in, represents the jth reference vector, represents particles, Represents the target value corresponding to the particle in the kth target dimension, represents the target value corresponding to the reference vector on the kth target dimension, M represents the number of target dimensions, Represents the angle between a particle and multiple reference vectors.

[0079] In the process of clustering multiple particles into multiple initial clusters based on the angles between the particles in the particle swarm and multiple reference vectors, the multiple reference vectors correspond to different clusters. For any particle in the particle swarm, the angles between the particle and the multiple reference vectors can be used to divide the particle into the cluster corresponding to the reference vector with the smallest angle with the particle. After the particles in the particle swarm are divided, multiple initial clusters can be obtained, and the number of particles in different initial clusters may be different.

[0080] Optionally, the clustering formula for multiple initial clusters is expressed as:

[0081]

[0082] in, represents the jth initial cluster, represents a particle swarm, represents the reference vector set, represents the angle between particle s and the jth reference vector.

[0083] In the process of obtaining multiple particle clusters by adjusting particles in initial clusters whose actual particle counts exceed the cluster capacity and particles in initial clusters whose actual particle counts are less than the cluster capacity based on multiple initial clusters and multiple reference vectors, the cluster capacity can be pre-set. For initial clusters whose actual particle counts exceed the cluster capacity, the particles to be removed from the initial cluster can be determined based on the angle between the particles in the initial cluster and the reference vector corresponding to the initial cluster. A target cluster can be determined in the initial cluster whose actual particle counts are less than the cluster capacity based on the angle between the particles to be removed and the reference vector corresponding to the initial cluster whose actual particle counts are less than the cluster capacity, and the particles to be removed can be assigned to the target cluster. After completing the adjustment of the particles in the multiple initial clusters, multiple particle clusters are obtained.

[0084] Optionally, the cluster capacity is 2, that is, each of the multiple particle clusters finally obtained includes two particles, so that the winner particle and the loser particle in each particle cluster can be determined in a subsequent competition process.

[0085] In one example, during the adjustment process, an initial cluster whose actual number of particles is greater than the cluster capacity retains only two particles closest to the reference vector corresponding to the initial cluster, and the remaining particles are moved to other unfilled clusters.

[0086] S204 , comparing the particles in the plurality of particle clusters respectively to determine winner particles in the plurality of particle clusters and loser particles in the plurality of particle clusters.

[0087] In this embodiment, among multiple particle clusters, performance comparisons can be performed on particles in the same particle cluster respectively, and particles that win the performance comparison are determined as winner particles in the particle cluster, and particles that fail the performance comparison are determined as loser particles in the particle cluster.

[0088] Optionally, the performance index of the particle adopts the Chebyshev value of the particle, and the winner particle and the loser particle in the same particle cluster are determined by comparing the Chebyshev values of particles in the same particle cluster.

[0089] In one possible implementation, S204 includes: for any particle cluster among the multiple particle clusters, determining, based on the target values corresponding to the particles in the particle cluster in the target dimension, the target values corresponding to the ideal particles corresponding to the particle cluster; determining, based on the target values corresponding to the particles in the particle cluster in the target dimension, the target values corresponding to the ideal particles in the particle cluster in the target dimension, and multiple reference vectors, the Chebyshev values of the particles in the particle cluster; and determining, by comparing the Chebyshev values of the particles in the particle cluster, a winner particle in the particle cluster and a loser particle in the particle cluster. This improves the accuracy of the Chebyshev values used to represent particle performance and improves the accuracy of the comparison to determine the winner and loser particles.

[0090] In the process of determining the target value corresponding to the ideal particle corresponding to any particle cluster in the target dimension according to the target value corresponding to the particles in any particle cluster in the target dimension, for any particle cluster, a minimum value can be determined among the target values corresponding to the particles in the any particle cluster in the target dimension, and the target value corresponding to the ideal particle corresponding to the any particle cluster in the target dimension is determined to be the minimum value.

[0091] In the process of determining the Chebyshev value of a particle in any particle cluster, for any particle cluster, the target value difference between the particle in the any particle cluster and the ideal particle in the target dimension can be calculated based on the target value corresponding to the particle in the any particle cluster in the target dimension and the target value corresponding to the ideal particle corresponding to the any particle cluster in the target dimension, and the target value difference reflects the gap between the particle and the ideal particle in the target dimension; the target value of the reference vector corresponding to the particle cluster to which the particle belongs in the target dimension can be determined according to the target value determination method of the target dimension; the Chebyshev value of the particle is determined based on the target value difference between the particle and the ideal particle in the target dimension and the target value of the reference vector corresponding to the particle cluster to which the particle belongs in the target dimension.

[0092] Optionally, there are multiple target dimensions, and the Chebyshev value of a particle may be determined as the maximum value of the target value difference between the particle and the ideal particle in the target dimension.

[0093] Optionally, the calculation formula of the particle's Chebyshev value is expressed as:

[0094]

[0095] in, represents an ideal particle, represents the target value corresponding to the ideal particle in the i-th target dimension, represents the target value corresponding to particle s in the i-th target dimension, represents the reference vector corresponding to the j-th particle cluster, Represents the target value of the reference vector corresponding to the j-th particle cluster in the target dimension.

[0096] In the process of determining the winner particle and the loser particle in any particle cluster by comparing the Chebyshev values of the particles in any particle cluster, the larger the Chebyshev value of the particle, the worse the performance of the particle. The Chebyshev values of the particles in the same particle cluster will be compared, and the particles with larger Chebyshev values in the same particle cluster will be determined as loser particles, and the particles with smaller Chebyshev values in the same particle cluster will be determined as winner particles.

[0097] S205 , performing an evolution operation on the winner particles in the plurality of particle clusters according to the evolution strategy corresponding to the winner particles, to obtain evolved particles corresponding to the winner particles in the plurality of particle clusters.

[0098] In this embodiment, winner and loser particles can adopt different evolutionary strategies. By designing different evolutionary strategies for the winner and loser particles, personalized evolution of the winner and loser particles can be achieved, improving the evolutionary effect. According to the evolutionary strategy corresponding to the winner particle, the winner particle in multiple particle clusters can be evolved. Specifically, the position matrix and velocity matrix of the winner particle are updated to obtain an updated particle. The updated particle is the evolved particle corresponding to the winner particle.

[0099] Optionally, the evolutionary strategy corresponding to the winner particle is an evolutionary strategy based on a Gaussian model. In the evolutionary strategy based on a Gaussian model, the Gaussian model is used to assist the evolution of the winner particle, thereby improving the diversity of the winner particle evolution and further improving the particle swarm evolution effect.

[0100] In one possible implementation, S205 includes: constructing a Gaussian model, wherein the mean matrix and covariance matrix in the Gaussian model are determined based on the position matrix of the winner particle in multiple particle clusters; performing random sampling based on the Gaussian model to obtain an updated particle position matrix; and guiding the evolution of the winner particles in the multiple particle clusters based on the updated particle position matrix to obtain evolved particles corresponding to the winner particles in the multiple particle clusters. Thus, the Gaussian model is used to assist the evolution of the winner particles, thereby improving the diversity of the winner particle evolution and overcoming the drawback of the competitive particle swarm optimization algorithm that ignores the quality of the winner particle, resulting in poor optimization performance. Furthermore, the evolutionary strategy based on the Gaussian model has a low computational load, which helps save computing resources and improve evolutionary efficiency.

[0101] In the process of constructing the Gaussian model, the mean matrix is calculated based on the position matrix of the winner particle in multiple particle clusters; the covariance matrix is calculated based on the position matrix and the mean matrix of the winner particle in multiple particle clusters; the mean matrix and the covariance matrix can be combined to construct a Gaussian model.

[0102] Optionally, the mean of the position matrices of the winner particles in multiple particle clusters is calculated to obtain a mean matrix. The calculation formula is expressed as:

[0103]

[0104] Where N represents the number of particles in the particle swarm, N / 2 represents the number of winner particles, represents the position matrix of the i-th winner particle, represents the mean matrix.

[0105] Optionally, the calculation formula of the covariance matrix is expressed as:

[0106]

[0107] Where cov represents the covariance matrix.

[0108] Optionally, the Gaussian model is expressed as:

[0109]

[0110] Here, x represents the input data of the Gaussian model.

[0111] During subsequent random sampling and winner particle evolution, random sampling can be performed based on a Gaussian model to obtain an updated particle position matrix. This updated particle position matrix can be used to guide the evolution of the winner particle, resulting in an evolved particle of the winner. During this guidance process, the velocity matrix of the evolving particle can be determined based on the updated particle position matrix, the position matrix of the winner particle, and the velocity matrix of the winner particle. Furthermore, the position matrix of the evolving particle can be determined based on the velocity matrix of the evolving particle and the velocity matrix of the winner particle. Thus, based on the updated particle position matrix, the position and velocity evolution of the winner particle can be guided, increasing the diversity of the position and velocity evolution of the winner particle.

[0112] Optionally, the position matrix difference between the updated particle position matrix and the position matrix of the winner particle is determined, and the velocity matrix of the evolving particle is obtained by weighting the position matrix difference and the velocity matrix of the winner particle. The formula for the winner particle evolution process can be expressed as:

[0113]

[0114] in, represents the position matrix of the winner particle, represents updating the particle position matrix, represents the velocity matrix of the winner particle, and They represent the weight parameters corresponding to the velocity matrix and the position matrix difference of the winner particle, respectively. The two weight parameters can be random values between 0 and 1. Represents the velocity matrix of the evolving particles corresponding to the successful particles, Represents the position matrix of the evolved particles corresponding to the successful particles.

[0115] S206 , performing an evolution operation on the loser particles in the plurality of particle clusters according to the evolution strategy corresponding to the loser particles, to obtain evolved particles corresponding to the loser particles in the plurality of particle clusters.

[0116] In this embodiment, the loser particles in multiple particle clusters can be evolved according to the evolutionary strategy corresponding to the loser particles, that is, the position matrix and velocity matrix of the loser particles are updated to obtain updated particles, which are the evolved particles corresponding to the winner particles.

[0117] Optionally, the evolutionary strategy corresponding to the loser particles is a constraint-based evolutionary strategy, which improves the convergence speed of the loser particles in the evolutionary process through the constraint-based evolutionary strategy, thereby improving the ability of the competitive particle swarm to solve the instance selection scheme optimization problem of large-scale website instance data.

[0118] In one possible implementation, the constraints include intra-domain learning conditions and inter-domain learning conditions. S206 includes: if the current evolution satisfies the intra-domain learning conditions, for a given particle cluster, the position matrix of the winner particle in the same particle cluster is used to guide the evolution of loser particles in the same particle cluster, thereby obtaining evolved particles corresponding to the loser particles in the same particle cluster; if the current evolution satisfies the inter-domain learning conditions, the position matrix of the winner particle in the same particle cluster and the position matrix of the winner particles in adjacent clusters of the same particle cluster are used to guide the evolution of loser particles in the same particle cluster, thereby obtaining evolved particles corresponding to the loser particles in the same particle cluster. This evolutionary process based on intra-domain learning conditions and inter-domain learning conditions improves the convergence speed of loser particles and ensures the diversity of their evolution. This addresses the poor performance of competitive particle algorithms in solving instance selection optimization problems for large-scale website instance data (particularly multi-objective optimization problems with multiple objective dimensions).

[0119] Among them, the intra-domain learning condition includes that the conditional probability value in the current evolution is less than the probability threshold, and the inter-domain learning condition includes that the conditional probability value is greater than or equal to the probability threshold. The conditional probability value is randomly generated in the current evolution to improve the diversity of the loser particle evolution.

[0120] When the current evolution meets the intra-domain learning conditions, the position matrix of the winner particles in the same particle cluster is used to guide the evolution of the loser particles in the same particle cluster, and the evolved particles corresponding to the loser particles in the same particle cluster are obtained, thereby realizing the evolution of the loser particles through intra-domain learning and improving the convergence speed of the loser particle evolution.

[0121] The evolutionary process learned within the domain may include: in the same particle cluster, the velocity matrix of the evolved particle corresponding to the loser particle may be determined based on the position matrix of the winner particle, the position matrix of the loser particle, and the velocity matrix of the loser particle; and the position matrix of the evolved particle may be determined based on the velocity matrix of the evolved particle and the position matrix of the loser particle.

[0122] Optionally, in the same particle cluster, the position matrix difference between the position matrix of the winner particle and the position matrix of the loser particle is determined, and the velocity matrix of the evolving particle corresponding to the loser particle is obtained by weighting the position matrix difference and the velocity matrix of the loser particle.

[0123] Alternatively, the formula for the evolution process in the loser particle domain is expressed as:

[0124]

[0125] in, represents the position matrix of the loser particle, represents the velocity matrix of the loser particle, and They represent the weight parameters corresponding to the velocity matrix and the position matrix difference of the loser particle, respectively. The two weight parameters can be random values between 0 and 1. represents the velocity matrix of the evolving particles corresponding to the loser particles, Represents the position matrix of the evolving particles corresponding to the loser particles.

[0126] When the current evolution meets the inter-domain learning conditions, the evolution of the loser particles in the same particle cluster is guided according to the position matrix of the winner particles in the same particle cluster and the position matrix of the winner particles in the adjacent clusters of the same particle cluster, and the evolved particles corresponding to the loser particles in the same particle cluster are obtained, thereby realizing the evolution of the loser particles through inter-domain learning.

[0127] The evolutionary process of inter-domain learning may include: determining a velocity matrix of an evolving particle corresponding to a loser particle based on a position matrix of a winner particle in the same particle cluster, a position matrix of winner particles in adjacent clusters of the same particle cluster, a position matrix of a loser particle in the same particle cluster, and a velocity matrix of the loser particle; and determining a position matrix of the evolving particle based on the velocity matrix corresponding to the loser particle and the position matrix of the loser particle.

[0128] Alternatively, the inter-domain evolution process of loser particles can be expressed as:

[0129]

[0130] in, represents the position matrix of the winners in the adjacent clusters, and They represent weight parameters respectively, and the two weight parameters can be random values between 0 and 1.

[0131] S207 , performing an environmental selection operation on particles in the particle swarm, evolved particles corresponding to winner particles in multiple particle clusters, and evolved particles corresponding to loser particles in multiple particle clusters to determine particles in the particle swarm that have undergone the current evolution.

[0132] In this embodiment, particles in a particle swarm refer to particles before the current evolution, including winner particles and loser particles. During the environmental selection process, instance selection can be performed on website instance data based on the instance selection scheme represented by the evolved particle corresponding to the winner particle to obtain an instance subset corresponding to the evolved particle. This instance subset can then be evaluated on the target dimension to obtain the target value corresponding to the evolved particle on the target dimension. Instance selection can be performed on website instance data based on the instance selection scheme represented by the evolved particle corresponding to the loser particle to obtain an instance subset corresponding to the evolved particle. This instance subset can then be evaluated on the target dimension to obtain the target value corresponding to the evolved particle on the target dimension. Based on the target values corresponding to the particles in the particle swarm, the target values corresponding to the evolved particles corresponding to the winner particle, and the target values corresponding to the evolved particles corresponding to the loser particle, particle selection is performed among the particles in the particle swarm, the evolved particles corresponding to the winner particle, and the evolved particles corresponding to the loser particle to obtain particles after the current evolution. The particles after the current evolution constitute the particle swarm after the current evolution.

[0133] In one possible implementation, S207 may include: merging particles in a particle swarm, evolved particles corresponding to winner particles in multiple particle clusters, and evolved particles corresponding to loser particles in multiple particle clusters into a temporary group; determining the non-dominated relationships of the particles in the temporary group based on the target values of the particles in the temporary group under the target dimension; stratifying the particles in the temporary group based on the non-dominated relationships of the particles in the temporary group to determine the non-dominated layer in which the particles in the temporary group are located; and selecting particles in the temporary group that have undergone the current evolution based on the non-dominated layer in which the particles in the temporary group are located and the target values of the particles in the temporary group under the target dimension. Thus, by stratifying the particles based on their non-dominated relationships, particles that perform better in the target dimension are selected as the particles that have undergone the current evolution, thereby improving the accuracy and efficiency of environmental selection.

[0134] In this implementation, the particles in the particle swarm, the evolved particles corresponding to the winner particles in multiple particle clusters, and the evolved particles corresponding to the loser particles in multiple particle clusters are first merged to form a temporary swarm. Assuming the number of particles in the particle swarm is N, the number of particles in the merged temporary swarm is 2N. Based on the instance selection scheme represented by the particles in the temporary swarm, instance selection is performed on the website instance data to obtain an instance subset corresponding to the particles in the temporary swarm. The instance subset corresponding to the particles in the temporary swarm is evaluated in the target dimension to obtain the target value of the particles in the temporary swarm in the target dimension. Based on the target value of the particles in the temporary swarm in the target dimension, the non-domination relationship of the particles in the temporary swarm is determined. The non-domination relationship of the particles includes whether the particle is dominated by other particles. If the particle is dominated by other particles, the non-domination relationship also includes the identification information of the other particles that dominate the particle. The other particles refer to the remaining particles in the temporary swarm except the particle in question.

[0135] Optionally, determining the non-domination relationship of particles in the temporary group based on target values of particles in the temporary group in target dimensions may include: if the target values of the first particle in all target dimensions are not inferior to the target values of the second particle in corresponding target dimensions, and the target value of the first particle in at least one target dimension is superior to the target value of the second particle in the at least one target dimension, then determining that the first particle dominates the second particle. The first particle and the second particle are any two particles in the temporary group.

[0136] If there is no particle dominating the first particle in the temporary group, then the non-domination relationship of the first particle can be determined as the first particle is not dominated by other particles.

[0137] When the target dimensions include a first dimension and a second dimension, the first particle is determined to dominate the second particle if the first and second particles satisfy the following first and second conditions. The first condition is: the target value of the first particle in the first dimension is not greater than the target value of the second particle in the first dimension, and the target value of the first particle in the second dimension is not greater than the target value of the second particle in the second dimension. The second condition is: the target value of the first particle in the first dimension is less than the target value of the second particle in the first dimension, or the target value of the first particle in the second dimension is less than the target value of the second particle in the second dimension. The first and second dimensions can be described with reference to the previous embodiments and will not be further described.

[0138] In this implementation, after determining the non-domination relationships of particles in the temporary group, particles whose non-domination relationships indicate that they are not dominated by other particles can be divided into the first non-dominated layer, and particles whose non-domination relationships indicate that they are not dominated by particles in the i-th non-dominated layer can be divided into the i+1-th non-dominated layer, where i is greater than or equal to 1. Ultimately, all particles in the temporary group are divided into the corresponding non-dominated layers. Subsequently, particles that have undergone the current evolution can be selected from the temporary group based on the hierarchical order of the non-dominated layers in which the particles in the temporary group reside and the target values of the particles in the temporary group under the target dimension.

[0139] Optionally, particles that have undergone the current evolution are selected from the temporary swarm according to the hierarchical order of the non-dominated layers in which the particles in the temporary swarm reside and the target values of the particles in the temporary swarm under the target dimension. This includes: starting from the first non-dominated layer, all particles in the entire non-dominated layer are selected as particles after the current evolution until a stopping condition is satisfied. The stopping condition is: if all particles in the Lth non-dominated layer are selected as particles after the current evolution, the number of particles after the current evolution will be greater than the particle swarm size (i.e., the number of particles in the particle swarm), and L is greater than or equal to 1. The number of particles remaining to be selected is determined based on the particle swarm size and the number of particles after the current evolution; and the number of particles remaining to be selected from the Lth non-dominated layer according to the target value of the particles in the Lth non-dominated layer under the target dimension. Finally, the selection of particles after the current evolution is completed. Thus, combining the hierarchical order and the target value improves the accuracy and efficiency of selecting particles after the current evolution.

[0140] The formula for calculating the remaining number of particles to be selected can be expressed as:

[0141]

[0142] In this formula, N represents the size of the particle swarm, K represents the number of remaining particles to be selected, Represents the i-th non-dominated layer. The number of particles remaining to be selected is obtained by subtracting the total number of particles in the first L-1 non-dominated layers (that is, the number of particles that have been selected after the current evolution) from N.

[0143] Optionally, selecting a remaining number of particles to be selected from the Lth non-dominated layer according to a target value of the particles in the Lth non-dominated layer in the target dimension may include: sorting the particles in the Lth non-dominated layer according to the target value of the particles in the Lth non-dominated layer in the target dimension to obtain a sorting order of the particles in the Lth non-dominated layer in the target dimension; determining a target congestion of the particles in the Lth non-dominated layer according to the sorting order of the particles in the Lth non-dominated layer in the target dimension; and selecting a remaining number of particles to be selected from the Lth non-dominated layer in descending order of the target congestion based on the target congestion of the particles in the Lth non-dominated layer, where the remaining number of particles to be selected are the particles after the current evolution.

[0144] The target congestion of a particle indicates its sparsity in the target space. A high target congestion indicates that the particle has fewer neighbors, meaning it is more sparse in the target space and can retain more "exploratory" solutions. Selecting particles in descending order of target congestion selects particles with high target congestion, eliminating the need to limit particle evolution to a small region and improving the exploratory capabilities of particle evolution.

[0145] In this optional solution, if there are multiple target dimensions, the particles in the Lth non-dominated layer can be sorted in each target dimension. For example, if the target dimensions include the first dimension and the second dimension, the particles in the Lth non-dominated layer are sorted according to their target values in the first dimension to obtain the sorting order of the particles in the Lth non-dominated layer in the first dimension. Furthermore, the particles in the Lth non-dominated layer are sorted according to their target values in the second dimension to obtain the sorting order of the particles in the Lth non-dominated layer in the second dimension.

[0146] In this optional solution, in the process of determining the target crowding of particles in the Lth non-dominated layer according to the sorting order of the particles in the Lth non-dominated layer in the target dimension, first, edge particles and intermediate particles can be determined according to the sorting order of the particles in the Lth non-dominated layer in the target dimension. Edge particles refer to particles at both ends of the sorting order, and intermediate particles refer to particles in the middle of the sorting order. The target crowding of the edge particles is determined to be a set value, which can be infinite. The target crowding of the intermediate particles can be determined based on the target value corresponding to the edge particles in the target dimension and the target value of the intermediate particle's neighbor particles in the target dimension. The neighbor particles of the intermediate particle refer to particles adjacent to the intermediate particle in the sorting order in the target dimension.

[0147] When the target dimension includes a first dimension and a second dimension, determining edge particles and intermediate particles according to the sorting order of particles in the Lth non-dominated layer in the target dimension may include: determining particles sorted at both ends of the Lth non-dominated layer in the first dimension as edge particles; determining particles sorted at both ends of the Lth non-dominated layer in the second dimension as edge particles; and determining the remaining particles in the Lth non-dominated layer, excluding the edge particles, as intermediate particles. This process may be referred to when the target dimension includes more dimensions.

[0148] Determining the target congestion of the intermediate particle based on the target value corresponding to the edge particle in the target dimension and the target value of the neighbor particle of the intermediate particle in the target dimension may include: determining the difference between the target values corresponding to the edge particles in the target dimension, determining the difference between the target values of the neighbor particles of the intermediate particle in the target dimension, and determining the target congestion of the intermediate particle based on the difference between the target values corresponding to the edge particles in the target dimension and the difference between the target values of the neighbor particles in the target dimension.

[0149] Optionally, the target congestion degree of the intermediate particles is calculated as:

[0150]

[0151] in, represents the target congestion of the ith intermediate particle, and represents the target value of the neighbor particle under the mth target dimension, and the neighbor particles include the particles in the Lth non-dominated layer that are adjacent to the i-th intermediate particle in the sorting order of the particles under the mth target dimension, and They represent the maximum and minimum target values of the particles in the Lth non-dominated layer in the mth target dimension, which can be obtained from the target values of the edge particles in the mth target dimension.

[0152] In an embodiment of the present application, an evolutionary process of a particle swarm is provided. During this evolutionary process, the evolutionary effect and efficiency of the particle swarm are improved from multiple aspects (such as the evolutionary strategy for the winner particles, the evolutionary strategy for the loser particles, and the environmental selection operation for the particles), thereby improving the effect and efficiency of optimizing the instance selection scheme for the website instance data and improving the quality of the target instance subset.

[0153] As an example, Figure 3 is an example diagram of the evolution process of particle swarm. Figure 3As shown in the figure, first, a particle swarm P is initialized. Next, P is evolved: spatial partitioning is performed within the particle swarm P (clustering and comparing the particles within the resulting clusters) to determine winner and loser particles. Winner particles are evolved using an evolutionary strategy based on a Gaussian model to obtain the corresponding evolved particles. Loser particles are evolved using an evolutionary strategy based on constraints to obtain the corresponding evolved particles. Then, all particles are subjected to environmental selection to select the particles that have evolved, resulting in the resulting particle swarm. The final condition for evolution (e.g., whether the number of evolutions is greater than or equal to a threshold) is determined. If so, evolution ends; otherwise, the next evolutionary step is performed. Finally, the final particle swarm P is output.

[0154] Corresponding to the website data processing method of the above embodiment, Figure 4 This is a structural block diagram of a website data processing device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 4 The website data processing device 400 includes: an acquisition unit 401, used to obtain website instance data, which includes multiple website instances; an evolution unit 402, used to evolve the initialized particle swarm through a competitive particle swarm algorithm and website instance data, and the particles in the particle swarm represent the instance selection scheme of the website instance data; a selection unit 403, used to select instances in the website instance data according to the evolved particle swarm to obtain a target instance subset, and the target instance subset is used for classification model training of phishing websites.

[0155] In some embodiments, during an evolution of a particle swarm, the evolution unit 402 is specifically configured to: select instances from website instance data according to an instance selection scheme represented by particles in the particle swarm to obtain an instance subset corresponding to the particles in the particle swarm; evaluate the instance subset corresponding to the particles in the particle swarm on a target dimension related to the instance selection to obtain a target value corresponding to the particles in the particle swarm on the target dimension; cluster the particles in the particle swarm according to the target values corresponding to the particles in the particle swarm on the target dimension to obtain a plurality of particle clusters; compare the particles in the plurality of particle clusters to determine winner particles in the plurality of particle clusters and loser particles in the plurality of particle clusters; evolve the winner particles in the plurality of particle clusters according to the evolution strategy corresponding to the winner particles to obtain evolved particles corresponding to the winner particles in the plurality of particle clusters; evolve the loser particles in the plurality of particle clusters according to the evolution strategy corresponding to the loser particles to obtain evolved particles corresponding to the loser particles in the plurality of particle clusters; and perform an environmental selection operation on the particles in the particle swarm, the evolved particles corresponding to the winner particles in the plurality of particle clusters, and the evolved particles corresponding to the loser particles in the plurality of particle clusters to determine the particles in the particle swarm after the current evolution.

[0156] In some embodiments, the target dimension includes a first dimension representing a phishing network detection error and / or a second dimension representing a subset complexity. The evolution unit 402 is specifically used to: use instance subsets corresponding to particles in the particle swarm as multiple independent training data sets to train the classification model respectively, and obtain model training errors of the instance subsets corresponding to the particles in the particle swarm; determine the target values corresponding to the particles in the particle swarm in the first dimension based on the model training errors of the instance subsets corresponding to the particles in the particle swarm; and / or determine the target values corresponding to the particles in the particle swarm in the second dimension based on complexity-related features of the instance subsets corresponding to the particles in the particle swarm.

[0157] In some embodiments, the evolution unit 402 is specifically used to: determine the angles between the particles in the particle swarm and multiple reference vectors according to the target values corresponding to the particles in the particle swarm in the target dimension, and the multiple reference vectors are evenly distributed in the target space formed by the target dimension; cluster the multiple particles into multiple initial clusters according to the angles between the particles in the particle swarm and the multiple reference vectors, and the multiple initial clusters correspond one-to-one to the multiple reference vectors; according to the multiple initial clusters and the multiple reference vectors, adjust the particles in the initial clusters whose actual number of particles is greater than the cluster capacity and the particles in the initial clusters whose actual number of particles is less than the cluster capacity to obtain multiple particle clusters.

[0158] In some embodiments, the evolution unit 402 is specifically used to: for any particle cluster among multiple particle clusters, determine the target value corresponding to the ideal particle corresponding to any particle cluster in the target dimension based on the target value corresponding to the particles in any particle cluster in the target dimension; determine the Chebyshev value of the particles in any particle cluster based on the target value corresponding to the particles in any particle cluster in the target dimension, the target value corresponding to the ideal particle corresponding to any particle cluster in the target dimension and multiple reference vectors; and determine the winner particle in any particle cluster and the loser particle in any particle cluster by comparing the Chebyshev values of the particles in any particle cluster.

[0159] In some embodiments, the evolutionary strategy corresponding to the winner particle is an evolutionary strategy based on a Gaussian model, and the evolution unit 402 is specifically used to: construct a Gaussian model, wherein the mean matrix and the covariance matrix in the Gaussian model are determined according to the position matrix of the winner particles in multiple particle clusters; perform random sampling according to the Gaussian model to obtain an updated particle position matrix; and guide the evolution of the winner particles in multiple particle clusters according to the updated particle position matrix to obtain evolved particles corresponding to the winner particles in multiple particle clusters.

[0160] In some embodiments, the evolution strategy corresponding to the loser particles is an evolution strategy based on constraint conditions, where the constraint conditions include intra-domain learning conditions and inter-domain learning conditions. The evolution unit 402 is specifically configured to: when the current evolution satisfies the intra-domain learning conditions, for the same particle cluster of multiple particle clusters, guide the evolution of the loser particles in the same particle cluster according to the position matrix of the winner particles in the same particle cluster, and obtain evolved particles corresponding to the loser particles in the same particle cluster; when the current evolution satisfies the inter-domain learning conditions, guide the evolution of the loser particles in the same particle cluster according to the position matrix of the winner particles in the same particle cluster and the position matrix of the winner particles in the adjacent clusters of the same particle cluster, and obtain evolved particles corresponding to the loser particles in the same particle cluster; wherein the intra-domain learning condition includes that the conditional probability value in the current evolution is less than the probability threshold, and the inter-domain learning condition includes that the conditional probability value is greater than or equal to the probability threshold, and the conditional probability value is randomly generated in the current evolution.

[0161] In some embodiments, the evolution unit 402 is specifically used to: merge particles in a particle group, evolved particles corresponding to winner particles in multiple particle clusters, and evolved particles corresponding to loser particles in multiple particle clusters into a temporary group; determine the non-dominated relationship of particles in the temporary group in the temporary group according to the target value of the particles in the temporary group under the target dimension; stratify the particles in the temporary group according to the non-dominated relationship of the particles in the temporary group and determine the non-dominated layer in which the particles in the temporary group are located; select particles that have undergone the current evolution in the temporary group according to the non-dominated layer in which the particles in the temporary group are located and the target value of the particles in the temporary group under the target dimension.

[0162] The website data processing device provided in this embodiment can be used to execute the technical solution of the embodiment of the above-mentioned website data processing method. Its implementation principle and technical effects are similar and will not be described in detail here.

[0163] refer to Figure 5 , which shows a schematic structural diagram of an electronic device 500 suitable for implementing an embodiment of the present disclosure. The electronic device 500 may be a terminal device or a server. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0164] like Figure 5 As shown, electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage device 508 into random access memory (RAM) 503. RAM 503 also stores various programs and data required for the operation of electronic device 500. Processing device 501, ROM 502, and RAM 503 are interconnected via bus 504. An input / output (I / O) interface 505 is also connected to bus 504.

[0165] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Figure 5 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0166] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0167] It should be noted that the computer-readable medium described above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable storage media may include, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wire, optical cable, RF (radio frequency), or any suitable combination thereof.

[0168] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0169] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0170] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0171] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0172] The units described in the embodiments of the present disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the acquisition unit may be described as "a unit for acquiring a page image and page description text of a web page to be detected."

[0173] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0174] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0175] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the present disclosure is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the scope of the above disclosure. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0176] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0177] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A website data processing method, characterized in that: include: Acquire website instance data, wherein the website instance data includes multiple website instances; Evolving an initialized particle swarm using a competitive particle swarm algorithm and the website instance data, wherein particles in the particle swarm represent instance selection schemes for the website instance data; Selecting instances from the website instance data based on the evolved particle swarm to obtain a target instance subset, wherein the target instance subset is used for training a classification model for phishing websites; The particle swarm evolution process includes: According to the instance selection scheme represented by the particles in the particle swarm, instances are selected from the website instance data to obtain instance subsets corresponding to the particles in the particle swarm; the instance subsets corresponding to the particles in the particle swarm are evaluated on a target dimension related to instance selection to obtain target values corresponding to the particles in the particle swarm on the target dimension; the particles in the particle swarm are clustered based on the target values corresponding to the particles in the particle swarm on the target dimension to obtain a plurality of particle clusters; the particles in the plurality of particle clusters are compared to determine winner particles in the plurality of particle clusters and loser particles in the plurality of particle clusters; the winner particles in the plurality of particle clusters are evolved according to the evolution strategy corresponding to the winner particles to obtain evolved particles corresponding to the winner particles in the plurality of particle clusters; the loser particles in the plurality of particle clusters are evolved according to the evolution strategy corresponding to the loser particles to obtain evolved particles corresponding to the loser particles in the plurality of particle clusters; and an environmental selection operation is performed on the particles in the particle swarm, the evolved particles corresponding to the winner particles in the plurality of particle clusters, and the evolved particles corresponding to the loser particles in the plurality of particle clusters to determine particles in the particle swarm that have undergone a current round of evolution.

2. The website data processing method according to claim 1, characterized in that: The target dimension includes a first dimension representing a phishing network detection error and / or a second dimension representing a subset complexity. The instance subset corresponding to the particles in the particle swarm is evaluated on the target dimension related to instance selection to obtain target values corresponding to the particles in the particle swarm on the target dimension, including: The classification model is trained using instance subsets corresponding to the particles in the particle swarm as multiple independent training data sets, respectively, to obtain model training errors of the instance subsets corresponding to the particles in the particle swarm; and target values corresponding to the particles in the particle swarm in the first dimension are determined based on the model training errors of the instance subsets corresponding to the particles in the particle swarm; And / or, determining target values corresponding to the particles in the particle swarm in the second dimension based on complexity-related features of a subset of instances corresponding to the particles in the particle swarm.

3. The website data processing method according to claim 1, characterized in that: The clustering operation is performed on the particles in the particle swarm according to the target values corresponding to the particles in the particle swarm on the target dimension to obtain a plurality of particle clusters, including: Determining angles between particles in the particle swarm and a plurality of reference vectors according to target values corresponding to particles in the particle swarm in the target dimension, wherein the plurality of reference vectors are uniformly distributed in a target space formed by the target dimension; Clustering the particles into a plurality of initial clusters according to angles between the particles in the particle swarm and the plurality of reference vectors, wherein the plurality of initial clusters correspond one-to-one to the plurality of reference vectors; According to the multiple initial clusters and the multiple reference vectors, particles in the initial clusters whose actual particle number is greater than the cluster capacity and particles in the initial clusters whose actual particle number is less than the cluster capacity are adjusted to obtain the multiple particle clusters.

4. The website data processing method according to claim 3, characterized in that: The comparing the particles in the plurality of particle clusters respectively to determine winner particles in the plurality of particle clusters and loser particles in the plurality of particle clusters comprises: For any particle cluster among the plurality of particle clusters, determining a target value corresponding to an ideal particle corresponding to the any particle cluster on the target dimension according to target values corresponding to particles in the any particle cluster on the target dimension; determining the Chebyshev values of the particles in any particle cluster according to the target values corresponding to the particles in any particle cluster in the target dimension, the target values corresponding to the ideal particles corresponding to any particle cluster in the target dimension, and the multiple reference vectors; The winner particle in any particle cluster and the loser particle in any particle cluster are determined by comparing the Chebyshev values of the particles in any particle cluster.

5. The website data processing method according to any one of claims 1 to 4, characterized in that: The evolutionary strategy corresponding to the winner particle is an evolutionary strategy based on a Gaussian model. The evolutionary operation is performed on the winner particles in the multiple particle clusters according to the evolutionary strategy corresponding to the winner particle to obtain the evolved particles corresponding to the winner particles in the multiple particle clusters, including: Constructing the Gaussian model, wherein the mean matrix and the covariance matrix in the Gaussian model are determined according to the position matrix of the winner particles in the plurality of particle clusters; Perform random sampling according to the Gaussian model to obtain an updated particle position matrix; According to the updated particle position matrix, the evolution of the winner particles in the plurality of particle clusters is guided to obtain evolved particles corresponding to the winner particles in the plurality of particle clusters.

6. The website data processing method according to any one of claims 1 to 4, characterized in that: The evolutionary strategy corresponding to the loser particles is an evolutionary strategy based on constraint conditions, wherein the constraint conditions include intra-domain learning conditions and inter-domain learning conditions. The evolutionary strategy corresponding to the loser particles is used to perform evolutionary operations on the loser particles in the multiple particle clusters to obtain evolved particles corresponding to the loser particles in the multiple particle clusters, including: When the current evolution satisfies the intra-domain learning condition, for a same particle cluster of the multiple particle clusters, guiding the evolution of loser particles in the same particle cluster according to the position matrix of the winner particles in the same particle cluster, to obtain evolved particles corresponding to the loser particles in the same particle cluster; When the current evolution satisfies the inter-domain learning condition, for a same particle cluster among the multiple particle clusters, guiding the evolution of loser particles in the same particle cluster according to the position matrix of the winner particles in the same particle cluster and the position matrix of the winner particles in the adjacent clusters of the same particle cluster, to obtain evolved particles corresponding to the loser particles in the same particle cluster; Among them, the intra-domain learning condition includes that the conditional probability value in the current evolution is less than the probability threshold, and the inter-domain learning condition includes that the conditional probability value is greater than or equal to the probability threshold, and the conditional probability value is randomly generated in the current evolution.

7. The website data processing method according to any one of claims 1 to 4, characterized in that: The step of performing an environmental selection operation on the particles in the particle swarm, the evolved particles corresponding to the winner particles in the plurality of particle clusters, and the evolved particles corresponding to the loser particles in the plurality of particle clusters to determine the particles in the particle swarm that have undergone the current evolution includes: Merging the particles in the particle group, the evolved particles corresponding to the winner particles in the plurality of particle clusters, and the evolved particles corresponding to the loser particles in the plurality of particle clusters into a temporary group; determining, in the temporary group, a non-dominance relationship of particles in the temporary group according to target values of the particles in the temporary group under the target dimension; stratifying the particles in the temporary group according to the non-dominated relationship of the particles in the temporary group, and determining the non-dominated layer where the particles in the temporary group are located; The particles that have undergone the current evolution are selected from the temporary group according to the non-dominated layer where the particles in the temporary group are located and the target values of the particles in the temporary group under the target dimension.

8. A website data processing device, characterized in that: include: An acquiring unit, configured to acquire website instance data, wherein the website instance data includes a plurality of website instances; An evolution unit, configured to evolve an initialized particle swarm using a competitive particle swarm algorithm and the website instance data, wherein particles in the particle swarm represent instance selection schemes for the website instance data; a selection unit, configured to select instances from the website instance data based on the evolved particle swarm to obtain a target instance subset, wherein the target instance subset is used for training a classification model for phishing websites; When the evolution unit evolves the initialized particle swarm, an evolution process of the particle swarm includes: According to the instance selection scheme represented by the particles in the particle swarm, instances are selected from the website instance data to obtain instance subsets corresponding to the particles in the particle swarm; the instance subsets corresponding to the particles in the particle swarm are evaluated on a target dimension related to instance selection to obtain target values corresponding to the particles in the particle swarm on the target dimension; the particles in the particle swarm are clustered based on the target values corresponding to the particles in the particle swarm on the target dimension to obtain a plurality of particle clusters; the particles in the plurality of particle clusters are compared to determine winner particles in the plurality of particle clusters and loser particles in the plurality of particle clusters; the winner particles in the plurality of particle clusters are evolved according to the evolution strategy corresponding to the winner particles to obtain evolved particles corresponding to the winner particles in the plurality of particle clusters; the loser particles in the plurality of particle clusters are evolved according to the evolution strategy corresponding to the loser particles to obtain evolved particles corresponding to the loser particles in the plurality of particle clusters; and an environmental selection operation is performed on the particles in the particle swarm, the evolved particles corresponding to the winner particles in the plurality of particle clusters, and the evolved particles corresponding to the loser particles in the plurality of particle clusters to determine particles in the particle swarm that have undergone a current round of evolution.

9. An electronic device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the website data processing method according to any one of claims 1 to 7.

Images