A project data priority screening system and method based on big data

Abstract

The present invention discloses a system and method for project data priority screening based on big data, including obtaining project data for preprocessing, introducing partial correlation coefficients to perform initial screening on the project data, calculating deviation values ​​for the project data based on the average value of historical project data, clustering the deviation values ​​to obtain cluster contribution, weighted combination based on the deviation values ​​and the cluster contribution to obtain data priority coefficients, allocating the project data into a first data queue and a second data queue based on an initial threshold, using a strategy combining exchange movement and insertion movement to perform neighborhood search on the first data queue and the second data queue to obtain a solution set, obtaining an optimal solution from the solution set based on an evaluation function, and completing the screening of project data priorities. This method can not only improve the efficiency and quality of project data priority screening, but also has good interpretability.

Description

Technical Field

[0001] The present invention relates to the technical field of data screening, and in particular to a system and method for screening project data priority based on big data. Background Art

[0002] In the field of data screening technology, data priority is one of the key indicators of project success. Data priority can guide project resource allocation and decision-making processes and improve the efficiency of project problem solving. Since different data usually have unique statistical characteristics, data priority screening technology is an important method to identify and analyze key data patterns and evaluate the value of project data, which helps to improve project efficiency and return rate.

[0003] Traditional technical solutions often focus on using statistical analysis methods and subjective judgment to screen data priorities, fail to fully utilize the complex relationships between data for in-depth analysis, lack a comprehensive assessment of the importance of data, and do not consider the potential impact revealed by the partial correlation coefficient. In addition, traditional technologies may be inefficient when processing large-scale data sets and may not be accurate in resource allocation, resulting in a low return on project investment. The shortcomings of traditional technologies are that they may not be able to adaptively handle the complexity and dynamic changes of data in a big data environment, and may be inefficient and lack precision when processing large-scale data sets. Summary of the invention

[0004] The purpose of the present invention is to provide a project data priority screening system and method based on big data.

[0005] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0006] The present invention comprises the following steps:

[0007] A obtains project data for preprocessing, and introduces a partial correlation coefficient to perform a primary screening on the project data; wherein the method for performing the primary screening is specifically as follows:

[0008] Step 1 calculates the partial correlation coefficient between each data and the target variable. The calculation formula of the partial correlation coefficient is:

[0009]

[0010] Among them, n is the amount of project data, X i is the i-th item data, is the average value of the project data, Y i is the variable value of the i-th item data, is the average correlation coefficient of the variable values ​​of the project data;

[0011] Step 2 uses the partial correlation coefficient to construct a comprehensive scoring formula. The calculation formula is:

[0012]

[0013] Among them, Score is the comprehensive score X i is the value of the i-th data, ω i is the weight coefficient of the ith data, a, b, c are the parameters of the sigmoid function, a is a positive adjustment factor, b is the parameter that controls the slope of the sigmoid function curve, c is the offset, ω i+1 is log(1+X i ) is the weight coefficient of the term, θ is the part that introduces nonlinear characteristics, ω i+2 for The weight coefficient, ω i+3 for The weight coefficient of the sine function term, ω is the angular frequency of the sine function, is the phase offset of the sine function term, n is the amount of project data, r i is the partial correlation coefficient;

[0014] Step 3: Based on the comprehensive score, the data is initially screened. When the comprehensive score is less than 0.4, the data is removed from the project data.

[0015] B. calculating a deviation value for the project data based on an average value of the historical project data, clustering the deviation values ​​to obtain a cluster contribution, and performing a weighted combination based on the deviation value and the cluster contribution to obtain a data priority coefficient;

[0016] C allocates the project data into a first data queue and a second data queue based on an initially set threshold; the data in the first data queue and the second data queue are sorted from large to small according to corresponding data priority coefficients;

[0017] D uses a strategy combining exchange movement and insertion movement to perform neighborhood search on the first data queue and the second data queue to obtain a solution set, obtains an optimal solution from the solution set based on an evaluation function, and completes the screening of the project data priority; wherein the evaluation function is specifically:

[0018] The calculation formula of the evaluation function is:

[0019]

[0020] Among them, d norm (i) is the deviation value of the i-th data, w i is the clustering contribution of the i-th data, ClusterQuality(C) is the quality metric of the cluster, and the calculation formula of the quality metric of the cluster is:

[0021]

[0022] Among them, a(i) is the average distance from the i-th data to all other points in the same cluster, and b(i) is the average distance from the data to all points in the nearest cluster;

[0023] The queue with the largest evaluation function value is selected to obtain the optimal solution, and the order in which the data is arranged in the optimal solution queue is used as the final project data priority.

[0024] As a further method, the preprocessing method described in step A includes: missing value processing, outlier processing, standardization processing, normalization processing, forward processing, and removal of duplicate data.

[0025] As a further method, the method of clustering the deviation values ​​to obtain cluster contribution includes:

[0026] Step 1: Set the clustering iteration standard ε = 10 and set the initial cluster center;

[0027] Step 2 calculates the membership matrix and cluster center of each iteration;

[0028] Step 3 calculates the distance between the two iterative cluster centers, and stops the iteration if it is less than the iteration standard;

[0029] Step 4 divides the data into several clusters through the membership matrix and cluster center matrix obtained through multiple iterations;

[0030] Step 5 calculates the distance from the data on each cluster to the cluster center to obtain the cluster contribution.

[0031] As a further method, the method of weighted combination based on deviation value and cluster contribution degree includes:

[0032] The priority coefficient is calculated by taking the comprehensive score of the data as the weighted coefficient:

[0033] Score i =α×d norm (i)+(1-α)×w i

[0034] Among them, α is the weighting coefficient.

[0035] As a further method, the method of initially setting the threshold allocation includes: dividing based on data weight value, storing data with weight value greater than or equal to 0.1 as first data in a first data queue, and storing data with weight value less than 0.1 as second data in a second data queue.

[0036] As a further method, the strategy of combining exchange movement and insertion movement includes: exchanging the positions of data in the first data queue and the second data queue and moving the data in one data queue to another data queue, generating multiple queues with different distributions to form a solution set.

[0037] The second aspect of the present invention also provides a system for prioritizing project data based on big data, comprising: a data extraction module, for acquiring project data and performing preprocessing;

[0038] A preliminary screening module, used for performing preliminary screening of the project data through partial correlation coefficients;

[0039] A priority acquisition module, used to calculate the deviation value and the cluster contribution, and obtain the data priority based on the weighted combination of the deviation value and the cluster contribution;

[0040] A data queue division module allocates project data into a first data queue and a second data queue according to an initial threshold value based on the order of data priority;

[0041] The optimal solution acquisition module adopts a strategy combining exchange movement and insertion movement to perform neighborhood search on the first data queue and the second data queue to obtain a solution set, obtains the optimal solution based on the evaluation function, and completes the screening of project data priorities.

[0042] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0043] (1) The present invention provides a project data priority screening system and method based on big data, which realizes project data priority sorting through partial correlation screening, deviation value clustering, priority sorting and neighborhood search strategy, improves data priority quality and improves project efficiency.

[0044] (2) The present invention calculates the deviation value of the project data based on the average value of the historical project data, clusters the deviation value to obtain the cluster contribution, and then obtains a more accurate cluster contribution, providing more accurate data for subsequent operations.

[0045] (3) The present invention adopts a strategy combining exchange movement and insertion movement to perform neighborhood search, which can explore the solution space more comprehensively, avoid falling into local optimality, obtain better solutions, and improve the quality of project data priority screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The present invention is a flowchart of the steps of a project data priority screening method based on big data.

[0047] Figure 2It is a schematic diagram of the structure of an electronic device in an embodiment of this specification. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0049] Reference Figure 1 As shown, the first aspect of the present invention provides a method for prioritizing project data based on big data, comprising:

[0050] A obtains project data for preprocessing, and introduces a partial correlation coefficient to perform a primary screening on the project data; wherein the method for performing the primary screening is specifically as follows:

[0051] Step 1 calculates the partial correlation coefficient between each data and the target variable. The calculation formula of the partial correlation coefficient is:

[0052]

[0053] Among them, n is the amount of project data, X i is the i-th item data, is the average value of the project data, Y i is the variable value of the i-th item data, is the average correlation coefficient of the variable values ​​of the project data;

[0054] Step 2 uses the partial correlation coefficient to construct a comprehensive scoring formula. The calculation formula is:

[0055]

[0056] Among them, Score is the comprehensive score X i is the value of the i-th data, ω i is the weight coefficient of the ith data, a, b, c are the parameters of the sigmoid function, a is a positive adjustment factor, b is the parameter that controls the slope of the sigmoid function curve, c is the offset, ω i+1 is log(1+X i ) is the weight coefficient of the term, θ is the part that introduces nonlinear characteristics, ω i+2 for The weight coefficient, ω i+3 for The weight coefficient of the sine function term, ω is the angular frequency of the sine function, is the phase offset of the sine function term, n is the amount of project data, ri is the partial correlation coefficient;

[0057] Step 3: Based on the comprehensive score, the data is initially screened. When the comprehensive score is less than 0.4, the data is removed from the project data.

[0058] In the actual assessment, 11 types of data related to the project are obtained, including:

[0059] Data 1, the total investment amount approved for the project, 5 million US dollars;

[0060] Data 2, the number of workers working on the project site, 120;

[0061] Data 3, total amount of materials in the project, 2,000 tons;

[0062] Data 4, the number of equipment put into project construction, 35;

[0063] Data 5, total area covered by the project, 50,000 square meters;

[0064] Data 6, the highest construction height of the building, 100 meters;

[0065] Data 7, construction noise level, 65 decibels;

[0066] Data 8, average daily wind speed at the construction site, 5 km / h;

[0067] Data 9, number of plant species in the construction area, 10 species;

[0068] Data 10, average daily temperature at the construction site, 22°C;

[0069] Data 11, average daily humidity at the construction site, 45%;

[0070] In the actual evaluation, the project data were processed for missing values, outliers, standardization, normalization, and forward processing, and duplicate data were removed. The partial correlation coefficient of the data was calculated for the initial screening of the project data, and finally data 8, data 9, and data 11 were eliminated.

[0071] B. calculating a deviation value for any two data of the project data, clustering the deviation values ​​to obtain a cluster contribution, and performing a weighted combination based on the deviation value and the cluster contribution to obtain a data priority coefficient;

[0072] In the actual evaluation, the deviation values ​​are calculated for data 1, data 2, data 3, data 4, data 5, data 6, data 7, and data 10, and the deviation values ​​are divided into different clusters. The cluster contribution is calculated for the data in each cluster. The data priority is obtained based on the weighted combination of the deviation value and the cluster contribution. The priority coefficients of data 1, data 2, data 3, data 4, data 5, data 6, data 7, and data 10 are 2.0, 1.4, 1.0, 1.2, 1.7, 1.6, 0.8, and 0.7;

[0073] C allocates the project data into a first data queue and a second data queue based on an initially set threshold; the data in the first data queue and the second data queue are sorted from large to small according to corresponding data priority coefficients;

[0074] In the actual evaluation, the project data is allocated to the first data queue and the second data queue based on the initial threshold, and the data in the first data queue and the second data queue are sorted from large to small according to the corresponding data priority coefficients. Further, the first data queue contains data 1, data 6, data 5, and data 4, and the second data queue contains data 2, data 3, data 7, and data 10.

[0075] D uses a strategy combining exchange movement and insertion movement to perform neighborhood search on the first data queue and the second data queue to obtain a solution set, obtains an optimal solution from the solution set based on an evaluation function, and completes the screening of the project data priority; wherein the evaluation function is specifically:

[0076] The calculation formula of the evaluation function is:

[0077]

[0078] Among them, d norm (i) is the deviation value of the i-th data, w i is the clustering contribution of the i-th data, ClusterQuality(C) is the quality metric of the cluster, and the calculation formula of the quality metric of the cluster is:

[0079]

[0080] Among them, a(i) is the average distance from the i-th data to all other points in the same cluster, and b(i) is the average distance from the data to all points in the nearest cluster;

[0081] The queue with the largest evaluation function value is selected to obtain the optimal solution, and the order in which the data is arranged in the optimal solution queue is used as the final project data priority.

[0082] In the actual evaluation, a solution set is obtained after performing neighborhood search on the first data queue and the second data queue. Based on the evaluation function, all queues in the solution set are evaluated, and the queue with the largest evaluation function value is calculated to be the optimal solution. The data in the optimal solution queue are sorted as data 1, data 4, data 5, data 2, and data 3. The data priority of the engineering project is further obtained as the total investment amount approved for the project, the number of equipment invested in the project construction, the total area covered by the project, the number of workers working on the project site, and the total amount of materials in the project.

[0083] In this embodiment, the preprocessing method described in step A includes: missing value processing, outlier processing, standardization processing, normalization processing, forward processing, and removal of duplicate data.

[0084] In this embodiment, the method of clustering the deviation values ​​to obtain the cluster contribution includes:

[0085] Step 1: Set the clustering iteration standard ε = 10 and set the initial cluster center;

[0086] Step 2 calculates the membership matrix and cluster center of each iteration;

[0087] Step 3 calculates the distance between the two iterative cluster centers, and stops the iteration if it is less than the iteration standard;

[0088] Step 4 divides the data into several clusters through the membership matrix and cluster center matrix obtained through multiple iterations;

[0089] Step 5 calculates the distance from the data on each cluster to the cluster center to obtain the cluster contribution.

[0090] In this embodiment, the method for weighted combination based on the deviation value and the cluster contribution includes:

[0091] The priority coefficient is calculated by taking the comprehensive score of the data as the weighted coefficient:

[0092] Score i =α×d norm (i)+(1-α)×w i

[0093] Among them, α is the weighting coefficient.

[0094] In this embodiment, the method of initially setting threshold allocation includes: dividing based on data weight value, storing data with weight value greater than or equal to 0.1 as first data in a first data queue, and storing data with weight value less than 0.1 as second data in a second data queue.

[0095] In this embodiment, the strategy of combining exchange movement and insertion movement includes: exchanging the positions of data in the first data queue and the second data queue and moving the data in one data queue to another data queue, generating multiple queues with different distributions to form a solution set.

[0096] The second aspect of the present invention also provides a system for prioritizing project data based on big data, comprising: a data extraction module for acquiring project data and performing preprocessing;

[0097] The initial screening module is used to perform initial screening of project data through partial correlation coefficients;

[0098] The priority coefficient acquisition module is used to calculate the deviation value and the cluster contribution, and obtain the data priority coefficient based on the weighted combination of the deviation value and the cluster contribution;

[0099] A data queue division module allocates the project data into a first data queue and a second data queue according to an initial threshold value based on the order of data priority coefficients;

[0100] The optimal solution acquisition module adopts a strategy combining exchange movement and insertion movement to perform neighborhood search on the first data queue and the second data queue to obtain a solution set, obtains the optimal solution from the solution set based on the evaluation function, and completes the screening of project data priority.

[0101] Figure 2 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 2 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0102] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0103] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0104] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an information push system for intelligent manufacturing services at the logical level. The processor executes the program stored in the memory and is specifically used to execute any of the aforementioned information push methods for intelligent manufacturing services.

[0105] The above application Figure 1 The project data priority screening method based on big data disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0106] The electronic device may also perform Figure 1 A project data priority screening method based on big data is implemented Figure 1 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0107] An embodiment of the present application also proposes a computer-readable storage medium, which stores one or more programs, and the one or more programs include instructions, which, when executed by an electronic device including multiple applications, execute any of the aforementioned big data-based project data priority screening systems.

[0108] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0109] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.

[0110] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0112] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0113] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0114] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0115] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0116] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0117] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A project data priority screening method based on big data, characterized in that: The following steps are involved: A obtains project data for preprocessing, and introduces partial correlation coefficients to perform initial screening on the project data; wherein the method for performing initial screening is specifically as follows: Step 1 calculates the partial correlation coefficient between each data and the target variable. The calculation formula of the partial correlation coefficient is: Among them, n is the amount of project data, X i is the i-th item data, is the average value of the project data, Y i is the variable value of the i-th item data, is the average correlation coefficient of the variable values ​​of the project data; Step 2 uses the partial correlation coefficient to construct a comprehensive scoring formula. The calculation formula is: Among them, Score is the comprehensive score, X i is the value of the i-th data, ω i is the weight coefficient of the ith data, a, b, c are the parameters of the sigmoid function, a is a positive adjustment factor, b is the parameter that controls the slope of the sigmoid function curve, c is the offset, ω i+1 is log(1+X i ) is the weight coefficient of the term, θ is the part that introduces nonlinear characteristics, ω i+2 for The weight coefficient, ω i+3 for The weight coefficient of the sine function term, ω is the angular frequency of the sine function, is the phase offset of the sine function term, n is the amount of project data, r i is the partial correlation coefficient; Step 3: Based on the comprehensive score, the data is initially screened. When the comprehensive score is less than 0.4, the data is removed from the project data. B. calculating a deviation value for the project data based on an average value of the historical project data, clustering the deviation values ​​to obtain a cluster contribution, and performing a weighted combination based on the deviation value and the cluster contribution to obtain a data priority coefficient; C allocates the project data into a first data queue and a second data queue based on an initially set threshold; the data in the first data queue and the second data queue are sorted from large to small according to corresponding data priority coefficients; D uses a strategy combining exchange movement and insertion movement to perform neighborhood search on the first data queue and the second data queue to obtain a solution set, obtains an optimal solution from the solution set based on an evaluation function, and completes the screening of the project data priority; wherein the evaluation function is specifically: The calculation formula of the evaluation function is: Among them, d norm (i) is the deviation value of the i-th data, w i is the clustering contribution of the i-th data, ClusterQuality(C) is the quality metric of the cluster, and the calculation formula of the quality metric of the cluster is: Among them, a(i) is the average distance from the i-th data to all other points in the same cluster, and b(i) is the average distance from the data to all points in the nearest cluster; The queue with the largest evaluation function value is selected to obtain the optimal solution, and the order in which the data is arranged in the optimal solution queue is used as the final project data priority.

2. According to the method for prioritizing project data based on big data in claim 1, it is characterized in that: The preprocessing method described in step A includes: missing value processing, outlier processing, standardization processing, normalization processing, forward processing, and removal of duplicate data.

3. The project data priority screening method based on big data according to claim 1 is characterized in that: The method of clustering the deviation values ​​to obtain cluster contribution comprises: Step 1: Set the clustering iteration standard ε = 10 and set the initial cluster center; Step 2 calculates the membership matrix and cluster center of each iteration; Step 3 calculates the distance between the two iterative cluster centers, and stops the iteration if it is less than the iteration standard; Step 4 divides the data into several clusters through the membership matrix and cluster center matrix obtained through multiple iterations; Step 5 calculates the distance from the data on each cluster to the cluster center to obtain the cluster contribution.

4. The project data priority screening method based on big data according to claim 1 is characterized in that: The method for weighted combination based on deviation value and cluster contribution degree includes: The priority coefficient is calculated by taking the comprehensive score of the data as the weighted coefficient: Score i =α×d norm (i)+(1-α)×w i Among them, α is the weighting coefficient.

5. The project data priority screening method based on big data according to claim 1 is characterized in that: The method for initially setting threshold allocation includes: dividing based on data weight values, storing data with weight values ​​greater than or equal to 0.1 as first data in a first data queue, and storing data with weight values ​​less than 0.1 as second data in a second data queue.

6. The project data priority screening method based on big data according to claim 1 is characterized in that: The strategy of combining exchange movement and insertion movement includes: exchanging the positions of data in the first data queue and the second data queue and moving the data in one data queue to the other data queue, generating multiple queues with different distributions to form a solution set.

7. A project data priority screening system based on big data, used to execute a project data priority screening method based on big data as claimed in any one of claims 1 to 6, characterized in that: The system comprises: Data extraction module, used to obtain project data and perform preprocessing; The initial screening module is used to perform initial screening of project data through partial correlation coefficients; The priority coefficient acquisition module is used to calculate the deviation value and the cluster contribution, and obtain the data priority coefficient based on the weighted combination of the deviation value and the cluster contribution; A data queue division module allocates the project data into a first data queue and a second data queue according to an initial threshold value based on the order of data priority coefficients; The optimal solution acquisition module adopts a strategy combining exchange movement and insertion movement to perform neighborhood search on the first data queue and the second data queue to obtain a solution set, obtains the optimal solution from the solution set based on the evaluation function, and completes the screening of project data priority.

Images