An enterprise capability evaluation method based on neural cognitive diagnosis technology

Abstract

The application discloses an enterprise capability evaluation method based on neural cognitive diagnosis technology, comprising the following steps: 1, dividing the technology achievement data into fields, and constructing negative samples of interactive data; 2, constructing a basic neural cognitive diagnosis enterprise capability evaluation learning module, obtaining embedded vector representations such as field mastery, technology achievement difficulty and discrimination, and performing representation learning on the target embedded vector; meanwhile, the embedded vector is interacted, and self-adaptive learning is performed after two full connection layers; 3, constructing a partial order sorting learning module, and constructing a BPR loss after a partial order learning module, an expected capability change module and an enterprise-technology achievement interactive result constraint design module; 4, using cross-entropy loss and contrast loss to optimize the self-adaptive diagnosis network of the whole model. The application utilizes enterprise-technology achievement interactive data to perform self-adaptive diagnosis on technology enterprise capability evaluation, thereby providing an effective method for enterprise capability evaluation in different fields.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of cognitive diagnosis technology in machine learning, artificial intelligence and smart education, and also relates to the field of data processing of scientific and technological achievements and evaluation of scientific research capabilities of different enterprises, in particular to an enterprise capability evaluation method based on neural cognitive diagnosis technology. BACKGROUND

[0002] Cognitive diagnosis (CD) is considered as one of the key tasks of intelligent education applications, aiming to diagnose the ability state of each examinee according to specific evaluation data, which has attracted more attention in many real-world scenarios such as education and games. In particular, cognitive diagnosis in the field of smart education is to find the mastery level of students on different skill / knowledge concepts, which is similar to our task goal. Existing cognitive diagnosis methods can be roughly divided into two categories: statistical-based methods and deep neural network-based methods. The former based on item response theory (IRT), multidimensional IRT (MIRT) and (DINA) method are all designed by hand to simulate the interaction between students and exercises. The latter based on neural networks to model the complex interaction pattern between students and exercises, while trying to incorporate richer context features and prior relationships between knowledge points to enhance the representation learning of students and exercises.

[0003] Early research on the evaluation of scientific and technological capabilities mainly studies the development status of high-end scientific and technological talents in China through qualitative analysis, and studies academic stars through quantitative indicators. Early evaluation of the capabilities of listed companies also adopts quantitative indicators and qualitative analysis, so as to measure the overall capabilities of a listed company by combining the two indicators; the key problems in the evaluation of enterprise-level scientific and technological achievements are as follows: previous methods only consider the capabilities of enterprises from the qualitative and quantitative perspectives, and cannot measure the capabilities of enterprises in a certain field in a fine-grained manner from the perspective of enterprises; different achievements made by enterprises usually belong to different specific research fields, and the completion difficulty and level of each field are not the same. Different achievements correspond to different types of achievements, and each achievement has a corresponding difficulty coefficient. Since different enterprises cannot well quantify the scientific research capabilities they show under these fine-grained influencing factors, there is an urgent need for a supervised method that combines these influencing factors to solve this problem. At the same time, a large amount of enterprise scientific and technological achievement interaction data has a long tail problem: a small number of enterprises interact with a large number of scientific and technological achievements, while a large number of enterprises interact with very few scientific and technological achievements. The long tail means that the recommendation system will tend to recommend popular projects, which is not conducive to overall project recommendation. This data distribution will cause the existing neural network-based recommendation model to be unable to well diagnose the capabilities of all enterprises, thereby affecting the overall prediction effect. Therefore, a method is needed to alleviate the long tail problem to deal with this data distribution phenomenon in scientific and technological achievements. SUMMARY

[0004] This invention aims to address the shortcomings of existing technologies in the assessment of technology enterprise capabilities by proposing a method for enterprise capability assessment based on neurocognitive diagnostic technology. This method is designed to provide adaptive diagnosis for technology enterprise capability assessment, thereby achieving a more effective enterprise capability assessment.

[0005] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0006] The present invention provides a method for assessing enterprise capabilities based on neurocognitive diagnostic technology, characterized by the following steps:

[0007] Step 1: Define enterprise technology data and use the FastText model to divide the enterprise technology data into domains, obtaining a technology achievement domain matrix. And construct the total interaction set of enterprise-technology achievements after negative samples. ;

[0008] Step 2: Construct a basic neurocognitive diagnostic enterprise capability assessment learning module, including: an initial embedding representation layer, an enterprise capability diagnostic layer, and a prediction layer, and sequentially apply this to the technology achievement domain matrix. The first in Scientific and technological achievements Belonging to the p-th domain Matrix of scientific and technological achievements Processing is performed to obtain the i-th company. For the p-th domain The scientific and technological achievements Predictive interaction results This leads to the construction of a basic neurocognitive diagnostic enterprise capability assessment network to assess losses. ;

[0009] Step 3: Construct a partial order ranking learning module, including: a partial order learning module, an expected capability change module, and a constraint design module for the interaction results between enterprises and scientific and technological achievements;

[0010] Step 3.1: Construct a partial-order learning module and use a partial-order response sampling method to process the total interaction set of enterprise-technology achievements after negative samples. Perform partial order sampling to obtain the i-th company. Reliable candidate interaction set ;

[0011] Step 3.2: Construct a module for predicting changes in capabilities, used to calculate reliable candidate interaction sets. The i-th company For the u Uninteractive technological achievements expected capability change of the interaction triplets , and after normalization, the expected capability change of the i-th enterprise to the m-th item of scientific and technological achievements u

[0012] Step 3.3, design the enterprise-scientific and technological achievements interaction result constraint, and construct the i-th enterprise BPR loss function of the i-th enterprise to all scientific and technological achievements

[0013] Step 4, construct the total loss function of the i-th enterprise to all scientific and technological achievements (19)

[0014] In formula (19), is a hyperparameter for balancing the two losses;

[0015] Step 5, train the neural cognitive diagnosis enterprise capability evaluation network composed of the basic neural cognitive diagnosis enterprise capability evaluation learning module and the partial order sorting learning module using the Adam optimizer, and minimize until converges, thereby obtaining the optimal neural cognitive diagnosis enterprise capability evaluation network for diagnosing the mastery of enterprises in different fields, thereby obtaining the capability evaluation results of enterprises in different fields.

[0016] The enterprise capability evaluation method based on neural cognitive diagnosis technology has the characteristics that the step 1 is performed according to the following steps:

[0017] Step 1.1, define the number of enterprises in the scientific and technological data as , the number of scientific and technological achievements as , and the number of fields as , let the i-th enterprise, the m-th item of scientific and technological achievements, and the p-th field be represented as , and , , , ;

[0018] Let the interaction result of the i-th enterprise to the m-th item of scientific and technological achievements be represented as , and , represent the total enterprise-scientific and technological achievements interaction set,​​​​​​ ;

[0019] Step 1.2: Use the FastText classification model to classify... By dividing the data into domains, a matrix of scientific and technological achievements can be obtained. , ,make The mth scientific and technological achievement in China Belonging to the p-th domain The matrix of scientific and technological achievements is denoted as ;

[0020] Step 1.3: Based on the total interaction set of enterprise-technology achievements Construct negative sample data for all enterprises;

[0021] Define the level of scientific and technological achievements as k. C indicates the highest level;

[0022] Search for the i-th company in descending order of its scientific and technological achievement level k. For all scientific and technological achievements, if there is a corresponding scientific and technological achievement at the current level, negative samples are not constructed; otherwise, in Randomly selected from each field For each non-interactive domain, negative samples are constructed to obtain the i-th enterprise. Constructed One negative sample, and ;in, To reverse the scientific and technological achievement ranking of the m-th scientific and technological achievement, For the i-th company The total number of all scientific and technological achievements made To control the threshold for the number of negative samples for each company, To control the threshold for the total number of negative samples for each company;

[0023] Let the total number of scientific and technological achievements after constructing negative samples be , order the The scientific and technological achievements are recorded as , The total interaction set of enterprises and scientific and technological achievements after constructing negative samples is: , Enterprise-Technology Achievement Interaction Hub The i-th company For the scientific and technological achievements The interactive triplet is denoted as ,in, Represents the i-th company For the scientific and technological achievements The interaction result, if , representing the i-th company For the actual completion of the first scientific and technological achievements ,like , representing the i-th company For the unfinished scientific and technological achievements Enterprise-Technology Achievement Interaction Hub The i-th company The interaction set of all scientific and technological achievements is denoted as ;

[0024] make The Middle scientific and technological achievements Belonging to the p-th domain The matrix of scientific and technological achievements is denoted as .

[0025] Furthermore, step 2 is performed according to the following steps:

[0026] Step 2.1: The initial embedded representation layer obtains the i-th enterprise using equations (1) and (2) respectively. Enterprise capability mastery vector and the scientific and technological achievements Belonging to the p-th domain The matrix vector of the results domain , , ;

[0027] (1)

[0028] (2)

[0029] In equations (1) and (2), Represents the i-th company One-hot encoded vector, , Indicates the first scientific and technological achievements One-hot encoded vector, , Represents the matrix to be trained. ;

[0030] Step 2.2, the initial embedded representation layer is constructed. scientific and technological achievements Technological metrics , The vector of difficulty in completing scientific and technological achievements , and the scientific and technological achievements Discrimination vector , ;

[0031] Step 2.3: The enterprise capability diagnosis layer includes an interaction layer and two fully connected layers, and sequentially performs... , as well as Processing is performed to obtain the i-th company. The p-th domain completed The scientific and technological achievements Feature representation and dimensionality reduction feature representation ;

[0032] Step 2.4: The prediction layer uses equation (9) to obtain the i-th enterprise. For the p-th domain The scientific and technological achievements Predictive interaction results :

[0033] (9)

[0034] In equation (9), The weight matrix to be trained for the prediction layer. The bias vector of the prediction layer;

[0035] Step 2.5: Use formula (10) to calculate the i-th company. Loss of all scientific and technological achievements : (10)

[0036] In equation (10), Represents the i-th company For the The completion status of the scientific and technological achievements. Represents the i-th company For the p-th domain The scientific and technological achievements The prediction results.

[0037] Furthermore, step 2.2 is performed according to the following steps:

[0038] Step 2.2.1, place the first scientific and technological achievements The level corresponding to the scientific and technological achievement is denoted as: , the first scientific achievement is corresponding to the number , so as to calculate the scientific measurement value of the first scientific achievement by using formula (3):

[0039] (3)

[0040] In formula (3), min-max represents a normalization process;

[0041] Step 2.2.2, the completion difficulty vector of the first scientific achievement is calculated by using formula (4) and formula (5), and the discrimination vector of the first scientific achievement is calculated by using formula (4) and formula (5), :

[0042] (4)

[0043] (5)

[0044] In formula (4) and formula (5), represents the one-hot encoding vector of the first scientific achievement , ,respectively represents two matrices to be trained, ,. Further, the step 2.3 is performed according to the following steps:

[0045] Step 2.3.1, the interaction layer establishes the interaction function of the first scientific achievement

[0046] of the first enterprise i in the pth field by using formula (6): (6)

[0047] In formula (6), is an element product;

[0048] Step 2.3.2, the full connection layer obtains the first scientific achievement

[0049] of the first enterprise i in the pth field by using formula (7) and formula (8):​​​​ the pth field accomplished by the i-th enterprise the pth field accomplished by the i-th enterprise the pth scientific achievement the feature representation of the pth scientific achievement and the reduced dimension feature representation :

[0050] (7)

[0051] (8)

[0052] In formula (7) - formula (8), represents the i-th enterprise the pth field accomplished by the i-th enterprise the pth field accomplished by the i-th enterprise the pth scientific achievement the feature representation of the pth scientific achievement the pth scientific achievement the pth scientific achievement the pth scientific achievement the pth scientific achievement the feature representation of the pth scientific achievement sigmoid activation function, , respectively, are the weight matrices to be trained of two fully connected layers, , respectively, are the bias vectors of two fully connected layers.

[0053] Further, the step 3.1 is performed according to the following steps:

[0054] Step 3.1.1, obtaining the interaction triple of the i-th enterprise ,to the pth scientific achievement from the enterprise-scientific achievement interaction set after constructing negative samples, and using formula (11) to extract the set of achievements similar to the pth scientific achievement

[0055] (11)

[0056] In formula (11), is the set of scientific achievements that the enterprise does not interact with, , is the pth non-interaction scientific achievement, u is the pth non-interaction scientific achievement is the pth non-interaction scientific achievement u is the pth non-interaction scientific achievement ​​​​​​the relevant domain set, the i-th enterprise to the i-th non-interacted technology achievement the relevant domain set;

[0057] Step 3.1.2, constructing the i-th enterprise to the i-th non-interacted technology achievement u the interaction triplet :

[0058] (12)

[0059] in the formula (12), the i-th enterprise to the i-th non-interacted technology achievement u the interaction result; Step 3.1.3, obtaining the reliable candidate interaction set

[0060] of the i-th enterprise using the formula (14) :

[0061] (14)

[0062] in the formula (14), the number of non-interacted samples in the i-th enterprise , and the interaction triplet of the i-th enterprise to the i-th non-interacted technology achievement u . Further, the step 3.2 is performed according to the following steps:

[0063] Step 3.2.1, obtaining the expected capability change

[0064] of the i-th enterprise on the i-th non-interacted technology achievement using the formula (15) :

[0065] (15)

[0066] in the formula (15), the expected capability change of the i-th enterprise on the i-th non-interacted technology achievement uses a diagnostic interaction function, the mastery degree of the i-th enterprise on each domain, ​​​​represents the i-th enterprise In addition, the u-th non-interacted technology achievement After the degree of mastery in each field;

[0067] Step 3.2.2, calculate the normalized expected ability change weight of the i-th enterprise u non-interacted technology achievement using formula (16) :

[0068] (16)

[0069] In formula (16), represents the number of technology achievements in the reliable candidate interaction set .

[0070] Further, the step 3.3 is performed according to the following steps:

[0071] Step 3.3.1, for the i-th enterprise , according to the different interaction results, the interaction set after constructing negative samples is divided into the positive interaction result set after constructing negative samples and the negative interaction result set after constructing negative samples ;

[0072] The reliable candidate interaction set is divided into the positive interaction result set in the reliable candidate set and the negative interaction result set in the reliable candidate set ;

[0073] From , , , , any positive interaction technology achievement , any candidate positive interaction technology achievement , any negative interaction technology achievement , any candidate negative interaction technology achievement , so as to construct the enterprise-technology achievement interaction result constraint using formula (17):

[0074] (17)

[0075] In formula (17), represents the interaction result of the i-th enterprise to the positive interaction technology achievement , represents the interaction result of the i-th enterprise to the negative interaction technology achievement interaction results of the i-th enterprise, representing the i-th enterprise to interaction results of the i-th enterprise, representing the i-th enterprise to interaction results of the i-th enterprise;

[0076] Step 3.3.2, constructing the i-th enterprise BPR loss of all scientific and technological achievements : (18)。

[0077] The electronic device comprises a memory and a processor, and the memory is used for storing a program supporting the processor to execute the enterprise capability evaluation method, and the processor is configured to execute the program stored in the memory.

[0078] The computer readable storage medium stores a computer program, and when the computer program is run by a processor, the steps of the enterprise capability evaluation method are executed.

[0079] Compared with the prior art, the beneficial effects of the present application are reflected in:

[0080] 1. The present application proposes an enterprise capability evaluation method based on neural cognitive diagnosis technology, which first models factors such as enterprises, achievements, fields, achievement levels, etc. in a neural cognitive diagnosis model, predicts the completion of different enterprises on different achievements, and realizes the diagnosis of the mastery degree of scientific and technological capability of different enterprises in different fields.

[0081] 2. The present application is based on a rule method, constructs negative sample data in the diagnosis task, defines the partial order relationship among enterprises, achievements and fields, and proposes a partial order response sampling framework, which provides reliable auxiliary training signals for subsequent diagnosis models, thereby alleviating the negative effects of the long tail problem and improving the accuracy of enterprise capability cognitive diagnosis.

[0082] 3. The present application also introduces an expected capability change weighting information sampling module, which adaptively selects samples from the perspective of enterprise capability change, ensures the reliability of the training signal, and makes the model converge faster, so that the mastery status of different enterprises in different fields can be diagnosed faster. BRIEF DESCRIPTION OF DRAWINGS

[0083] Figure 1 is a flowchart of the overall method of the present application;

[0084] Figure 2 is a graph of the running results of three algorithms in the cstad dataset divided into different groups;

[0085] Figure 3 is a graph of the running results of three algorithms in the magazine dataset divided into different groups. DETAILED DESCRIPTION

[0086] In this embodiment, as shown in Figure 1 , an enterprise capability evaluation method based on neural cognitive diagnosis technology is to use a supervised deep learning method to model various fine-grained influencing factors in enterprise-technology achievement interaction, that is, to model the existing enterprise-technology achievement interaction data and technology achievement level factors in a neural cognitive diagnosis model, and to use basic technology achievement interaction data to expand the data to construct interaction data for adaptive diagnosis; so as to realize the capability evaluation of enterprises in different technology fields. Specifically, the method is performed according to the following steps:

[0087] Step 1, define enterprise technology data, and use FastText model to divide the enterprise technology data into fields to obtain the technology achievement field matrix , and construct the total enterprise-technology achievement interaction set after negative samples .

[0088] Step 1.1, define the number of enterprises in the technology data as , the number of technology achievements as , and the number of fields as , let the i-th enterprise, the m-th technology achievement, and the p-th field be , and , , , ;

[0089] Let the interaction result of the i-th enterprise to the m-th technology achievement be , and , denote the total enterprise-technology achievement interaction set .

[0090] Step 1.2, use the FastText classification model to divide into fields to obtain the technology achievement field matrix , , let the technology achievement field matrix of the m-th technology achievement belonging to the p-th field in be .

[0091] Step 1.3, based on the total enterprise-science achievement interaction set Construct the negative sample data of all enterprises;

[0092] Define the science achievement level as k, C represents the maximum level;

[0093] According to the science achievement level k from high to low, find the i-th enterprise All science achievements made, if there is a corresponding science achievement in the current science achievement level, do not construct negative samples; otherwise, randomly select Uninteracted fields to construct negative samples for each uninteracted field, thereby obtaining the i-th enterprise Construct Negative samples, and ; wherein, is the reverse order of the science achievement level of the m-th science achievement, is the total number of all science achievements made by the i-th enterprise , is the threshold value for controlling the number of negative samples for each enterprise, is the threshold value for controlling the total number of negative samples for each enterprise.

[0094] Let the total number of science achievements after constructing negative samples be , let the -th science achievement be denoted as , , and the total enterprise-science achievement interaction set after constructing negative samples is , The i-th enterprise in the total enterprise-science achievement interaction set The interaction triple of the i-th enterprise to the -th science achievement is denoted as , wherein, represents the interaction result of the i-th enterprise to the -th science achievement , if , it represents that the i-th enterprise completed the -th science achievement , if , it represents that the i-th enterprise did not complete the -th science achievement The interaction set of the i-th enterprise to all science achievements in the total enterprise-science achievement interaction set is denoted as ;

[0095] make The Middle scientific and technological achievements Belonging to the p-th domain The matrix of scientific and technological achievements is denoted as .

[0096] Step 2: Construct a basic neurocognitive diagnostic enterprise capability assessment learning module, including: an initial embedding representation layer, an enterprise capability diagnostic layer, and a prediction layer, and perform a matrix analysis of scientific and technological achievements. The first in scientific and technological achievements Belonging to the p-th domain Matrix of scientific and technological achievements Processing is performed to obtain the i-th company. For the p-th domain The scientific and technological achievements Predictive interaction results This leads to the construction of a basic neurocognitive diagnostic enterprise capability assessment network to assess losses. .

[0097] Step 2.1: The initial embedded representation layer obtains the i-th enterprise using equations (1) and (2) respectively. Enterprise capability mastery vector and the scientific and technological achievements Belonging to the p-th domain The matrix vector of the results domain , , ;

[0098] (1)

[0099] (2)

[0100] In equations (1) and (2), Represents the i-th company One-hot encoded vector, , Indicates the first scientific and technological achievements One-hot encoded vector, , Represents the matrix to be trained. .

[0101] Step 2.2, the initial embedded representation layer is constructed. scientific and technological achievements a technology metric value of the i-th scientific and technological achievement , a difficulty vector of the i-th scientific and technological achievement , a distinguishing vector of the i-th scientific and technological achievement ,

[0102] Step 2.2.1, the grade of the i-th scientific and technological achievement is denoted as , the number of the i-th scientific and technological achievement is denoted as , and the number of the i-th scientific and technological achievement is denoted as , so as to calculate the technology metric value of the i-th scientific and technological achievement by using formula (3):

[0103] (3)

[0104] In formula (3), min-max represents a normalization process.

[0105] Step 2.2.2, the difficulty vector of the i-th scientific and technological achievement is calculated by using formula (4) and formula (5): and the distinguishing vector of the i-th scientific and technological achievement is calculated by using formula (4) and formula (5):

[0106] (4)

[0107] (5)

[0108] In formula (4) and formula (5), denotes the one-hot encoding vector of the i-th scientific and technological achievement, denotes two matrices to be trained, .

[0109] Step 2.3, the enterprise capability diagnosis layer includes an interaction layer and two fully connected layers;

[0110] Step 2.3.1, the interaction layer of the i-th enterprise is established by using formula (6):​​​​​​​​​​​​​​​​​​​​​ the pth field accomplished by the ith enterprise the pth field accomplished by the ith enterprise the pth technological achievement accomplished by the ith enterprise the interaction function of the pth technological achievement accomplished by the ith enterprise :

[0111] (6)

[0112] In formula (6), is an element product.

[0113] Step 2.3.2, the fully connected layer utilizes formula (7) and formula (8) to obtain a feature representation of the pth technological achievement accomplished by the ith enterprise the pth field accomplished by the ith enterprise the pth field accomplished by the ith enterprise the pth technological achievement accomplished by the ith enterprise a feature representation of the pth technological achievement accomplished by the ith enterprise and a reduced dimension feature representation :

[0114] (7)

[0115] (8)

[0116] In formula (7)-formula (8), represents a feature representation of the pth technological achievement accomplished by the ith enterprise the pth field accomplished by the ith enterprise the pth field accomplished by the ith enterprise the pth technological achievement accomplished by the ith enterprise a feature representation of the pth technological achievement accomplished by the ith enterprise a feature representation of the pth technological achievement accomplished by the ith enterprise the pth field accomplished by the ith enterprise the pth field accomplished by the ith enterprise the pth technological achievement accomplished by the ith enterprise a feature representation of the pth technological achievement accomplished by the ith enterprise is a sigmoid activation function, , are respectively a to-be-trained weight matrix of two fully connected layers, , are respectively a bias vector of two fully connected layers.

[0117] Step 2.4, the prediction layer utilizes formula (9) to obtain a predicted interaction result of the pth technological achievement accomplished by the ith enterprise containing the pth field the pth field accomplished by the ith enterprise the pth technological achievement accomplished by the ith enterprise :

[0118] (9)

[0119] In formula (9), ​The weight matrix to be trained for the prediction layer. The bias vector of the prediction layer;

[0120] Step 2.5: Use formula (10) to calculate the i-th company. Loss of all scientific and technological achievements :

[0121] (10)

[0122] In equation (10), Represents the i-th company For the The completion status of the scientific and technological achievements. Represents the i-th company For the p-th domain The scientific and technological achievements The prediction results.

[0123] Step 3: Construct a partial order ranking learning module, including: a partial order learning module, an expected capability change module, a constraint design module for enterprise-technology achievement interaction results, and the construction of BPR loss;

[0124] Step 3.1: Construct a partial-order learning module and use a partial-order response sampling method to process the total interaction set of enterprise-technology achievements after negative samples. Perform partial order sampling to obtain the i-th company. Reliable candidate interaction set ;

[0125] Step 3.1.1: From the enterprise-technology achievement interaction set after constructing negative samples Obtain the i-th company For the scientific and technological achievements Interactive triples And use equation (11) to extract the first scientific and technological achievements Similar results sets :

[0126] (11)

[0127] In equation (11), It is a collection of technological achievements that enterprises have not interacted with. For the first u Uninteractive technological achievements In order to be with the first u Uninteractive technological achievements Related domain set, In order to be with the first Technological Achievements Related domain set.

[0128] Step 3.1.2: Construct the i-th enterprise using equation (12). For the u Uninteractive technological achievements Interactive triples :

[0129] (12)

[0130] In equation (12), For the i-th company For the u Uninteractive technological achievements The interaction results;

[0131] Step 3.1.3: Calculate the i-th company using equation (13). For the This project has already achieved interactive scientific and technological results. With the u Uninteractive technological achievements The similarity, and as a basis for constructing the i-th enterprise Non-interactive technology sampling probability :

[0132] (13)

[0133] In equation (13), This represents the i-th enterprise in the output of the neurocognitive diagnostic enterprise capability assessment network. For the Scientific and technological achievements eigenfactors, Indicates the first u Uninteractive technological achievements eigenfactors, Indicates belonging to a similar results set The jth scientific and technological achievement The characteristic factor of , where T represents transpose;

[0134] from The top k non-interactive scientific and technological achievements with the highest sampling probability are selected as the i-th enterprise. The sampling technology achievements are recorded as .

[0135] Step 3.1.4: Use equation (14) to obtain the i-th company. Reliable candidate interaction set :

[0136] (14)

[0137] In equation (14), Represents the i-th company For the u Uninteractive technological achievements Interactive triples.

[0138] Step 3.2: Construct a module for predicting changes in capabilities, used to calculate reliable candidate interaction sets. The i-th company For the u Uninteractive technological achievements The expected change in the interactive triplet ,and After normalization, the i-th company is obtained. For the u Uninteractive technological achievements Weight of expected capability changes .

[0139] Step 3.2.1: Use equation (15) to obtain the i-th company. In the Uninteractive technological achievements Changes in expected capabilities :

[0140] (15)

[0141] In equation (15), Represents the i-th company In the Uninteractive technological achievements The expected change in ability is assessed using a diagnostic interaction function. Represents the i-th company exist The degree of mastery over various fields Represents the i-th company exist Adding the uth non-interactive technological achievement The degree of mastery over various fields afterwards.

[0142] Step 3.2.2: Calculate the first step using equation (16). u Uninteractive technological achievements Normalized expected capability change weights :

[0143] (16)

[0144] In equation (16), Represents a reliable candidate interaction set The number of interactive technological achievements in China.

[0145] Step 3.3: Design constraints on the interaction results between enterprises and scientific and technological achievements, and construct the BPR loss function;

[0146] For the i-th company Depending on the interaction results, the interaction set after constructing negative samples will be used. Divided into positive interaction result sets after constructing negative samples and the negative interaction result set after constructing negative samples ;

[0147] Reliable candidate interaction set Positive interaction result set divided into reliable candidate sets Negative interaction result set with reliable candidate set .

[0148] Step 3.3.1, from , , , Take any positive interactive scientific and technological achievement from each of the following. Any candidate positive interaction scientific and technological achievement Any negative interaction technology achievement Any candidate negative interaction scientific and technological achievement Thus, the enterprise-technology achievement interaction result constraint is constructed using equation (17):

[0149] (17)

[0150] In equation (17), Represents the i-th company Positive interactive technological achievements The interaction results Represents the i-th company Negative interaction technology achievements The interaction results Represents the i-th company right The interaction results Represents the i-th company right The interaction results.

[0151] Step 3.3.2: Construct the i-th enterprise using equation (18) BPR loss of all scientific and technological achievements :

[0152] (18)

[0153] Step 4, training of the neural cognitive diagnosis enterprise capability evaluation network;

[0154] Step 4.1, constructing the i-th enterprise using formula (19) Total loss function of all scientific and technological achievements :

[0155] (19)

[0156] In formula (19), is a hyperparameter for balancing the two losses.

[0157] Step 4.2, training the neural cognitive diagnosis enterprise capability evaluation network composed of the basic neural cognitive diagnosis enterprise capability evaluation learning module and the partial order sorting learning module using the Adam optimizer, and minimizing until converges, thereby obtaining the optimal neural cognitive diagnosis enterprise capability evaluation network for diagnosing the mastery of enterprises in different fields, thereby obtaining the capability evaluation results of enterprises in different fields.

[0158] In this embodiment, an electronic device includes a memory for storing a program supporting a processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0159] In this embodiment, a computer readable storage medium stores a computer program on the computer readable storage medium, and the computer program is executed by a processor to perform the steps of the above method.

[0160] Embodiment:

[0161] In order to verify the effectiveness of the method, the experimental data set of the present application is obtained from four types of scientific and technological data provided by Qinghai Provincial Science and Technology Department, which are project (project data), magazine (journal data), patent (patent data) and cstad (achievement data) four types of data sets.

[0162] Since the true ability level of the enterprise is unknown, it is extremely difficult to directly evaluate the ability performance of a CDM. In the field of education, a reasonable solution is proposed in cognitive diagnosis, that is, to obtain the diagnosis result by predicting the task of the learning achievement of the learner, so as to measure the learning achievement of the student by using the prediction score in the diagnosis model. Here, the evaluation method in the field of education is followed, and AUC (area under the curve), ACC (accuracy) and RMSE (root mean square error) are used as evaluation criteria. The present application selects 7 methods for comparison, which are SVM (Support Vector Machine, SVM), KNN (K-Nearest Neighbors, KNN), DT (Decision Tree, DT), RF (Random Forest, RF), NCDM (Wang, Fei, et al. "Neural cognitive diagnosis for intelligent education systems." Proceedings of the AAAI conference on artificial intelligence. Vol. 34. No. 04. 2020), EIRS (Yao, Fangzhou, et al. "Exploiting non-interactive exercises in cognitive diagnosis." Interaction 100.200 (2023): 300), TNCDM (a kind of enterprise ability diagnosis method based on NCDM), and the performance of long tail data and overall data is compared by using the baseline method. The long tail data extracted from the test data includes the response records of the enterprises whose enterprise-achievement interactions are less than 10. Specifically, Table 1 shows the experimental results in the above data set, and it can be observed that the present application is better than the comparison method in terms of ACC, AUC and RMSE.

[0163] Table 1. Running results of multiple algorithms on patent, project data set

[0164]

[0165] Table 2. Running results of multiple algorithms on magazine, cstad data set

[0166]

[0167] In order to further verify the diagnosis effect of the method for long tail data, further divided into four groups according to the number of interactions to verify the prediction of different groups. The results are as followsFigure 2 Figure 3

[0168] It can be seen that the model achieves good results in all groups, especially in enterprises with fewer interactions (5-15, 15-25), the prediction results are more obvious than the baseline model. Therefore, it can be concluded that the contribution of the present application in diagnosing enterprise data with less interaction is more prominent.

[0169] Here, the present model is used to predict the completion of scientific and technological achievements of enterprises, and the ability mastering vector of the enterprise can be obtained is an L-dimensional vector, and the Lth component of the vector represents the ability value of the enterprise in the lth field. Since experiments are conducted in four types of scientific and technological achievements, four ability vectors can be obtained. According to the weight division of Qinghai scientific and technological achievements and the different importance of different scientific research achievements to enterprises, weighted summation is performed according to the weight of project: achievement: patent: paper = 4: 2: 1: 1, and the mastering degree of the enterprise in all fields is obtained. According to the field division, the top K enterprises with high mastering degree in the recommended field are recommended as the field prediction results.

[0170] In order to verify the accuracy of the present model for scientific and technological enterprise evaluation and scientific and technological enterprise recommendation, all Qinghai units and scientific research achievement data are compared according to the real recommendation results fed back by Qinghai Provincial Science and Technology Department, and the evaluation indexes use Precision@K (precision), Recall@K (recall) and F-score@K (F score).

[0171] The comparison results are shown in Table 3, in which TNCD is a recent talent evaluation model, and EIRS is a diagnosis model in the education field. The results of the three indexes are compared under the conditions of field recommendation length 5 and 8. It can be found that the present model is better than the recent talent evaluation model TNCD and the EIRS model in the education field in overall performance, because the present model considers the quality factor and quantity factor of the achievement in the difficulty value and discrimination value modeling, and we also fully consider the long tail problem of enterprise interaction in the modeling process, that is, the partial order response sampling scheme is adopted, so that the model is more robust and closer to the real recommendation needs of the field.

[0172] Table 3. Comparison of expert annotation of multiple algorithm field recommendation results

[0173]

[0174] ​​​Based on the above experiments, the method proposed in the application is significantly better than many comparison methods, thereby proving the feasibility of the method proposed in the application. By considering the quality and quantity factors of scientific and technological achievements, the achievements are embedded into the model as initial learning parameters of the model, and the achievement-aware sampling method is used to alleviate the long tail problem of data, so that the model is more effective and robust. Experiments are performed on four scientific and technological data sets in Qinghai, the effectiveness of the model is verified by comparing the baseline model, and the robustness of the model to long tail data is verified by long tail experiments. The results based on the ability diagnosis also verify the effectiveness of the model in the ability diagnosis.

Claims

1. A method for evaluating the capability of an enterprise based on a neural cognitive diagnosis technique, characterized by, is performed according to the following steps: Step 1, define enterprise technology data, and use FastText model to divide enterprise technology data into fields to get technology achievement field matrix , and construct the total interaction set of enterprises and technology achievements after negative samples ; Step 2: Construct a basic neurocognitive diagnostic enterprise capability assessment learning module, including: an initial embedding representation layer, an enterprise capability diagnostic layer, and a prediction layer, and sequentially apply this to the technology achievement domain matrix. The first in scientific and technological achievements Belonging to the p-th domain Matrix of scientific and technological achievements Processing is performed to obtain the i-th company. For the p-th domain The scientific and technological achievements Predictive interaction results This leads to the construction of a basic neurocognitive diagnostic enterprise capability assessment network to assess losses. ; Step 3, constructing a partial order sorting learning module, including: a partial order learning module, an expected capability change module, an enterprise-technology achievement interaction result constraint design module; Step 3.1: Construct a partial-order learning module and use a partial-order response sampling method to process the total interaction set of enterprise-technology achievements after negative samples. Perform partial order sampling to obtain the i-th company. Reliable candidate interaction set ; Step 3.2, build the expected capability change module for calculating the reliable candidate interaction set The i-th enterprise The i-th enterprise u The i-th un-interacted technological achievement The expected capability change of the i-th un-interacted technological achievement , and After normalization, the expected capability change weight of the i-th un-interacted technological achievement The i-th enterprise u The i-th un-interacted technological achievement The expected capability change of the i-th un-interacted technological achievement ; Step 3.3, Design the enterprise-technology achievement interaction result constraint, and build the ith enterprise BPR loss function for all technology achievements ; Step 4, building the ith firm using formula (19) Total loss function for all technology : (19) In formula (19), hyperparameters balancing the two losses; Step 5, training the neural cognitive diagnosis enterprise ability assessment network composed of the basic neural cognitive diagnosis enterprise ability assessment learning module and the partial order sorting learning module using the Adam optimizer and minimizing , until converges, thereby obtaining an optimal neural cognitive diagnosis enterprise ability assessment network for diagnosing the mastery of enterprises in different fields, thereby obtaining the enterprise ability assessment results in different fields.

2. The enterprise capability assessment method based on neural cognitive diagnosis technology according to claim 1, characterized in that, The step 1 is performed according to the following steps: Step 1.1, define the number of enterprises in the science and technology data as , the number of scientific and technological achievements as , the number of fields as , let the i-th enterprise, the m-th scientific and technological achievement and the p-th field be respectively denoted as , and , , , ; Let the ith firm The interaction result of the mth scientific achievement is denoted as , and , The total interaction set of the firm-scientific achievement is denoted as ; Step 1.2, divide the field by using FastText classification model on , , , , , , ; Step 1.3, Total interaction set based on enterprise-technology achievement Constructing negative sample data for all enterprises; defining the level of scientific and technological achievements as k, C represents the maximum level; According to the technology achievement level k from high to low, find the i-th enterprise All the technology achievements are made, if the current technology achievement level has corresponding technology achievements, no negative samples are constructed; otherwise, in Randomly select Uninteracted fields respectively construct negative samples of each uninteracted field, so as to obtain the i-th enterprise The constructed Negative samples, and ; wherein, The technology achievement level of the m-th technology achievement is in reverse order, The total number of all technology achievements made by the i-th enterprise , The threshold value for controlling the number of negative samples of each enterprise, The threshold value for controlling the total number of negative samples of each enterprise; Let the total number of scientific achievements after constructing negative samples be , , Let the , , the total enterprise-scientific achievement interaction set after constructing negative samples be , , the total enterprise-scientific achievement interaction set , , the interaction triple of the i-th enterprise , , the i-th scientific achievement , , wherein , , the i-th scientific achievement , if , , the i-th enterprise , , if , , the i-th enterprise , , the interaction set of the i-th enterprise , , all scientific achievements is ; Let the p th technology achievement in the p th field be denoted by 3.The enterprise capability evaluation method based on neural cognitive diagnosis technology according to claim 2, characterized in that, The step 2 is performed according to the following steps: Step 2.1: The initial embedded representation layer obtains the i-th enterprise using equations (1) and (2) respectively. Enterprise capability mastery vector and the scientific and technological achievements Belonging to the p-th domain The matrix vector of the results domain , , ; (1) (2) In formula (1) and formula (2), represents a one-hot encoding vector of the i-th enterprise, , represents a one-hot encoding vector of the i-th technology achievement, , represents a one-hot encoding vector of the i-th technology achievement, represents a matrix to be trained,​​​ Step 2.2, the initial embedding representation layer constructs the first Item scientific achievement The scientific metric value of , The scientific achievement completion difficulty vector , And the first Item scientific achievement The distinction degree vector of , ; Step 2.3, the enterprise capability diagnosis layer includes an interaction layer and two fully connected layers, and sequentially processes the input data to obtain the feature representation and the reduced dimension feature representation of the pth field of the ith enterprise , and achieved by the enterprise . ​​​​​ Step 2.4: The prediction layer uses equation (9) to obtain the i-th enterprise. For the p-th domain The scientific and technological achievements Predictive interaction results : (9) In formula (9), is a weight matrix to be trained for the prediction layer, is a bias vector for the prediction layer. Step 2.5, use formula (10) to calculate the i-th firm Loss to all scientific and technological achievements : (10) In formula (10), represents the i-th enterprise the i-th enterprise the completion of the i-th scientific and technological achievement, represents the i-th enterprise the prediction result of the i-th scientific and technological achievement containing the p-th field .​ 4. The enterprise capability assessment method based on neural cognitive diagnosis technology according to claim 3, characterized in that, The step 2.2 is performed according to the following steps: Step 2.2.1, the first Item of scientific and technological achievements The corresponding series of the scientific and technological achievements level is recorded as , the first Item of scientific and technological achievements The corresponding number under Is recorded as , so as to calculate the scientific and technological measurement value of the first Item of scientific and technological achievements Using formula (3) : (3) In formula (3), min-max represents a normalization process; Step 2.2.2: Calculate the first step using equations (4) and (5). scientific and technological achievements Completion Difficulty Vector , and the scientific and technological achievements Discrimination vector , : (4) (5) In formula (4) and formula (5), indicates the first item of scientific and technological achievements one-hot encoding vector, , respectively represent two matrices to be trained, , .

5. The enterprise capability assessment method based on neural cognitive diagnosis technology according to claim 4, characterized in that, The step 2.3 is performed according to the following steps: Step 2.3.1, the interaction layer uses formula (6) to establish the ith enterprise the completed pth field the item of technological achievement interaction function : (6) In formula (6), is the product of the elements; Step 2.3.2, the fully connected layer utilizes the i-th business' feature representation of formula (7) and formula (8) to obtain the i-th business the p-th completed field the i-th scientific and technological achievement the feature representation of the i-th and the reduced dimension feature representation of the i-th : (7) (8) in formula (7) - formula (8), represents the i-th enterprise the p-th field completed by the i-th enterprise the i-th enterprise the p-th field completed by the i-th enterprise the i-th enterprise the p-th field completed by the i-th enterprise the i-th enterprise the p-th field completed by the i-th enterprise the i-th enterprise the i-th enterprise is a sigmoid activation function, , are respectively the weight matrices to be trained of the two fully connected layers, , are respectively the bias vectors of the two fully connected layers.

6. The enterprise capability assessment method based on neural cognitive diagnosis technology according to claim 5, characterized in that, The step 3.1 is performed according to the following steps: Step 3.1.

1. Obtaining the i-th enterprise-science achievement interaction set from the constructed negative sample , :​​​​​​​ (11) In formula (11), is a collection of technology achievements that are not interacted by the enterprise, , is a first u technology achievement that is not interacted, is a set of fields related to the first u technology achievement that is not interacted, is a set of fields related to the first technology achievement that is not interacted, is a set of fields related to the first technology achievement that is not interacted. Step 3.1.2, constructing the ith enterprise with formula (12) the ith enterprise u non-interactive technology achievements interactive triples : (12) In formula (12), is the i-th enterprise to the i-th u uninteracted technology achievements interacted results; Step 3.1.

3. Obtain the reliable candidate interaction set for the i-th enterprise using formula (14) :​ (14) In formula (14), is the number of non-interaction samples in represents the i-th enterprise the j-th u item of non-interaction scientific and technological achievements the interaction triplets of the i-th 7. The enterprise capability assessment method based on neural cognitive diagnosis technology according to claim 6, characterized in that, The step 3.2 is performed according to the following steps: Step 3.2.1, obtaining the ith firm with formula (15) In the first Item of the interaction technology achievements The expected change in ability : (15) In formula (15), represents the i-th enterprise In the first un-interacted technology achievement The expected ability change uses a diagnostic interaction function, represents the i-th enterprise In the degree of mastery of each field, represents the i-th enterprise In the plus the u-th un-interacted technology achievement after the degree of mastery of each field; Step 3.2.

2. Calculating the normalized expected ability change weight of the item using formula (16) u Item non-interaction technology achievements of the normalized expected ability change weight : (16) In formula (16), represents the number of interaction technology achievements in the reliable candidate interaction set ​ 8. The enterprise capability assessment method based on neural cognitive diagnosis technology according to claim 7, characterized in that, The step 3.3 is performed according to the following steps: Step 3.3.1, for the i-th enterprise According to the different interaction results, the interaction set after constructing negative samples is divided into the positive interaction result set after constructing negative samples and the negative interaction result set after constructing negative samples and the negative interaction result set after constructing negative samples ; set of reliable candidate interactions set of positive interaction outcomes in reliable candidate set and set of negative interaction outcomes in reliable candidate set ; From , , , any orthogonal interaction technology achievement , any candidate orthogonal interaction technology achievement , any negative interaction technology achievement , any candidate negative interaction technology achievement , the enterprise-technology achievement interaction result constraint is constructed by using formula (17): (17) In formula (17), represents the i-th enterprise for the positive interaction technology achievement interaction result, represents the i-th enterprise for the negative interaction technology achievement interaction result, represents the i-th enterprise for the interaction result, represents the i-th enterprise for the interaction result; Step 3.3.2, building the ith business with formula (18) BPR loss for all technology achievements : (18)。 9. An electronic device comprising a memory and a processor, characterized in that The memory is used to store a program supporting the processor to execute the enterprise capability evaluation method in any one of claims 1-8, and the processor is configured to execute the program stored in the memory.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to perform the steps of the enterprise capability evaluation method in any one of claims 1-8.

Images