Method and apparatus for providing laboratory evaluation service, and learning method for neural network for providing laboratory evaluation service

Abstract

Disclosed are a method and apparatus for providing a laboratory evaluation service, and a learning method for a neural network for providing a laboratory evaluation service. The method for providing a laboratory evaluation service, according to an embodiment, may comprise the steps of: calculating an evaluation score of each of a plurality of laboratories on the basis of thesis information of the plurality of laboratories belonging to each of a plurality of universities and value information for each academy corresponding to the values of a plurality of academies publishing thesis; and providing a list in which rankings of the plurality of laboratories are indicated on the basis of the score of each of the plurality of laboratories.

Description

Method and apparatus for providing laboratory evaluation services, and learning method of a neural network for providing laboratory evaluation services

[0001] The following disclosure relates to a method and apparatus for providing laboratory evaluation services, and a method for learning a neural network for providing laboratory evaluation services.

[0002] Conventional university-level evaluation methods may struggle to reflect the actual research achievements of the various laboratories affiliated with a university. Consequently, there has been a problem where students seeking graduate school find it difficult to find information about these laboratories.

[0003] The background technology described above is possessed or acquired by the inventor in the process of deriving the content of the disclosure of the present application, and cannot necessarily be considered as prior art disclosed to the general public prior to the filing of this application.

[0004] One embodiment may provide a technology that offers laboratory-level evaluation services to more accurately identify the strengths and specialized research fields of a specific laboratory.

[0005] However, technical challenges are not limited to the technical challenges described above, and other technical challenges may exist.

[0006] A method for providing a laboratory evaluation service according to one embodiment may include the operation of calculating an evaluation score for each of the plurality of laboratories based on paper information of the plurality of laboratories belonging to each of the plurality of universities and value information for each academic society corresponding to the value of the plurality of academic societies that publish papers, and the operation of providing a list showing the ranking of the plurality of laboratories based on the evaluation score of each of the plurality of laboratories.

[0007] The paper information of the aforementioned multiple laboratories may include the number of citations of papers published by each of the aforementioned multiple laboratories and information regarding the academic society that published the papers.

[0008] The operation of calculating the evaluation score of each of the plurality of laboratories may include the operation of calculating a paper score for papers published by each of the plurality of laboratories based on the number of citations of the papers, and the operation of calculating the evaluation score of each of the plurality of laboratories through a neural network based on the paper score and the value information by academic society.

[0009] The operation of calculating the above paper score may include the operation of adjusting the number of citations of the above papers and the operation of calculating the above paper score for each of the above multiple laboratories by applying a decay rate according to the publication year of the above papers to the adjusted number of citations.

[0010] The operation of adjusting the citation count of the above papers may include analyzing the purpose of citing the above papers and applying a weight according to the purpose to the citation count of the above papers.

[0011] The operation of adjusting the number of citations of the above papers may include the operation of calculating the internal citation ratio of the said papers based on the relationship between the subject citing the said papers and the research laboratory that published the said papers, and the operation of adjusting the number of citations of the said papers based on the internal citation ratio of the said research laboratory.

[0012] The operation of calculating the evaluation score of each of the plurality of laboratories through the neural network may include the operation of performing a linear transformation on the paper score and the operation of performing an attention operation on the linearly transformed paper score and the value information for each academic society to calculate the evaluation score of each of the plurality of laboratories.

[0013] The operation of providing a list showing the ranking of the plurality of laboratories may include the operation of classifying the plurality of laboratories by research field and the operation of providing a list showing the ranking of laboratories having the same research field based on the evaluation score of each of the laboratories having the same research field.

[0014] According to one embodiment, an electronic device providing a laboratory evaluation service may include a processor and a memory for storing instructions. The instructions may be executed individually or collectively by the processor to enable the electronic device to calculate an evaluation score for each of the plurality of laboratories based on paper information of the plurality of laboratories belonging to each of the plurality of universities and value information for each academic society corresponding to the value of the plurality of academic societies publishing the papers. The instructions may be executed individually or collectively by the processor to enable the electronic device to provide a list showing the ranking of the plurality of laboratories based on the evaluation scores of each of the plurality of laboratories.

[0015] The paper information of the aforementioned multiple laboratories may include the number of citations of papers published by each of the aforementioned multiple laboratories and information regarding the academic society that published the papers.

[0016] The above instructions may be executed individually or collectively by the processor to enable the electronic device to calculate paper scores for papers published by each of the plurality of laboratories based on the number of citations of the papers. The above instructions may be executed individually or collectively by the processor to enable the electronic device to calculate evaluation scores for each of the plurality of laboratories through a neural network based on the paper scores and the academic society value information.

[0017] The above instructions may be executed individually or collectively by the processor to cause the electronic device to adjust the citation counts of the papers. The above instructions may be executed individually or collectively by the processor to cause the electronic device to calculate the paper scores for each of the plurality of laboratories by applying a decay rate based on the publication year of the papers to the adjusted citation counts.

[0018] The above instructions may be executed individually or collectively by the processor to enable the electronic device to analyze the purpose of citing the papers and apply a weight according to the purpose to the number of citations of the papers.

[0019] The above instructions may be executed individually or collectively by the processor to enable the electronic device to calculate the internal laboratory citation rate for the papers based on the relationship between the subject citing the papers and the laboratory that published the papers. The above instructions may be executed individually or collectively by the processor to enable the electronic device to adjust the number of citations of the papers based on the internal laboratory citation rate.

[0020] The above instructions may be executed individually or collectively by the processor to cause the electronic device to perform a linear transformation on the paper score. The above instructions may be executed individually or collectively by the processor to cause the electronic device to perform an attention operation on the linearly transformed paper score and the academic society value information to calculate the evaluation score of each of the plurality of laboratories.

[0021] The above instructions may be executed individually or collectively by the processor to enable the electronic device to classify the plurality of laboratories by research field. The above instructions may be executed individually or collectively by the processor to enable the electronic device to provide a list showing the ranking of laboratories having the same research field, based on the evaluation score of each of the laboratories having the same research field.

[0022] A method for training a neural network according to one embodiment may include: an operation of calculating an evaluation score for each of the plurality of laboratories through the neural network based on paper information of the plurality of laboratories belonging to each of the plurality of universities and value information for each academic society that publishes the papers; an operation of predicting the ranking of the plurality of universities by adding the evaluation scores of laboratories belonging to the same university for each of the plurality of universities; and an operation of training the neural network by comparing the predicted ranking of the plurality of universities with an actual value.

[0023] The operation of training the neural network may include adjusting the parameters of the neural network through a backpropagation algorithm based on the difference between the predicted rankings of the multiple universities and the actual values.

[0024] The operation of adjusting the parameters of the above neural network may include applying logarithmic weights and comparing the predicted rankings of multiple universities with the actual values.

[0025] The operation of adjusting the parameters of the neural network may include applying a hinge loss to the difference between the predicted rankings of the multiple universities and the actual values, and updating the parameters so that the predicted rankings of the multiple universities and the actual values ​​are minimized.

[0026] According to one embodiment, a system for establishing a community to provide a laboratory evaluation service may include a plurality of electronic devices and a server connected to the plurality of electronic devices via a network. The server may assign a member ID to each of the plurality of electronic devices in response to a membership request transmitted from each of the plurality of electronic devices via the network. The server may establish the community by transmitting data related to the community to the electronic device corresponding to the member ID upon the login of the electronic device corresponding to the member ID. The server may provide a laboratory evaluation service on the community. The laboratory evaluation service may be provided through a method for providing a laboratory evaluation service. The method for providing a laboratory evaluation service may include an operation of calculating an evaluation score for each of the plurality of laboratories based on paper information of the plurality of laboratories affiliated with each of the plurality of universities and value information by academic society corresponding to the value of the plurality of academic societies that publish the papers, and an operation of providing a list showing the ranking of the plurality of laboratories based on the evaluation scores of each of the plurality of laboratories.

[0027] The paper information of the aforementioned multiple laboratories may include the number of citations of papers published by each of the aforementioned multiple laboratories and information regarding the academic society that published the papers.

[0028] The operation of calculating the evaluation score of each of the plurality of laboratories may include the operation of calculating a paper score for papers published by each of the plurality of laboratories based on the number of citations of the papers, and the operation of calculating the evaluation score of each of the plurality of laboratories through a neural network based on the paper score and the value information by academic society.

[0029] The operation of calculating the above paper score may include the operation of adjusting the number of citations of the above papers and the operation of calculating the above paper score for each of the above multiple laboratories by applying a decay rate according to the publication year of the above papers to the adjusted number of citations.

[0030] The operation of adjusting the citation count of the above papers may include analyzing the purpose of citing the above papers and applying a weight according to the purpose to the citation count of the above papers.

[0031] The operation of adjusting the number of citations of the above papers may include the operation of calculating the internal citation ratio of the said papers based on the relationship between the subject citing the said papers and the research laboratory that published the said papers, and the operation of adjusting the number of citations of the said papers based on the internal citation ratio of the said research laboratory.

[0032] The operation of calculating the evaluation score of each of the plurality of laboratories through the neural network may include the operation of performing a linear transformation on the paper score and the operation of performing an attention operation on the linearly transformed paper score and the value information for each academic society to calculate the evaluation score of each of the plurality of laboratories.

[0033] The operation of providing a list showing the ranking of the plurality of laboratories may include the operation of classifying the plurality of laboratories by research field and the operation of providing a list showing the ranking of laboratories having the same research field based on the evaluation score of each of the laboratories having the same research field.

[0034] FIG. 1 is an example of a system providing a laboratory evaluation service according to one embodiment.

[0035] FIG. 2 is an example of an electronic device according to one embodiment.

[0036] FIG. 3 is an example of a screen where a laboratory evaluation service is provided according to one embodiment.

[0037] FIG. 4 is a diagram illustrating a method for calculating the evaluation score of a laboratory according to one embodiment.

[0038] FIG. 5 is an example of a flowchart of a method for providing a laboratory evaluation service according to one embodiment.

[0039] FIG. 6 is an example of a learning device according to one embodiment.

[0040] FIG. 7 is an example of a flowchart of a learning method for a neural network according to one embodiment.

[0041] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions included in the technical concept described by the embodiments.

[0042] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0043] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or joined to that other component, or that there may be other components in between.

[0044] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to specify the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0045] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0046] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0047]

[0048] FIG. 1 is an example of a laboratory evaluation service system according to one embodiment.

[0049] Referring to FIG. 1, a lab assessment service system (100) may include an electronic device (200) and a server (150). However, FIG. 1 is an example for explaining the present invention and should not be interpreted as limiting the scope of the present invention thereto. For example, the electronic device (200) may provide a lab assessment service to a user independently without a server (150).

[0050] The electronic device (200) may be a smartphone, cellular phone, personal computer, laptop, notebook, netbook or tablet, personal digital assistant (PDA), digital camera, game console, MP3 player, personal multimedia player (PMP), e-book, navigation device, or home appliance, but is not limited thereto.

[0051] The server (150) and the electronic device (200) can communicate using a network (not shown). For example, the network may include a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a mobile radio communication network, a satellite communication network, and combinations thereof. The network is a comprehensive data communication network that enables the server (150) and the electronic device (200) to communicate smoothly with each other, and may include wired internet, wireless internet, and mobile wireless communication networks. Additionally, the wireless communication network may include, for example, Wi-Fi, Bluetooth, Bluetooth Low Energy, Zigbee, Wi-Fi Direct (WFD), Ultra-Wideband (UWB), Infrared Data Association (IrDA), Near Field Communication (NFC), but is not limited thereto.

[0052] The server (150) can establish a community for graduate students (e.g., a community for providing laboratory evaluation services) and provide it to users through an electronic device (200). The server (150) can provide a community for graduate students and / or a mentoring app to the electronic device (200). For example, the server (150) can assign a member ID to each of the multiple electronic devices (e.g., electronic device (200)) in response to a membership request transmitted from each of the multiple electronic devices via a network. The server (150) can establish a community by transmitting data related to the community to the electronic device corresponding to the member ID upon the login of the electronic device corresponding to the member ID. The server (150) can provide laboratory evaluation services on the community.

[0053] The server (150) can provide a laboratory evaluation service to a user on a community through the electronic device (200), as described in FIGS. 2 to 7. Additionally, the electronic device (200) can provide a laboratory evaluation service to a user on a community provided by the server (150), as described in FIGS. 2 to 7.

[0054] Some and / or all of the operations performed in the electronic device (200) may be performed in the electronic device (200) and / or server (150). The following description will continue on the premise that the electronic device (200) is performing the operations.

[0055] The electronic device (200) can analyze research performance by multiple laboratories and evaluate research performance by laboratory unit. The electronic device (200) can rank multiple laboratories based on the evaluation of research performance by laboratory unit (e.g., evaluation score of a laboratory). The electronic device (200) can provide a list of rankings to the user. A screen showing a list of rankings of multiple laboratories will be described in detail with reference to FIG. 3.

[0056]

[0057] FIG. 2 is an example of an electronic device according to one embodiment.

[0058] Referring to FIG. 2, the electronic device (200) can calculate an evaluation score (e.g., laboratory score) for each of the multiple laboratories based on paper information of the multiple laboratories and value information for each academic society corresponding to the value of the multiple academic societies that publish the papers.

[0059] The electronic device (200) can calculate an evaluation score for each of the multiple laboratories based on paper information and academic society value information through a neural network. The electronic device (200) can generate a list showing the rankings of the multiple laboratories based on the evaluation scores of the multiple laboratories.

[0060] Neural networks (or artificial neural networks) in machine learning and cognitive science can include statistical learning algorithms that mimic biological neurons. A neural network can refer to a model in general that possesses problem-solving capabilities by having artificial neurons (nodes), which form a network through synaptic connections, change the strength of these connections through learning.

[0061] Neurons in a neural network may include a combination of weights or biases. A neural network may include one or more neurons or one or more layers composed of nodes. A neural network can infer a result to be predicted from an arbitrary input by changing the weights of the neurons through learning.

[0062] Neural networks can include deep neural networks. Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto Encoder), and DAE (Denoising Auto). Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial Network), Liquid State Machine (LSM), Extreme Learning Machine (ELM), Echo State Network (ESN), Deep Residual Machine (DRN) It may include Network), DNC (Differentiable Neural Computer), NTM (Neural Turning Machine), CN (Capsule Network), KN (Kohonen Network), Transformer, and AN (Attention Network).

[0063] The electronic device (200) may include memory (210) and a processor (230).

[0064] Memory (210) can store a neural network or parameters of a neural network. Memory (210) can store instructions (or programs) executable by a processor. For example, the instructions may include instructions for executing the operation of the processor and / or the operation of each component of the processor.

[0065] The memory (210) can be implemented as a volatile memory device or a non-volatile memory device.

[0066] Volatile memory devices can be implemented as DRAM (dynamic random access memory), SRAM (static random access memory), T-RAM (thyristor RAM), Z-RAM (zero capacitor RAM), or TTRAM (Twin Transistor RAM).

[0067] Non-volatile memory devices can be implemented as EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, MRAM (Magnetic RAM), Spin-Transfer Torque (STT)-MRAM, Conductive Bridging RAM (CBRAM), FeRAM (Ferroelectric RAM), PRAM (Phase change RAM), Resistive RAM (RRAM), Nanotube RRAM, Polymer RAM (PoRAM), Nano Floating Gate Memory (NFGM), holographic memory, Molecular Electronic Memory Device, or Insulator Resistance Change Memory.

[0068] The processor (230) can process data stored in memory (210). The processor (230) can execute computer-readable code (e.g., software) stored in memory (210) and instructions triggered by the processor (200).

[0069] The "processor (230)" may be a data processing device implemented in hardware having a circuit having a physical structure for executing desired operations. For example, the desired operations may include code or instructions included in a program.

[0070] For example, a data processing device implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an Application-Specific Integrated Circuit (ASIC), and a Field Programmable Gate Array (FPGA).

[0071] The processor (230) can enable the electronic device (200) to perform one or more operations by executing code and / or instructions stored in memory (210). Hereinafter, operations that can be performed by the electronic device (200) will be described in detail with reference to FIGS. 2 to 5.

[0072] The electronic device (200) can calculate an evaluation score (e.g., laboratory score) for each of the multiple laboratories based on paper information of the multiple laboratories and value information for each academic society corresponding to the value of the multiple academic societies that publish the papers.

[0073] The paper information may include the number of citations for papers published by multiple laboratories and information regarding the academic societies in which the papers were presented. For example, it is assumed that the first laboratory has published 100 papers (e.g., including papers 1 through 100). The paper information may include information regarding how many times the first paper from the first laboratory has been cited in other papers (or academic journals, etc.). Additionally, the paper information may include information regarding the academic society in which the first paper was presented (e.g., the field of the society, the name of the society, and the credibility of the society, etc.).

[0074] The value information for each academic society may be determined based on whether the society publishing the paper is a verified academic society and / or the reputation of the society. In one embodiment, the value of multiple academic societies may include one of the metrics used by members of the academic community to evaluate the society. For example, the value of multiple academic societies may include a score evaluation of the authority, recognition, and / or reputation of the society as assessed by others. For example, in the case of an academic society with a relatively high value, information such as papers published by that society may be easier to gain the trust of members of the academic community compared to information published by an academic society with a relatively low value. For example, in the case of an academic society with a relatively low value, information such as papers published by that society may be relatively more difficult to gain the trust of members of the academic community compared to information published by an academic society with a relatively high value. For example, a value may be assigned to each academic society by comprehensively considering the year of establishment of the society, the difficulty of the society, and the level of its members (e.g., degrees). Value information by society may be set by the processor (230) or by the user.

[0075] The electronic device (200) can obtain paper information and / or academic society value information through AI (artificial intelligence)-based crawling (e.g., web crawling). The electronic device (200) can process vast amounts of data in a complex manner through AI-based crawling to systematically analyze large-scale research activities and research results that are difficult for humans to understand, and can reflect the latest data in real time.

[0076] The electronic device (200) can calculate a paper score for each paper for each of the multiple laboratories based on the number of citations of papers published for each of the multiple laboratories. For example, the electronic device (200) can calculate a first paper score for the first paper based on the number of citations of the first paper published in the first laboratory. The electronic device (200) can calculate a second paper score for the second paper based on the number of citations of the second paper published in the first laboratory. The electronic device (200) can calculate a total paper score for the papers published in the first laboratory by performing neural network operations (e.g., including non-linear operations and / or linear operations) on the paper scores for all papers published in the first laboratory (e.g., first paper score and second paper score).

[0077] The electronic device (200) can adjust the number of citations of papers and use the adjusted number of citations when calculating the paper score. If the number of citations is not adjusted, the following problems may occur. For example, the number of citations of a paper may be inflated through unnecessary citations (e.g., the process in which fellow researchers cite a paper even though it has no citation value). In this way, if a paper is cited exaggeratedly even though it has no actual citation value, and the number of citations is not adjusted, the objectivity of the laboratory's evaluation score may be lost. To prevent this, the electronic device (200) can calculate an objective evaluation score by determining whether the citations are actually meaningful by adjusting the number of citations of papers.

[0078] The electronic device (200) can calculate the internal laboratory citation ratio (e.g., the first) for the papers based on the relationship between the subject citing the papers and the laboratory that published the papers. The electronic device (200) can adjust the number of citations for the papers based on the internal laboratory citation ratio for the papers. The electronic device (200) can reduce the weight of inbreeding of the number of paper citations (e.g., citing papers published in the same laboratory within the same laboratory) through the analysis of the graph structure. For example, the electronic device (200) can create (or classify) multiple groups for multiple laboratories by clustering the laboratories and the papers corresponding to the laboratories (e.g., papers published in the laboratories). The electronic device (200) can determine whether the generated groups cite each other. If the generated groups cite each other, the electronic device (200) can adjust the number of citations for the corresponding papers to be lower. That is, the electronic device (200) can determine whether the first paper published in the first laboratory is being cited in the first laboratory. If the electronic device (200) finds that the first paper published in the first laboratory is being cited in the first laboratory frequently (e.g., if the internal citation rate of the first laboratory (e.g., the rate at which the first paper published in the first laboratory is being cited in the first laboratory) is higher than a preset threshold value), it can reduce the number of citations of the first paper by applying a weight (e.g., less than 1) to the number of citations of the first paper.

[0079] The electronic device (200) can analyze the purpose of citing the papers and apply a weight based on the purpose of citing the papers to the number of citations of the papers. For example, the electronic device (200) can analyze the citation phrases of the papers to analyze how the papers were cited and / or what the citation phrases mean. If the purpose of citing the papers is not valuable, the electronic device (200) can apply a corresponding weight (e.g., less than 1) to the number of citations of the papers. As another example, the number of citations of the first paper may be 10. The electronic device (200) can analyze other papers that cited the first paper through natural language processing to analyze the purpose for which the first paper was cited. If the purpose for which the first paper was cited is meaningful (e.g., to obtain results in terms of obtaining formulas listed in the first paper or continuing important results), the electronic device (200) can set the weight to 1. In this case, the electronic device (200) may apply a weight (e.g., 1) to the number of citations (e.g., 10) of the first paper. On the other hand, if the purpose for which the first paper was cited is meaningless (e.g., merely to provide an example), the electronic device (200) may set the weight to a real number between 0 and 1 (e.g., 0.5). In this case, the electronic device (200) may apply a weight (e.g., 0.5) to the number of citations (e.g., 10) of the first paper.

[0080] The electronic device (200) can use a decay rate based on the year of publication by considering the year of publication of the papers when calculating the paper score. If the decay rate based on the year of publication of the papers is not applied, the following problems may occur. For example, a laboratory may undergo rapid changes as internal personnel are replaced in a relatively short period of time (e.g., on average 5 years, or as short as 2 years). That is, if weighting based on time is not applied, it may be difficult to reflect such changes when reflecting research performance. The processor (230) can apply a decay rate based on the year of publication of the papers to reflect these changes when calculating the paper score.

[0081] Specifically, the electronic device (200) can calculate the paper score for each laboratory through the following mathematical formula 1.

[0082]

[0083] S A represents the paper score of Laboratory A (e.g., the sum of the paper scores of papers published in Laboratory A), N represents the number of papers published in Laboratory A, and C i represents the adjusted citation count of the i-th paper among the papers published in Laboratory A, and Y {current} represents the current year, and Y i represents the publication year of the i-th paper, and represents the attenuation rate according to the year of publication of the paper.

[0084] Above, a method for calculating the paper score for each laboratory (e.g., the sum of the paper scores of papers published in the laboratory) was described. The electronic device (200) can calculate the evaluation score of each of the multiple laboratories through a neural network based on the paper score and the value information of each academic society, and this will be explained in detail with reference to FIG. 4.

[0085]

[0086] FIG. 3 is an example of a screen where a laboratory evaluation service is provided according to one embodiment.

[0087] Referring to FIG. 3, the screen (300) displays a laboratory ranking list provided by an electronic device (e.g., the electronic device (200) of FIG. 1).

[0088] The electronic device (200) may provide a list showing the rankings of multiple laboratories belonging to each of multiple universities. The electronic device (200) may rank different laboratories (e.g., Laboratory A, Laboratory B, and Laboratory C) belonging to different universities (e.g., University X, University Y, and University Z) in order of highest evaluation score (e.g., laboratory score). For example, if the evaluation scores are highest in the order of Laboratory A, Laboratory B, and Laboratory C, the electronic device (200) may provide (or display) a list showing the rankings of Laboratory A, Laboratory B, and Laboratory C, as shown in the screen (300).

[0089] The electronic device (200) may provide a ranking list according to the field of study. The electronic device (200) may classify multiple laboratories by field of study. The electronic device (200) may provide a list showing the rankings of laboratories with the same field of study based on the score of each laboratory with the same field of study. For example, laboratory A and laboratory B may have the same field of study, while laboratory C may have a different field of study from laboratory A and laboratory B. In this case, unlike the screen (300), a list showing the rankings of laboratory A and laboratory B, excluding laboratory C, may be provided.

[0090] The method of indicating laboratory rankings is not limited to the method described above. For example, the electronic device (200) may classify multiple laboratories by the same university and provide a ranking list for multiple laboratories belonging to the same university. As another example, the electronic device (200) may provide a ranking list for multiple laboratories belonging to some universities (e.g., universities selected by the user). As yet another example, the electronic device (200) may provide a ranking list for multiple laboratories according to some research fields (e.g., research fields selected by the user).

[0091]

[0092] FIG. 4 is a diagram illustrating a method for calculating the evaluation score of a laboratory according to one embodiment.

[0093] Referring to FIG. 4, the electronic device (200) may include neural networks (4001 to 400-3). The neural networks (400-1 to 400-3) may include a plurality of layers. The plurality of layers may include a linear layer (410-1) and an attention layer (420-1). For example, the neural network (400-1) may include a linear layer (410-1) and an attention layer (420-1). The neural networks (400-2 and 400-3) may also be substantially identical in structure to the neural network (400-1).

[0094] The electronic device (200) can calculate the evaluation scores (450-1 to 450-3) of each of the multiple laboratories based on the paper information (410) of the multiple laboratories and the value information (430) of the academic society.

[0095] The electronic device (200) can calculate the paper scores of papers published by multiple laboratories based on the paper information (410) of multiple laboratories. For example, the electronic device (200) can calculate the paper scores of the first to third papers based on the paper information regarding three papers published by the first laboratory (e.g., the first to third papers). This has been explained in detail with reference to FIG. 2, so redundant explanations will be omitted below.

[0096] The electronic device (200) can calculate evaluation scores (450-1 to 450-3) based on the paper scores of multiple laboratories and value information by academic society (430) through a neural network (400-1 to 400-3). The electronic device (200) can perform a linear transformation on the paper scores of multiple laboratories. The electronic device (200) can calculate the evaluation score of each of the multiple laboratories by performing an attention operation on the linearly transformed paper scores and value information by academic society (430).

[0097] For example, let us assume a case where three papers (e.g., the first to third papers) have been published by the first laboratory. The linear layer (410-1) can perform a linear transformation on the paper score of the first paper published by the first laboratory. The linear layer (410-1) can output the linearly transformed paper score of the first paper to the attention layer (420-1). The attention layer (420-1) can perform an attention operation using the linearly transformed paper score of the first paper as a query and the value information by conference (430) as a key and value. The second and third papers can also be processed by the neural network (400-2 and 400-3) in substantially the same way as the first paper. The electronic device (200) can calculate the evaluation score (450-1) of the first laboratory by calculating the processing results of three papers (e.g., the first to third papers) published in the first laboratory using neural networks (400-1 to 400-3). Although the above example describes only three papers, the number of papers is not limited to this. If the number of papers is N, the above operation can be performed through N neural networks corresponding to each paper.

[0098] The electronic device (200) can calculate evaluation scores (450-2, 450-3) for multiple laboratories (e.g., second laboratory and / or third laboratory) by substantially the same method as calculating the evaluation score (450-1) for the first laboratory.

[0099] The electronic device (200) can predict the university ranking (470) for multiple universities by summing the evaluation scores of multiple laboratories belonging to the same university (460-1). For example, the electronic device (200) can add the evaluation scores (450-1 to 450-3) of the first to third laboratories belonging to the first university (460-1). The electronic device (200) can also add the evaluation scores of laboratories belonging to each university for universities other than the first university (460-1). The electronic device (200) can predict the university ranking (470) for multiple universities by comparing the total sum of evaluation scores for each university.

[0100] The neural network (400-1 to 400-3) can be trained based on the difference between the predicted university ranking (470) and the ground truth (e.g., actual university ranking) (e.g., university rankings published by external organizations (e.g., QS (Quacquarelli Symonds)). This will be explained in detail with reference to FIGS. 6 and FIGS. 7.

[0101]

[0102] FIG. 5 is an example of a flowchart of a method for providing a laboratory evaluation service according to one embodiment.

[0103] Referring to FIG. 5, operations 510 and 530 may be performed sequentially, but are not limited thereto. For example, two or more operations may be performed in parallel. Operations 510 and 530 may be substantially identical to the operation of the electronic device (e.g., the electronic device (200) of FIG. 1) described with reference to FIG. 1 through 4. Accordingly, a detailed description is omitted.

[0104] In operation 510, the electronic device (200) can calculate an evaluation score for each of the multiple laboratories based on paper information of the multiple laboratories belonging to each of the multiple universities and value information for each academic society corresponding to the value of the multiple academic societies that publish the papers.

[0105] In operation 530, the electronic device (200) may provide a list showing the ranking of multiple laboratories based on the evaluation score of each of the multiple laboratories. The screen showing the ranking of multiple laboratories (e.g., the screen (300) of FIG. 3) has been described in detail with reference to FIG. 3, so redundant descriptions will be omitted.

[0106]

[0107] FIG. 6 is an example of a learning device according to one embodiment.

[0108] Referring to FIG. 6, the learning device (600) may include memory (610) and a processor (630). The learning device (600) may train a neural network. The neural network trained by the learning device (600) (e.g., neural networks of FIG. 4 (400-1 to 400-3)) may be stored in an electronic device (e.g., electronic device of FIG. 1 (200)). The electronic device (200) may provide laboratory evaluation services through the trained neural network.

[0109] Memory (610) may store a neural network or parameters of a neural network. Memory (610) may store instructions (e.g., programs) executable by the processor (630). For example, the instructions may include instructions for executing the operation of the processor (630) and / or the operation of each component of the processor (630).

[0110] The memory (610) can be implemented as a volatile memory device or a non-volatile memory device.

[0111] Volatile memory devices can be implemented as DRAM (dynamic random access memory), SRAM (static random access memory), T-RAM (thyristor RAM), Z-RAM (zero capacitor RAM), or TTRAM (Twin Transistor RAM).

[0112] Non-volatile memory devices can be implemented as EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, MRAM (Magnetic RAM), Spin-Transfer Torque (STT)-MRAM, Conductive Bridging RAM (CBRAM), FeRAM (Ferroelectric RAM), PRAM (Phase change RAM), Resistive RAM (RRAM), Nanotube RRAM, Polymer RAM (PoRAM), Nano Floating Gate Memory (NFGM), holographic memory, Molecular Electronic Memory Device, or Insulator Resistance Change Memory.

[0113] The processor (630) can process data stored in memory (610). The processor (630) can execute computer-readable code (e.g., software) stored in memory (610) and instructions triggered by the processor (630).

[0114] The processor (630) may be a data processing device implemented in hardware having a circuit having a physical structure for executing desired operations. For example, the desired operations may include code or instructions included in a program.

[0115] For example, a data processing device implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an Application-Specific Integrated Circuit (ASIC), and a Field Programmable Gate Array (FPGA).

[0116] The processor (630) can enable the learning device (600) to perform one or more operations by executing code and / or instructions stored in memory (610). Hereinafter, operations that can be performed by the learning device (600) will be described in detail with reference to FIGS. 6 and FIGS. 7.

[0117] The learning device (600) can calculate the evaluation scores of each of the multiple laboratories (e.g., evaluation scores (450-1 to 450-3) of FIG. 4) through a neural network (e.g., neural network (400-1 to 400-3) of FIG. 4) based on the paper information of multiple laboratories belonging to each of the multiple universities and the value information of each academic society that publishes the papers. As the method for calculating the evaluation scores of the laboratories has been explained in detail with reference to FIG. 1 to 5, redundant explanations will be omitted below.

[0118] The learning device (600) can calculate the scores of laboratories belonging to the same university for each of the multiple universities and predict the ranking of the multiple universities (e.g., the university ranking (470) of FIG. 4). This has been explained in detail with reference to FIG. 4, so redundant explanations will be omitted below.

[0119] The learning device (600) can train a neural network (e.g., the neural network in FIG. 4 (400-1 to 400-3)) by comparing the predicted rankings of multiple universities (e.g., the university rankings in FIG. 4 (470)) with actual values ​​(e.g., actual university rankings) (e.g., university rankings published by an external organization (e.g., QS (Quacquarelli Symonds))). The learning device (600) can adjust the parameters of the neural network through a backpropagation algorithm based on the difference between the predicted rankings of multiple universities and the actual values. The loss function used in the backpropagation algorithm will be described below.

[0120] The learning device (600) can train the neural network through an improved pairwise ranking loss function to improve the performance of the neural network (e.g., the neural network of FIG. 4 (400-1 to 400-3)) in the backpropagation algorithm. The ranking loss function can efficiently calculate the difference between the predicted score (e.g., the evaluation score by laboratory predicted through the neural network) and the actual value (e.g., the actual score) and / or the difference between the predicted scores through matrix operations, and can significantly improve accuracy by applying log-based weights. The main elements of the ranking loss function are as follows.

[0121] The learning device (600) can calculate the difference between the predicted rankings of multiple universities and the actual values ​​using matrix operations. The learning device (600) can efficiently compare the ranking differences of each pair and, in particular, can learn the relationship between the predicted rankings (and / or evaluation scores) of multiple universities more precisely.

[0122] The learning device (600) can compare the predicted rankings of multiple universities with the actual values ​​by applying logarithmic weights. The ranking loss function applies logarithmic weights when comparing rankings so that the importance of the relative ranking can be expressed as a weight. Logarithmic weights can be defined as shown in Equation 2 below.

[0123]

[0124] In mathematical formula 2, w represents the log weight, and i and j represent indices for laboratory rankings, respectively. For example, when applying the log weight to the 1st and 10th ranked laboratories, i can be 1 and j can be 10.

[0125] Applying logarithmic weights can make neural networks more sensitive to differences in rank. In particular, higher weights are assigned when higher-ranked laboratories are compared to lower-ranked ones, enabling the neural network to accurately predict the rankings among top laboratories.

[0126] The rank loss function may include a masking function based on an upper triangular matrix. The learning device (600) can efficiently perform rank comparisons and reduce unnecessary calculations by using only the upper triangular portion of the matrix. The learning device (600) can calculate the rank difference only when the condition i < j is satisfied. By performing operations only when the above condition is satisfied, the learning device (600) can reduce the training time of the neural network by minimizing unnecessary operations.

[0127] The learning device (600) can minimize the difference between the rank predicted by the neural network and the actual rank by applying a difference correction function (e.g., difference correction function) to the difference between each data pair (e.g., the rank and actual value of multiple universities being compared). The difference correction function may include a ReLU (rectified linear unit) function, an absolute value function, and / or a squared function. The learning device (600) can improve the accuracy of the rank comparison by applying two log weights to the difference of each data pair to which the difference correction function has been applied.

[0128]

[0129] FIG. 7 is an example of a flowchart of a learning method for a neural network according to one embodiment.

[0130] Referring to FIG. 7, operations 710 through 750 may be performed sequentially, but are not limited thereto. For example, two or more operations may be performed in parallel. Operations 710 through 750 may be substantially identical to the operations of the learning device (e.g., the learning device (600) of FIG. 6) described with reference to FIG. 6. Accordingly, a detailed description is omitted.

[0131] In operation 710, the learning device (600) can calculate the evaluation score of each of the multiple laboratories through a neural network (e.g., the neural network of FIG. 4 (400-1 to 400-3)) based on the paper information of the multiple laboratories belonging to each of the multiple universities and the value information of each academic society corresponding to the value of the multiple academic societies that publish the papers.

[0132] In operation 710, the learning device (600) can predict the ranking of multiple universities by adding the scores of laboratories belonging to the same university for each of the multiple universities.

[0133] In operation 710, the learning device (600) can train the neural network by comparing the predicted rankings of multiple universities with the actual values.

[0134]

[0135] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0136] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0137] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may store program instructions, data files, data structures, etc., either individually or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0138] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0139] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0140] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

1. Regarding the method of providing laboratory evaluation services, An operation to calculate an evaluation score for each of the multiple research laboratories based on paper information of multiple research laboratories affiliated with each of the multiple universities and value information by academic society corresponding to the value of the multiple academic societies publishing the papers; An operation to provide a list showing the ranking of the plurality of laboratories based on the evaluation score of each of the plurality of laboratories. A method including 2. In Paragraph 1, The paper information of the aforementioned multiple laboratories is, Information regarding the citation counts of papers published by each of the aforementioned multiple laboratories and the academic societies that published the said papers A method including 3. In Paragraph 2, The operation of calculating the evaluation score for each of the aforementioned multiple laboratories is, The operation of calculating a paper score for papers published by each of the plurality of laboratories based on the number of citations of the above papers; and The operation of calculating the evaluation score of each of the aforementioned multiple laboratories through a neural network based on the aforementioned paper scores and the aforementioned value information for each academic society. A method including 4. In Paragraph 3, The operation of calculating the above paper score is, An operation to adjust the citation count of the above papers; and The operation of calculating the paper scores for each of the aforementioned multiple laboratories by applying a decay rate based on the publication year of the aforementioned papers to the adjusted citation counts. A method including 5. In Paragraph 4, The operation of adjusting the citation counts of the aforementioned papers is, An operation of analyzing the purpose of citing the above papers and applying a weight according to the said purpose to the number of citations of the said papers. A method including 6. In Paragraph 4, The operation of adjusting the citation counts of the aforementioned papers is, An operation to calculate the internal laboratory citation rate for the said papers based on the relationship between the subject citing the said papers and the laboratory that published the said papers; and Action of adjusting the citation counts of the above papers based on the internal citation ratio of the above laboratory A method including 7. In Paragraph 3, The operation of calculating the score of each of the plurality of laboratories through the above neural network is, The operation of performing a linear transformation on the above paper score; and An operation to calculate the evaluation score of each of the multiple laboratories by performing an attention operation on the linearly transformed paper scores and the value information for each academic society. A method including 8. In Paragraph 1, The operation of providing a list showing the ranking of the aforementioned multiple laboratories is, The operation of classifying the above plurality of laboratories by research field; and The operation of providing a list showing the rankings of laboratories having the same research field, based on the evaluation scores of each of the laboratories having the same research field. A method including 9. In an electronic device providing laboratory evaluation services, processor; and Memory that stores instructions Includes, The above instructions are executed individually or collectively by the processor, causing the electronic device, Based on paper information of multiple research laboratories affiliated with each of multiple universities and value information by academic society corresponding to the value of multiple academic societies publishing papers, an evaluation score for each of the said multiple research laboratories is calculated, and An electronic device that provides a list showing the ranking of the plurality of laboratories based on the evaluation score of each of the plurality of laboratories.

10. In Paragraph 9, The paper information of the aforementioned multiple laboratories is, Information regarding the citation counts of papers published by each of the aforementioned multiple laboratories and the academic societies that published the said papers An electronic device including 11. In Paragraph 10, The above instructions are executed individually or collectively by the processor, causing the electronic device, Based on the citation counts of the above papers, paper scores for papers published by each of the above multiple laboratories are calculated, and An electronic device that calculates an evaluation score for each of the plurality of laboratories through a neural network based on the above paper score and the above academic society value information.

12. In Paragraph 11, The above instructions are executed individually or collectively by the processor, causing the electronic device, Adjust the citation counts of the above papers, and An electronic device for calculating the paper scores for each of the plurality of laboratories by applying a decay rate according to the publication year of the papers to the adjusted citation count.

13. In Paragraph 12, The above instructions are executed individually or collectively by the processor, causing the electronic device, An electronic device that analyzes the purpose of citing the above papers and applies a weight according to the purpose to the number of citations of the above papers.

14. In Paragraph 12, The above instructions are executed individually or collectively by the processor, causing the electronic device, Based on the relationship between the entities citing the aforementioned papers and the research labs that published the aforementioned papers, the internal lab citation rate for the aforementioned papers is calculated, and An electronic device that adjusts the number of citations of the papers based on the internal citation ratio of the above laboratory.

15. In Paragraph 11, The above instructions are executed individually or collectively by the processor, causing the electronic device, Perform a linear transformation on the above paper score, and An electronic device that calculates the score of each of the plurality of laboratories by performing an attention operation on linearly transformed paper scores and the value information for each academic society.

16. In Paragraph 9, The above instructions are executed individually or collectively by the processor, causing the electronic device, Classify the aforementioned multiple laboratories by research field, and An electronic device that provides a list showing the rankings of laboratories having the same research field, based on the evaluation scores of each of the laboratories having the same research field.

17. Regarding methods for training neural networks, An operation of calculating an evaluation score for each of the multiple laboratories through the neural network, based on paper information of multiple laboratories affiliated with each of the multiple universities and value information for each academic society corresponding to the value of the multiple academic societies publishing the papers; The operation of predicting the rankings of the plurality of universities by adding the evaluation scores of laboratories belonging to the same university for each of the plurality of universities; and The operation of training the neural network by comparing the rankings of the aforementioned multiple universities with the actual values. A method including 18. In Paragraph 17, The operation of training the above neural network is, The operation of adjusting the parameters of the neural network through a backpropagation algorithm based on the difference between the predicted rankings of multiple universities and the actual values. A method including 19. In Paragraph 17, The operation of adjusting the parameters of the above neural network is, The operation of comparing the predicted rankings of multiple universities with the actual values ​​by applying logarithmic weights. A method including 20. In Paragraph 17, The operation of adjusting the parameters of the above neural network is, An operation to apply hinge loss to the difference between the predicted rankings of multiple universities and the actual values, and to update the parameters so that the predicted rankings of multiple universities and the actual values ​​are minimized. A method including

Images