Electronic device for evaluating specificity of answer in employment document and method for evaluating specificity of answer in employment document using same

Abstract

An electronic device for evaluating the specificity of an answer in an employment document according to the present disclosure may comprise: a memory for storing at least one instruction; and at least one processor for executing the at least one instruction. The at least one processor may extract answer data from employment document data, analyze the answer data on the basis of at least four evaluation criteria, and calculate a specificity evaluation score of the answer data. The at least four evaluation criteria may include at least one of sentence length evaluation, keyword evaluation, morpheme evaluation, and personal experience evaluation.

Description

Electronic device for evaluating the specificity of answers in recruitment documents and method for evaluating the specificity of answers in recruitment documents using the same

[0001] The present disclosure relates to a technology for evaluating the specificity of a response, and more specifically, to an electronic device for evaluating the specificity of an applicant's response in a recruitment document and a method for evaluating the specificity of a response in a recruitment document using the same.

[0002] Recruitment documents are crucial materials used to comprehensively evaluate an applicant's capabilities, experience, and achievements during the hiring process. In particular, documents such as personal statements and career summaries serve as key sources of information for assessing who the applicant is and whether they possess the competencies suitable for the job. Traditionally, the analysis of these documents has been performed manually by recruiters and has been regarded as the most important initial step in evaluating an applicant's qualifications.

[0003] In recruitment documents, the specificity of an applicant's responses serves as a crucial factor in assessing their sincerity and competence. The more specific the response, the clearer one can understand the applicant's experience and tangible contributions, which helps determine if they are a suitable candidate for the job. Conversely, if the response is not specific, it may give the impression that the applicant failed to properly explain their abilities or lacks sincerity. Therefore, evaluating the specificity of responses is a critical process in the analysis of recruitment documents.

[0004] Traditional methods for reviewing application documents relied on recruiters manually analyzing each document one by one. However, this approach is not only subjective but also time-consuming and inefficient when reviewing a large volume of documents. In particular, the criteria for judging the specificity of an applicant's responses may be unclear or inconsistent. Accordingly, there is a need for an automated system that enables HR personnel to review application documents more quickly and consistently, and to objectively evaluate the specificity of responses.

[0005] One objective of the present disclosure is to provide an electronic device and a method for evaluating the specificity of responses in recruitment documents using the same, which automates the evaluation of the specificity of recruitment documents, thereby enabling consistent evaluation according to each evaluation criterion, increasing fairness among applicants, and increasing the efficiency of reviewing recruitment documents.

[0006] Another objective of the present disclosure is to provide an electronic device that effectively extracts and evaluates specific information from an applicant's responses to more clearly determine job suitability and analyze the applicant's experience and capabilities in detail, and a method for evaluating the specificity of responses in recruitment documents using the same.

[0007] However, the problems that this disclosure aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below.

[0008] An electronic device for evaluating the specificity of a response in a recruitment document according to the present disclosure for achieving the technical problem described above may include a memory storing at least one instruction and at least one processor executing said at least one instruction. The at least one processor may extract response data from recruitment document data, analyze said response data based on at least four evaluation criteria, and calculate a specificity evaluation score for said response data. The at least four evaluation criteria may include at least one of sentence length evaluation, keyword evaluation, morphological evaluation, and personal experience evaluation.

[0009] In one embodiment, the at least one processor performs binary classification or natural language processing on the answer data in sentence units and can analyze the answer data using at least one artificial intelligence model.

[0010] In one embodiment, the sentence length evaluation can evaluate the specificity of the answer based on the sentence length by dividing the answer data into sentence units to calculate the number of words in each sentence and comparing the sentence length of the answer data with a preset standard length range.

[0011] In one embodiment, the keyword evaluation can extract a keyword including at least one of an example keyword, a spatiotemporal keyword, a performance keyword, and a core keyword from the answer data, and evaluate the specificity of the answer according to the keyword based on the frequency of use of the keyword in the answer data.

[0012] In one embodiment, the at least one processor evaluates the answer specificity as higher as the frequency of at least one of the example keyword, the time / space keyword, and the performance keyword in the answer data is high, and evaluates the answer specificity as higher as the density of the core keyword in the answer data is high.

[0013] In one embodiment, the morpheme evaluation analyzes the answer data in morpheme units to calculate the ratio of adjectives and adverbs, and evaluates the specificity of the answer according to the morpheme based on whether the ratio of adjectives and adverbs in the answer data exceeds a preset standard ratio.

[0014] In one embodiment, the personal experience evaluation can evaluate the specificity of the answer based on personal experience by using a deep learning model to extract experience sentences from the answer data and evaluating at least one of reliability, initiative, and emotional expression from the experience sentences.

[0015] In one embodiment, the at least one processor can calculate the specificity evaluation score by applying weights to individual evaluation scores calculated from the sentence length evaluation, the keyword evaluation, the morphological evaluation, and the personal experience evaluation, performing score normalization by evaluation criteria, and summing the normalized scores by evaluation criteria.

[0016] In one embodiment, the at least one processor can scale the scores for each evaluation criterion by standardizing the scores for each evaluation criterion using [Formula 1] below, so that the average of the scores for each evaluation criterion is 0 and the standard deviation of the scores for each evaluation criterion is 1.

[0017] [Formula 1]

[0018] x' = (X - μ) / σ

[0019] (Here, x' is the normalized score, X is the raw score for each evaluation criterion, μ is the mean of the scores for each evaluation criterion, and σ is the standard deviation of the scores for each evaluation criterion)

[0020] Additionally, the method for evaluating the specificity of answers in recruitment documents according to the present disclosure may include the step of extracting answer data from recruitment document data, the step of analyzing the answer data based on at least four evaluation criteria, and the step of calculating a specificity evaluation score for the answer data. The at least four evaluation criteria may include at least one of sentence length evaluation, keyword evaluation, morphological evaluation, and personal experience evaluation.

[0021] In addition to this, a computer program stored on a computer-readable recording medium for implementing the present disclosure may be further provided.

[0022] In addition to this, a computer-readable recording medium for recording a computer program for implementing the present disclosure may be further provided.

[0023] According to the aforementioned means for solving the problem of the present disclosure, the electronic device of the present disclosure and the method for evaluating the specificity of answers in recruitment documents using the same automate the evaluation of the specificity of recruitment documents, thereby enabling consistent evaluation according to each evaluation criterion, which can increase fairness among applicants and increase the efficiency of reviewing recruitment documents.

[0024] In addition, the electronic device of the present disclosure and the method for evaluating the specificity of answers in recruitment documents using the same can effectively extract and evaluate specific information from the applicant's answers, thereby more clearly determining job suitability and analyzing the applicant's experience and capabilities in detail.

[0025] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below.

[0026] FIG. 1 is a drawing showing the block configuration of an electronic device of the present disclosure.

[0027] FIG. 2 is a conceptual diagram illustrating the operation of an electronic device of the present disclosure.

[0028] FIG. 3 is a flowchart illustrating the operation of the electronic device of the present disclosure.

[0029] FIG. 4 is a flowchart illustrating the operation of an electronic device of the present disclosure analyzing answer data based on sentence length evaluation.

[0030] FIG. 5 is a flowchart illustrating the operation of an electronic device of the present disclosure analyzing answer data based on keyword evaluation.

[0031] FIG. 6 is a flowchart illustrating the operation of an electronic device of the present disclosure analyzing answer data based on morphological evaluation.

[0032] FIG. 7 is a flowchart illustrating the operation of an electronic device of the present disclosure analyzing response data based on a personal experience evaluation.

[0033] FIG. 8 is a flowchart illustrating the operation of an electronic device of the present disclosure to calculate a specificity evaluation score of answer data.

[0034] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and general content in the art to which this disclosure pertains or content that overlaps between embodiments is omitted. The terms 'part, module, component, block' as used in the specification may be implemented in software or hardware, and depending on the embodiments, a plurality of 'parts, modules, components, blocks' may be implemented as a single component, or a single 'part, module, component, block' may include a plurality of components.

[0035] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are directly connected but also cases where they are indirectly connected, and indirect connections include connections made via a wireless communication network.

[0036] Furthermore, when it is stated that a part "includes" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0037] Throughout the specification, when it is stated that a component is located "on" another component, this includes not only cases where a component is in contact with another component, but also cases where another component exists between the two components.

[0038] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0039] Singular expressions include plural expressions unless there is an obvious exception in the context.

[0040] In each step, identification codes are used for convenience of explanation and do not describe the order of the steps; the steps may be performed differently from the specified order unless a specific order is clearly indicated in the context.

[0041] The operating principles and embodiments of the present disclosure will be described below with reference to the attached drawings.

[0042] In this specification, the term "device according to the present disclosure" includes all various devices capable of performing computational processing and providing results to a user. For example, the device according to the present disclosure may include all of a computer, a server device, and a portable terminal, or may be in the form of any one of these.

[0043] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0044] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0045] The above portable terminal may include, for example, all types of handheld-based wireless communication devices such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smartphones, etc., as well as wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMDs).

[0046] Functions related to artificial intelligence according to the present disclosure are operated through a processor and memory. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or artificial intelligence-dedicated processors such as NPUs. The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, if the one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0047] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a basic artificial intelligence model is trained using a number of training data by a learning algorithm, thereby creating predefined rules of operation or artificial intelligence models configured to perform desired characteristics (or objectives). Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0048] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through operations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model during the learning process is reduced or minimized. Artificial neural networks may include deep neural networks (DNNs), such as Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), or Deep Q-Networks, but are not limited to the examples mentioned above.

[0049] According to an exemplary embodiment of the present disclosure, a processor can implement artificial intelligence. Artificial intelligence refers to a machine learning method based on an artificial neural network that enables a machine to learn by mimicking human biological neurons. Methodologies of artificial intelligence can be classified according to the learning method into supervised learning, where input and output data are provided together as training data and the solution (output data) to the problem (input data) is predetermined; unsupervised learning, where only input data is provided without output data and the solution (output data) to the problem (input data) is not predetermined; and reinforcement learning, where a reward is given from an external environment whenever an action is taken from the current state, and learning proceeds in a direction that maximizes such reward. In addition, artificial intelligence methodologies can be classified according to the architecture, which is the structure of the learning model. The architectures of widely used deep learning technologies can be classified into Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Transformers, and Generative Adversarial Networks (GAN).

[0050] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0051] FIG. 1 is a drawing showing the block configuration of the electronic device (100) of the present disclosure, and FIG. 2 is a conceptual diagram (200) showing the operation of the electronic device (100) of the present disclosure.

[0052] Referring to FIG. 1, an electronic device (100) according to one embodiment may include a processor (120), a memory (110), and a communication unit (not shown). The configuration shown in FIG. 1 illustrates a configuration for describing an embodiment according to the present disclosure, and other configurations (e.g., a communication unit) for performing the functions of the electronic device may be added in addition to the illustrated configuration, and the illustrated configurations may be omitted.

[0053] A memory (110) according to one embodiment is a storage medium used by an electronic device (100) and can store data such as at least one instruction or setting information corresponding to at least one program. The program may include an operating system (OS) program and various application programs.

[0054] In one embodiment, the memory (110) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), RAM (random access memory, RAM), SRAM (static random access memory), ROM (read only memory, ROM), EEPROM (electrically erasable programmable ROM), PROM (programmable ROM), magnetic memory, a magnetic disk, and an optical disk.

[0055] In one embodiment, the communication unit may include a wireless communication unit (e.g., a cellular communication module, a short-range wireless communication module, or a GNSS (global navigation satellite system) communication module) or a wired communication unit. The communication unit may communicate with an external electronic device through a network (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a wide-area communication network such as a computer network (e.g., a LAN or WAN).

[0056] In one embodiment, the communication unit may support 5G networks and next-generation communication technologies, for example, new radio access technology. NR access technology may support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low-latency (ultra-reliable and low-latency communications (URLLC)). The communication unit may support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The communication unit may support various technologies for securing performance in the high-frequency band, for example, beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beamforming, or large-scale antenna.

[0057] A processor (120) according to one embodiment can, for example, execute software (e.g., a program) to control at least one other component (e.g., a hardware or software component) of an electronic device (100) connected to the processor (120) and can perform various data processing or operations.

[0058] In one embodiment, the processor (120) may store commands or data received from another component (e.g., a communication unit) in volatile memory as at least part of data processing or computation, process the commands or data stored in volatile memory, and store the result data in non-volatile memory.

[0059] In one embodiment, the processor (120) may include a main processor (e.g., a central processing unit or an application processor) or an auxiliary processor that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), or a communication processor). For example, the processor (120) may include at least one of a CPU (Central Processing Unit), an AP (Application Processor), or a microprocessor.

[0060] A processor (120) according to one embodiment can perform the operation of an electronic device (100) described below through drawings. For example, as shown in FIG. 2, the processor (120) can quantify the specificity of an applicant's answer by automatically evaluating the answer to a recruitment document according to various evaluation criteria.

[0061] Specifically, the processor (120) can extract answer data from recruitment document data, analyze the answer data based on at least four evaluation criteria, and calculate a specificity evaluation score for the answer data. Here, the at least four evaluation criteria may include at least one of sentence length evaluation, keyword evaluation, morphological evaluation, and personal experience evaluation.

[0062] In one embodiment, the processor (120) performs binary classification or natural language processing on the answer data in sentence units and can analyze the answer data using at least one artificial intelligence model.

[0063] In one embodiment, the sentence length evaluation can evaluate the specificity of the answer based on the sentence length by dividing the answer data into sentence units to calculate the number of words in each sentence and comparing the sentence length of the answer data with a preset standard length range.

[0064] In one embodiment, the keyword evaluation can extract a keyword including at least one of an example keyword, a spatiotemporal keyword, a performance keyword, and a core keyword from the answer data, and evaluate the specificity of the answer according to the keyword based on the frequency of use of the keyword in the answer data.

[0065] In one embodiment, the processor (120) evaluates the answer specificity as higher as the frequency of at least one of the example keyword, the time / space keyword, and the performance keyword in the answer data is high, and evaluates the answer specificity as higher as the density of the core keyword in the answer data is high.

[0066] In one embodiment, the morpheme evaluation analyzes the answer data in morpheme units to calculate the ratio of adjectives and adverbs, and evaluates the specificity of the answer according to the morpheme based on whether the ratio of adjectives and adverbs in the answer data exceeds a preset standard ratio.

[0067] In one embodiment, the personal experience evaluation can evaluate the specificity of the answer based on personal experience by using a deep learning model to extract experience sentences from the answer data and evaluating at least one of reliability, initiative, and emotional expression from the experience sentences.

[0068] In one embodiment, the processor (120) can calculate the specificity evaluation score by applying weights to individual evaluation scores calculated from the sentence length evaluation, the keyword evaluation, the morpheme evaluation, and the personal experience evaluation, performing score normalization by evaluation criteria, and summing the normalized scores by evaluation criteria.

[0069] In one embodiment, the processor (120) can scale the scores for each evaluation criterion by standardizing them using the following [Formula 1] so that the average of the scores for each evaluation criterion is 0 and the standard deviation of the scores for each evaluation criterion is 1.

[0070] [Formula 1]

[0071] x' = (X - μ) / σ

[0072] Here, x' is the normalized score, X is the raw score for each evaluation criterion, μ is the average value of the scores for each evaluation criterion, and σ is the standard deviation value of the scores for each evaluation criterion.

[0073] In this way, the processor (120) can evaluate the recruitment documents objectively, fairly, and efficiently by evaluating the specificity of the answers in the recruitment documents by combining elements such as sentence length, keywords, morphemes, and personal experience.

[0074] FIG. 3 is a flowchart showing the operation of the electronic device (100) of the present disclosure.

[0075] Referring to FIG. 3, the electronic device (100) of the present disclosure can automate the evaluation of the specificity of recruitment documents, thereby enabling consistent evaluation according to each evaluation criterion, which can increase fairness among applicants and increase the efficiency of reviewing recruitment documents. For example, the electronic device (100) can perform binary classification or natural language processing on the sentence-unit answer data included in the recruitment documents and analyze the answer data using at least one artificial intelligence model.

[0076] Here, the artificial intelligence model used by the electronic device (100) to analyze the answer data included in the recruitment documents may be a single artificial intelligence model or may be implemented as multiple artificial intelligence models. The artificial intelligence model may be composed of a neural network (or artificial neural network) and may include statistical learning algorithms that mimic biological neurons in machine learning and cognitive science. A neural network may refer to a model that possesses problem-solving capabilities by changing the strength of synaptic connections through learning, where artificial neurons (nodes) form a network through the connection of synapses. The neurons of the neural network may include a combination of weights or biases. The neural network may include one or more layers composed of one or more neurons or nodes. For example, the device may include an input layer, a hidden layer, and an output layer. The neural network constituting the device can infer a result (output) to be predicted from an arbitrary input by changing the weights of the neurons through learning.

[0077] At least one processor included in the electronic device (100) can generate a neural network, train or learn a neural network, perform operations based on received input data, generate an information signal based on the results of the operation, or retrain the neural network. The neural network models may include, but are not limited to, various types of models such as Convolutional Neural Network (CNN), Region with Convolutional Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based Deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restricted Boltzmann Machine (RBM), Fully Convolutional Network, Long Short-Term Memory Network (LSTM), and Classification Network, such as GoogleNet, AlexNet, and VGG Network. The processor may include one or more processors for performing operations according to the neural network models. For example, neural networks can include deep neural networks.

[0078] Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto) Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial Network), LSM (Liquid State Machine), ELM (Extreme Learning Machine), ESN (Echo A person skilled in the art will understand that any neural network may be included, but is not limited to, State Network, Deep Residual Network, Differential Neural Computer, Neural Turing Machine, Capsule Network, Kohonen Network, and Attention Network.

[0079] According to exemplary embodiments of the present disclosure, the processor comprises a Convolutional Neural Network (CNN) such as GoogleNet, AlexNet, VGG Network, Region with Convolutional Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based Deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restricted Boltzmann Machine (RBM), Fully Convolutional Network, Long Short-Term Memory (LSTM) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT, SP-BERT, MRC / QA, Text Analysis, Dialog System, GPT-3, GPT-4 for Natural Language Processing, Visual Analytics, Visual Understanding, Video Synthesis for Vision Processing, Anomaly Detection, Prediction, Time-Series Forecasting, Optimization, Recommendation for ResNet Data Intelligence, Various artificial intelligence structures and algorithms, such as data creation, may be used, but are not limited thereto.

[0080] Specifically, the electronic device (100) can extract answer data from recruitment document data (operation 310), analyze the answer data based on at least four evaluation criteria (operation 320), and calculate a specificity evaluation score for the answer data (operation 330).

[0081] According to one example, in operation 310, the electronic device (100) can extract answer data from recruitment document data. For example, the electronic device (100) can extract answer data from recruitment document data. To do this, the electronic device (100) can apply various text analysis techniques to effectively identify and extract answer sentences within the recruitment document.

[0082] The electronic device (100) can first divide the text into sentence units using a sentence segmentation algorithm to understand the overall structure of the recruitment document data. Based on the data divided into sentence units, the electronic device (100) can analyze the nature of each sentence to specifically extract the part corresponding to the answer of the recruitment document. The electronic device (100) can separate the sentences identified as responses to questions in the recruitment document by analyzing the keywords and context of each sentence through a natural language processing (NLP) algorithm.

[0083] The electronic device (100) can filter sentences related to answers from recruitment document data using a list of predefined keywords and subject words. For example, since sentences containing keywords such as "experience," "project," and "achievement" are likely to be considered answers, the electronic device (100) can prioritize extracting sentences containing these keywords.

[0084] Additionally, the electronic device (100) can evaluate the relevance of answers on a sentence-by-sentence basis by utilizing a machine learning-based classification model. For example, the electronic device (100) can predict whether each sentence of a recruitment document corresponds to an answer through a trained classification model and extract only the sentences that correspond to an answer. Based on the results of this classification model, the electronic device (100) can accurately identify sentences containing the applicant's specific response.

[0085] In one embodiment, the electronic device (100) can analyze the semantic similarity of sentences through a deep learning model. The electronic device (100) can extract sentences that have a direct relationship with the question by utilizing a deep learning model to calculate the semantic agreement between the question content of the recruitment document and the answer sentence.

[0086] According to one example, in operation 320, the electronic device (100) can analyze the answer data based on at least four evaluation criteria. For example, the electronic device (100) may set at least four evaluation criteria to evaluate the specificity of the answer data and analyze the answer data according to each criterion. Here, the at least four evaluation criteria may include at least one of sentence length evaluation, keyword evaluation, morphological evaluation, and personal experience evaluation. Hereinafter, operation 320 of the electronic device (100) will be described in detail with reference to FIGS. 4 to 7.

[0087] FIG. 4 is a flowchart illustrating the operation of an electronic device (100) of the present disclosure analyzing answer data based on sentence length evaluation.

[0088] Referring to FIG. 4, the electronic device (100) can perform a sentence length evaluation as the first evaluation criterion. The sentence length evaluation can be performed by dividing each sentence in the answer data into units, calculating the number of words in each sentence, and comparing it with a preset length range. The electronic device (100) can evaluate the specificity of the answer based on the length of the sentence by assigning a high specificity score if each sentence falls within the preset length range.

[0089] Specifically, the electronic device (100) can divide the answer data into sentences (operation 410), calculate the number of words in the answer data (operation 420), determine the length range of the answer data (operation 430), and calculate a first score based on the sentence length evaluation (operation 440).

[0090] The electronic device (100) can calculate the number of words in each sentence for the answer data divided into sentence units and compare it with a preset length range. For example, if each sentence is between 10 and 50 words, the electronic device (100) may consider the sentence as a highly specific answer. The electronic device (100) may assign a specificity score based on these criteria, and may assign a low score if each sentence exceeds the standard length or is below the standard.

[0091] The electronic device (100) may lower the specificity score by determining that the sentence lacks information when the sentence length is short. For example, if the sentence is 5 words or less, such as "I did planning work in the team," it may be evaluated as having low specificity because the amount of information is small. On the other hand, if the sentence is 15 words or more, such as "I took charge of project planning in the team and performed overall tasks ranging from schedule planning to budget management," the electronic device (100) may evaluate that the answer contains more specific and rich information.

[0092] Additionally, the electronic device (100) may determine that the specificity may be reduced if the sentence becomes excessively long, and may reduce the score for sentences that exceed a certain number of words. For example, the electronic device (100) may determine that a sentence exceeding 50 words may contain an excessive number of listed information or may include unnecessary content, and may lower the specificity score.

[0093] The electronic device (100) can calculate a first score for each sentence based on these word count criteria and sum the first scores of each sentence to reflect in the specificity evaluation score of the entire answer data.

[0094] FIG. 5 is a flowchart illustrating the operation of an electronic device (100) of the present disclosure analyzing answer data based on keyword evaluation.

[0095] Referring to FIG. 5, the electronic device (100) can perform a keyword evaluation as a second evaluation criterion. The keyword evaluation can extract specific keywords, such as example keywords, time / space keywords, performance keywords, and core keywords, from the answer data and evaluate the specificity of the answer based on the frequency of use of each keyword. The electronic device (100) can assign a higher score as the frequency of use of example keywords, time / space keywords, and performance keywords increases, and can evaluate the specificity score more highly as the density of core keywords increases.

[0096] Specifically, the electronic device (100) can divide the answer data into sentences (operation 510), extract keywords from the answer data (operation 520), calculate the frequency of the extracted keywords (operation 530), and calculate a second score based on the keyword evaluation (operation 540).

[0097] The electronic device (100) can extract at least one keyword and a sentence including said at least one keyword from the answer data using a keyword extraction module.

[0098] The keyword extraction module may include technology for extracting specific keywords from response data. The keyword extraction module can extract keywords using two main methods: the dictionary method and the deep learning method. For example, the dictionary method may be a method of analyzing text data using a predefined list of keywords. For example, the deep learning method may be a method of understanding context and identifying keywords using a trained model.

[0099] In one embodiment, the keyword extraction module can perform preprocessing on the input answer data. For example, the keyword extraction module can perform at least one of sentence splitting, tokenization, and normalization on the answer data, and accurately extract keywords based thereon.

[0100] The electronic device (100) can first divide the answer data into sentence units for keyword evaluation (operation 510) and extract specific keywords such as example keywords, time / space keywords, performance keywords, and core keywords from each sentence. For example, example keywords may include "for example," "for instance," and "for example," and time / space keywords may include "time," "during," and "period." Performance keywords include "achievement," "improvement," and "completion," and core keywords may include specific information such as people, places, organizations, and dates through named entity recognition.

[0101] The electronic device (100) can calculate the usage frequency of each extracted keyword and assign a higher specificity score as the usage frequency of a specific keyword increases.

[0102] In one embodiment, the electronic device (100) can evaluate the specificity of the answer data by analyzing the frequency of example keywords. Example keywords are words used to introduce specific examples or explanations, and are frequently used in the answer data when an applicant wishes to explain their experience or examples more clearly. The electronic device (100) can set a specific list of example keywords to evaluate the specificity of the answer through the example keywords, and can calculate the frequency of how often the keywords in the list appear in the answer.

[0103] For example, the electronic device (100) can set example keywords such as "for example," "for example," "for instance," "as an example," "as an example," "for instance." Whenever these example keywords appear in the answer data, additional points can be added to the specificity evaluation score.

[0104] The electronic device (100) can analyze each sentence to identify sentences containing example keywords and assign a score based on the frequency of the keywords. For example, in the sentence "For example, in a previous project, as a team leader, I planned all the schedules and put them into action," since "for example" is included, the electronic device (100) can evaluate that this sentence contains specific examples and assign a high score.

[0105] The electronic device (100) may assign a higher specificity evaluation score when example keywords appear multiple times. For example, if a sentence such as "For example, in a previous project, I collaborated with team members to solve an unexpected problem" appears continuously across multiple sentences, the electronic device (100) may determine that the applicant has increased specificity by mentioning various examples.

[0106] In one embodiment, the electronic device (100) can evaluate the specificity of the answer data by analyzing the frequency of spatiotemporal keywords. Spatiotemporal keywords are words that indicate a specific temporal or spatial context and are frequently used when an applicant wishes to specifically describe their experience by mentioning a specific time or place in an answer. The electronic device (100) can analyze sentences containing spatiotemporal keywords and add points to the specificity evaluation score if the answer includes temporal or spatial information.

[0107] The electronic device (100) can set a list of words such as "time," "during," "period," "at that time," "place," "location," and "middle" as spatiotemporal keywords. Whenever these keywords appear in the answer data, the frequency of the corresponding keyword is calculated, and an additional specificity score can be assigned to the answer containing spatiotemporal information.

[0108] For example, in the sentence “During my college years, I developed problem-solving skills by participating in various team projects,” the spatiotemporal keyword “college years” is included, so the electronic device (100) can determine that the sentence provides a specific temporal context. The electronic device (100) can evaluate the answer containing this temporal context as specific and give it a high score.

[0109] The electronic device (100) may also award additional points if the sentence “In a project that lasted for one year, as a team leader, I collaborated with all team members to achieve the goal” contains the spatiotemporal keyword “for one year”, the electronic device (100) determines that the answer includes a specific period of time.

[0110] The electronic device (100) may assign a higher specificity score when time / space keywords appear multiple times. For example, if a sentence such as "I participated in an internship last summer and then focused on a research project in the fall" continuously mentions a specific time or period, the electronic device (100) may assign an additional specificity score by evaluating that the applicant described their activities in chronological order.

[0111] In one embodiment, the electronic device (100) can evaluate the specificity of the answer data by analyzing the frequency of performance keywords. Performance keywords are words used by the applicant to emphasize the results achieved through their activities or experiences, and through the performance keywords, the electronic device (100) can determine whether the applicant has presented specific achievements. The electronic device (100) can analyze sentences containing performance keywords in the answer data and add a specificity evaluation score to answers in which achievements are specifically stated.

[0112] The electronic device (100) can set a list of words such as "achieved," "completed," "improved," "increased," "performance," and "improved" as performance keywords. The electronic device (100) can identify sentences containing such performance keywords in the response data and assign a score based on the frequency of the keywords.

[0113] For example, in the sentence “I successfully completed the team project and received a high evaluation,” since “completed” and “evaluation” are used as keywords indicating performance, the electronic device (100) can determine that this sentence contains specific performance. The electronic device (100) can assign a high specificity score to the sentence containing such performance.

[0114] The electronic device (100) can assign a higher specificity score when performance keywords are used multiple times. For example, in the sentence “I improved existing sales by 20% and increased the productivity of team members by 15% during the project period,” the performance keywords “improved” and “increased” are each included, so the electronic device (100) can assign additional points by determining that this answer specifically describes various performances.

[0115] Additionally, the electronic device (100) may give a higher score if specific figures or comparison targets of the performance are mentioned in sentences containing performance keywords. For example, a sentence describing performance with specific figures, such as "achieved a performance of increasing sales by 30% compared to last year," may receive an additional specificity evaluation score from the electronic device (100).

[0116] The electronic device (100) can calculate additional scores by evaluating the density of key keywords included in the answer data. The electronic device (100) can identify specific entity information, such as people (PS), places (LC), organizations (OG), and dates (DT), included in the answer data by using a Named Entity Recognition (NER) model. For example, in the sentence "I participated in a research project at the AI ​​research institute of Company A," key keywords such as "Company A," "AI research institute," and "project" are included, so the electronic device (100) can determine that this sentence contains a lot of meaningful information and assign a high specificity score.

[0117] The electronic device (100) can calculate a specificity evaluation score by applying a preset weight to each object information. For example, organization information (OG) and person information (PS) mentioned by the applicant are considered important factors indicating career and cooperative relationships, so they may be assigned a relatively high weight (e.g., 0.9, 0.8). On the other hand, place information (LC) or date information (DT) are relatively less important, so they may be assigned a low weight (e.g., 0.5).

[0118] The electronic device (100) can evaluate specificity based on whether keywords in the answer data are used by reflecting the second score calculated in this way into the specificity evaluation score.

[0119] FIG. 6 is a flowchart illustrating the operation of the electronic device (100) of the present disclosure analyzing answer data based on morphological evaluation.

[0120] Referring to FIG. 6, the electronic device (100) can perform a morphological evaluation as a third evaluation criterion. The morphological evaluation can evaluate the specificity of a sentence by analyzing the usage ratio of adjectives and adverbs in the answer data. The electronic device (100) can calculate the ratio of adjectives and adverbs extracted from each sentence and, if the ratio exceeds a preset standard, assign a high specificity score.

[0121] Specifically, the electronic device (100) can set morpheme tags (610), divide the answer data into sentence units (operation 620), calculate the ratio of morphemes in the answer data (operation 630), and calculate a third score based on the morpheme evaluation (operation 640).

[0122] The electronic device (100) can first set morpheme tags to be analyzed in the answer data for morpheme evaluation (operation 610). The electronic device (100) can pre-define morpheme tags for adjectives and adverbs, so that adjectives are set as VA, JKG, VA+ETM, MM tags, and adverbs are set as MAC, MAG tags. Based on this setting, the electronic device (100) can divide the answer data into sentences (operation 620) and identify and extract morphemes with the set tags in each sentence.

[0123] The electronic device (100) can determine the number of adjectives and adverbs extracted from each sentence and calculate the proportion of adjectives and adverbs among all morphemes (operation 630). For example, in the sentence “I solved the problem very efficiently in the team project,” “very” (adverb) and “efficient” (adjective) can be extracted, and the electronic device (100) can evaluate the specificity of this sentence by calculating the proportion of adjectives and adverbs among all morphemes.

[0124] The electronic device (100) can assign a high specificity score to a sentence if the ratio of the calculated adjectives and adverbs exceeds a preset standard. For example, the electronic device (100) can assign a high score to a sentence by evaluating that the sentence contains specific expressions if the ratio of the adjectives and adverbs is 20% or more. In the sentence "I set new goals every year and actively improved my performance," the words "active" (adverb) and "new" (adjective) are included, so the electronic device (100) can evaluate that this sentence has rich specificity.

[0125] The electronic device (100) can calculate a third score based on the ratio of extracted adjectives and adverbs and reflect this score in the final specificity evaluation score. Through morphological evaluation, the electronic device (100) can determine whether the use of various morphemes in the answer data contributes to increasing the specificity of the answer.

[0126] FIG. 7 is a flowchart illustrating the operation of the electronic device (100) of the present disclosure analyzing response data based on personal experience evaluation.

[0127] Referring to FIG. 7, the electronic device (100) can perform a personal experience evaluation as a fourth evaluation criterion. The personal experience evaluation can be performed by using a deep learning model to extract sentences revealing personal experience from response data and analyzing elements such as reliability, initiative, and emotional expression in the experience sentences. The electronic device (100) can evaluate the specificity of the personal experience by assigning a high score if the experience sentences include an initiative or positive emotional expression.

[0128] Specifically, the electronic device (100) can divide the answer data into sentences (operation 710), extract experience sentences from the answer data (operation 720), evaluate at least one of reliability, initiative, and emotional expression from the experience sentences (operation 730), and calculate a fourth score based on the personal experience evaluation (operation 740).

[0129] The electronic device (100) utilizes a trained deep learning model to accurately distinguish whether or not there is experience, and this model can predict whether each sentence is based on the applicant's actual experience. The electronic device (100) can identify personal experience by labeling each sentence as 1 if the experience is evident and as 0 if it is not.

[0130] The electronic device (100) can collect various training data for this evaluation and train a model by dividing the collected self-introduction data into sentence units and assigning a label of 0 or 1. For example, a sentence such as "I strengthened my will to solve problems through conversations with members" can be labeled as 1 because it contains content that the applicant actually experienced. On the other hand, a sentence such as "I am conducting intensive research in the fields of AI, Big data, and cloud" can be labeled as 0 because it does not clearly reveal an experience directly related to the applicant.

[0131] After the model is trained, the electronic device (100) can determine whether there is experience by performing binary classification on the input answer data in sentence units. For example, if the subject of a sentence starts with "I," "myself," "myself," etc., the model can recognize the sentence as a sentence revealing the applicant's experience and classify it as 1. Even in the case of a sentence where the subject is omitted, the electronic device (100) can analyze the context to infer whether the potential subject is the applicant, and if it is determined to be the applicant's experience, output 1.

[0132] The electronic device (100) can calculate a fourth score based on the personal experience evaluation by further evaluating elements such as reliability, initiative, and emotional expression for the identified personal experience sentences. For example, a high score can be given to a sentence that clearly reveals an initiative role, such as, "As a team leader, I led the project and strengthened communication with the members." The electronic device (100) can evaluate the specificity and reliability of the experience in this way and reflect them in the final specificity evaluation score.

[0133] According to one example, in operation 330, the electronic device (100) can calculate a specificity evaluation score of the answer data. Hereinafter, operation 330 of the electronic device (100) will be described in detail with reference to FIG. 8.

[0134] FIG. 8 is a flowchart illustrating the operation of an electronic device (100) of the present disclosure calculating a specificity evaluation score of answer data.

[0135] Referring to FIG. 8, the electronic device can apply a first weight to a fourth weight to each of the first to fourth scores (operation 810), perform score normalization for each evaluation criterion (operation 820), and calculate a specificity evaluation score (operation 830) by summing the normalized scores for each evaluation criterion.

[0136] To calculate the specificity evaluation score of the answer data, the electronic device (100) may first apply weights to the first to fourth scores derived from each of the sentence length evaluation, keyword evaluation, morphological evaluation, and personal experience evaluation. For example, the electronic device (100) may set the importance of the sentence length evaluation to a relatively high level, the weights for the keyword evaluation and morphological evaluation to a medium level, and the weights for the personal experience evaluation to a relatively low level. In this way, the electronic device (100) can more accurately evaluate the specificity of the answer by reflecting the importance of each evaluation criterion.

[0137] After weights are applied to each evaluation criterion, the electronic device (100) can perform a standardization process to convert all evaluation criterion scores to the same scale, since the scores for each evaluation criterion may have different scales.

[0138] The electronic device (100) may use the following [Formula 1] for the above normalization.

[0139] [Formula 1]

[0140] x' = (X - μ) / σ

[0141] Here, x' is the score after normalization, X is the raw score for each evaluation criterion, μ is the average value of the score for each evaluation criterion, and σ is the standard deviation value of the score for each evaluation criterion. The electronic device (100) can scale each score so that the mean is 0 and the standard deviation is 1 by applying [Equation 1].

[0142] The electronic device (100) can calculate a final specificity evaluation score by summing the scores for each normalized evaluation criterion. For example, the electronic device (100) can calculate a final specificity evaluation score representing the overall specificity of the answer data by summing all the normalized first, second, third, and fourth scores. The final specificity evaluation score can be used as a value to quantify and express the specificity of the answer in the recruitment document data. For example, the electronic device (100) can quantitatively evaluate the specificity of the answer written by the applicant based on this final score.

[0143] As such, the electronic device of the present disclosure and the method for evaluating the specificity of answers in recruitment documents using the same automate the evaluation of the specificity of recruitment documents, thereby enabling consistent evaluation according to each evaluation criterion, which can enhance fairness among applicants and increase the efficiency of reviewing recruitment documents.

[0144] In addition, the electronic device of the present disclosure and the method for evaluating the specificity of answers in recruitment documents using the same can effectively extract and evaluate specific information from the applicant's answers, thereby more clearly determining job suitability and analyzing the applicant's experience and capabilities in detail.

[0145] However, as this has been explained above, a redundant explanation thereof will be omitted.

[0146] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0147] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0148] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents. Therefore, other implementations, other embodiments, and equivalents to the claims below are also within the scope of the claims.

Claims

1. In an electronic device for evaluating the specificity of answers in employment documents, Memory for storing at least one instruction; and It includes at least one processor that executes the above at least one instruction, and The above-mentioned at least one processor is, Extract response data from recruitment document data, and Analyze the above response data based on at least four evaluation criteria, and Calculate the specificity evaluation score of the above answer data, and The above at least four evaluation criteria are, including at least one of sentence length evaluation, keyword evaluation, morphological evaluation, and personal experience evaluation, Electronic device.

2. In Paragraph 1, The above-mentioned at least one processor is, Perform binary classification or natural language processing operations on the above answer data at the sentence level, and Analyzing the above answer data using at least one artificial intelligence model, Electronic device.

3. In Paragraph 1, The above sentence length evaluation is, Divide the above answer data into sentence units to calculate the number of words in each sentence, and By comparing the pre-set standard length range with the sentence length of the above answer data, evaluating the specificity of the answer based on sentence length, Electronic device.

4. In Paragraph 1, The above keyword evaluation is, Extract keywords from the above answer data that include at least one of example keywords, spatiotemporal keywords, performance keywords, and core keywords, and Evaluating the specificity of answers according to keywords based on the usage frequency of the above keywords in the above answer data, Electronic device.

5. In Paragraph 4, The above-mentioned at least one processor is, In the above answer data, the higher the frequency of at least one of the above example keyword, the above time / space keyword, and the above performance keyword, the higher the answer specificity is evaluated, and In the above answer data, the higher the density of the above core keywords, the higher the evaluation of the above answer specificity, Electronic device.

6. In Paragraph 1, The above morpheme evaluation is, Analyze the above answer data at the morpheme level to calculate the ratio of adjectives and adverbs, and In the above answer data, evaluating the specificity of answers according to morphemes based on whether the above ratio of adjectives and adverbs exceeds a preset standard ratio, Electronic device.

7. In Paragraph 1, The above personal experience evaluation is, Using a deep learning model, experience sentences are extracted from the above answer data, and Evaluating the specificity of the answer based on personal experience by evaluating at least one of reliability, initiative, and emotional expression from the above experience sentence, Electronic device.

8. In Paragraph 1, The above-mentioned at least one processor is, Weights are applied to the individual evaluation scores calculated from the above sentence length evaluation, the above keyword evaluation, the above morphological evaluation, and the above personal experience evaluation, and Perform score normalization based on evaluation criteria, and Calculating the specificity evaluation score by summing the normalized scores for each of the above evaluation criteria, Electronic device.

9. In Paragraph 8, The above-mentioned at least one processor is, Scaling the scores for each evaluation criterion by standardizing them using [Formula 1] below so that the mean of the scores for each evaluation criterion is 0 and the standard deviation of the scores for each evaluation criterion is 1, [Formula 1] x' = (X - μ) / σ (Here, x' is the normalized score, X is the raw score for each evaluation criterion, μ is the mean of the scores for each evaluation criterion, and σ is the standard deviation of the scores for each evaluation criterion) Electronic device.

10. A method performed by a processor of a device, Step of extracting response data from recruitment document data; A step of analyzing the above-mentioned answer data based on at least four evaluation criteria; and, The method includes the step of calculating a specificity evaluation score for the above-mentioned answer data; The above at least four evaluation criteria are, including at least one of sentence length evaluation, keyword evaluation, morphological evaluation, and personal experience evaluation, Method for evaluating the specificity of answers in recruitment documents.

Images