Electronic apparatus and method for performing indexing based on key map of index database

Abstract

An electronic device for performing a keymap-based index for an index database according to the present disclosure comprises: a memory storing at least one process for performing a keymap generation operation mapping the presence or absence of any index key within the index database; and at least one processor for performing said operation according to said process; wherein the at least one processor may be configured to generate a hash value for each of a plurality of index keys stored in said index database, calculate a bit index within a predetermined hash table size based on said hash value, and set a bit value corresponding to said bit index to generate a keymap for said index key.

Description

Technology Field

[0001] The present disclosure relates to an electronic device and method for performing plagiarism checks on natural language-based document data, and more specifically, to an electronic device and method for performing a keymap-based indexing of an index database to accelerate search speed through a keymap using a bit index per index database. Background Technology

[0002] With the rise of document-based information services and the increasing use of generative artificial intelligence, there is a growing need to determine the similarity between various text materials and verify for plagiarism. In particular, rapid plagiarism checks are crucial, as educational institutions and large-scale recruitment processes often require the analysis of massive amounts of text data.

[0003] In conventional technology, a method of comparing a comparison key extracted from a text document with the index keys of other documents stored in an index database is widely used; however, in actual operating environments, there is a problem in that most comparison keys do not exist in the index database.

[0004] For example, sentences within text data consist of units of at least 5 to 6 clauses, and more than 97% of the 1 million comparison keys extracted in units of 5 to 6 clauses do not exist in any index database. The process of performing inverse index searches for these comparison keys consumes unnecessary CPU computation and disk I / O, preventing sufficient resource allocation to the critical 3% of index databases. This repetitive and unnecessary searching causes delays in the overall plagiarism check process and increases system load when performing large-scale plagiarism checks.

[0005] Therefore, the need has arisen for an invention that minimizes the load on the index database and enables efficient use in large-scale search environments, so as to quickly determine whether an index key exists within the index database. Prior art literature

[0006] Republic of Korea Registered Publication 1666758 B1(2016.10.10) The problem to be solved

[0007] The purpose of the embodiments disclosed in this disclosure is to provide an electronic device and method for performing a key map-based indexing of an index database, wherein the device performs an index key search after a preliminary search for the possibility of existence of an index key within the index database based on a key map in which a bitmap is constructed based on hash values ​​for index keys stored in the index database.

[0008] The problems that this disclosure aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below. means of solving the problem

[0009] An electronic device for performing a key map-based index for an index database according to the present disclosure, for achieving the technical problem described above, comprises: a memory storing at least one process for performing a key map generation operation mapping the presence or absence of any index key within the index database; and at least one processor for performing said operation according to said process; wherein the at least one processor may be configured to generate a hash value for each of a plurality of index keys stored in said index database, calculate a bit index within a predetermined hash table size based on said hash value, and set a bit value corresponding to said bit index to generate a key map for said index key.

[0010] A method for performing a keymap-based index for an index database, performed by a processor of an apparatus according to the present disclosure to achieve the above-described technical problem, wherein the method may include: a step of generating a hash value for each of a plurality of index keys stored in the index database; a step of calculating a bit index within a predetermined hash table size based on the hash value; and a step of creating a keymap for the index keys by setting a bit value corresponding to the calculated bit index.

[0011] In addition to this, a computer program stored on a computer-readable recording medium for implementing the present disclosure may be further provided.

[0012] In addition to this, a computer-readable recording medium for recording a computer program for implementing the present disclosure may be further provided. Effects of the invention

[0013] According to the aforementioned means for solving the problem of the present disclosure, when a comparison key extracted from an input document does not exist in the index database, it is possible to immediately determine whether to perform a search on the index database by merely querying the bit referenced by the comparison key in the key map, thereby significantly reducing unnecessary reverse index searches and saving costs and time, and providing the effect of improving plagiarism detection performance.

[0014] Furthermore, according to the aforementioned means for solving the problem of the present disclosure, by generating a key map through a unique indexing logic for a static (read-only) index database that does not require data deletion or updating, the process of pre-querying for the existence of an index key is simplified, and an efficient resource saving effect is provided by immediately performing an actual document similarity determination through an external index database based on the query result.

[0015] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0016] FIG. 1 is a block diagram briefly illustrating the configuration of an electronic device for performing a keymap-based indexing for an index database according to the present disclosure. FIG. 2 is a block diagram illustrating the structure between an index database and a key map of an electronic device that performs a key map-based indexing for an index database according to the present disclosure. FIG. 3 is a flowchart illustrating a method for generating a keymap of an electronic device that performs a keymap-based indexing for an index database according to the present disclosure. FIG. 4 is a flowchart illustrating a specific keymap generation process of an electronic device that performs a keymap-based indexing for an index database according to the present disclosure. FIG. 5 is a flowchart illustrating a search method of an electronic device that performs a keymap-based indexing for an index database according to the present disclosure. FIG. 6 is a flowchart illustrating a process for determining whether to perform a search based on the bit value of an electronic device that performs a key map-based index for an index database according to the present disclosure. FIG. 7 is a configuration diagram illustrating the keymap configuration of an electronic device that performs a keymap-based indexing for an index database according to the present disclosure. FIG. 8 is a conceptual diagram illustrating bit settings according to a byte index and a bit index of an electronic device that performs a keymap-based index for an index database according to the present disclosure. FIG. 9 is a conceptual diagram illustrating the bit collision process of an electronic device performing a keymap-based index for an index database according to the present disclosure. Specific details for implementing the invention

[0017] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and general content in the art to which this disclosure pertains or content that overlaps between embodiments is omitted. The terms 'part, module, component, block' as used in the specification may be implemented in software or hardware, and depending on the embodiments, a plurality of 'parts, modules, components, blocks' may be implemented as a single component, or a single 'part, module, component, block' may include a plurality of components.

[0018] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are directly connected but also cases where they are indirectly connected, and indirect connections include connections made via a wireless communication network.

[0019] Furthermore, when it is stated that a part "includes" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0020] Throughout the specification, when it is stated that a component is located "on" another component, this includes not only cases where a component is in contact with another component, but also cases where another component exists between the two components.

[0021] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0022] Singular expressions include plural expressions unless there is an obvious exception in the context.

[0023] In each step, identification codes are used for convenience of explanation and do not describe the order of the steps; the steps may be performed differently from the specified order unless a specific order is clearly indicated in the context.

[0024] The operating principles and embodiments of the present disclosure will be described below with reference to the attached drawings.

[0025] In this specification, the term "device according to the present disclosure" includes all various devices capable of performing computational processing and providing results to a user. For example, the device according to the present disclosure may include all of a computer, a server device, and a portable terminal, or may be in the form of any one of these.

[0026] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0027] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0028] The above portable terminal may include, for example, all types of handheld-based wireless communication devices such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smartphones, etc., as well as wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMDs).

[0029] Functions related to artificial intelligence according to the present disclosure are operated through a processor and memory. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or artificial intelligence-dedicated processors such as NPUs. The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, if the one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0030] The predefined operating rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a predefined operating rules or artificial intelligence models configured to perform desired characteristics (or objectives) are created by a basic artificial intelligence model being trained using multiple learning data by a learning algorithm. Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0031] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through operations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained by the artificial intelligence model during the learning process is reduced or minimized. Artificial neural networks may include deep neural networks (DNNs), such as Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), or Deep Q-Networks, but are not limited to the examples mentioned above.

[0032] According to an exemplary embodiment of the present disclosure, a processor can implement artificial intelligence. Artificial intelligence refers to a machine learning method based on an artificial neural network that enables a machine to learn by mimicking human biological neurons. Methodologies of artificial intelligence can be classified according to the learning method into supervised learning, where input and output data are provided together as training data and the solution (output data) to the problem (input data) is predetermined; unsupervised learning, where only input data is provided without output data and the solution (output data) to the problem (input data) is not predetermined; and reinforcement learning, where a reward is given from an external environment whenever an action is taken from the current state, and learning proceeds in a direction that maximizes such reward. In addition, artificial intelligence methodologies can be classified according to the architecture, which is the structure of the learning model. The architectures of widely used deep learning technologies can be classified into Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Transformers, and Generative Adversarial Networks (GAN).

[0033] The device and system may include an artificial intelligence model. The artificial intelligence model may be a single model or may be implemented as multiple models. The artificial intelligence model may be composed of a neural network (or artificial neural network) and may include statistical learning algorithms that mimic biological neurons in machine learning and cognitive science. A neural network may refer to a model that possesses problem-solving capabilities by having artificial neurons (nodes) that form a network through synaptic connections and change the strength of synaptic connections through learning. The neurons of a neural network may include combinations of weights or biases. A neural network may include one or more layers composed of one or more neurons or nodes. For example, the device may include an input layer, a hidden layer, and an output layer. The neural network constituting the device can infer a result (output) to be predicted from an arbitrary input by changing the weights of the neurons through learning.

[0034] The processor can create a neural network, train or learn a neural network, perform computations based on received input data, generate an information signal based on the results of the computation, or retrain the neural network. Neural network models may include, but are not limited to, various types of models such as Convolutional Neural Networks (CNN), Region with Convolutional Neural Networks (R-CNN), Region Proposal Networks (RPN), Recurrent Neural Networks (RNN), Stacking-based Deep Neural Networks (S-DNN), State-Space Dynamic Neural Networks (S-SDNN), Deconvolutional Networks, Deep Belief Networks (DBN), Restricted Boltzmann Machines (RBM), Fully Convolutional Networks, Long Short-Term Memory Networks (LSTM), and Classification Networks, such as GoogleNet, AlexNet, and VGG Network. The processor may include one or more processors to perform computations according to the neural network models. For example, a neural network is a deep neural It may include a network (Deep Neural Network).

[0035] Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Function), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto) Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial Network), LSM (Liquid State Machine), ELM (Extreme Learning Machine), ESN (Echo A person skilled in the art will understand that any neural network may be included, but is not limited to, State Network, Deep Residual Network, Differential Neural Computer, Neural Turing Machine, Capsule Network, Kohonen Network, and Attention Network.

[0036] According to an exemplary embodiment of the present disclosure, the processor comprises a Convolutional Neural Network (CNN) such as GoogleNet, AlexNet, VGG Network, Region with Convolutional Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based Deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolutional Network, Deep Belief Network (DBN), Restricted Boltzmann Machine (RBM), Fully Convolutional Network, Long Short-Term Memory (LSTM) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT, SP-BERT, MRC / QA, Text Analysis, Dialog System, GPT-3, GPT-4 for Natural Language Processing, Visual Analytics, Visual Understanding, Video Synthesis for Vision Processing, Anomaly Detection, Prediction, Time-Series Forecasting, Optimization for ResNet Data Intelligence, Various artificial intelligence structures and algorithms, such as recommendation and data creation, may be used, but are not limited thereto. Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0037] FIG. 1 is a block diagram briefly illustrating the configuration of an electronic device that performs a key map-based index for an index database according to the present disclosure.

[0038] Referring to FIG. 1, the electronic device (100) according to the present disclosure may include an input / output module (110), a communication module (120), a memory (130), and a processor (140). Hereinafter, the electronic device (100) is an electronic device that generates a key map and pre-queries whether an index key exists according to the generated key map, and the method thereof is assumed to be implemented through the electronic device (100).

[0039] The input / output module (110) may be various interfaces or connection ports that receive user input or output information to the user. The input / output module (110) may be divided into an input module and an output module.

[0040] The input module receives user input from the user. The input module is for inputting video information (or signals), audio information (or signals), data, or information input by the user, and may include at least one of at least one camera, at least one microphone, and a user input unit. Voice data or image data collected by the input unit may be analyzed and processed into a user control command.

[0041] User input can take various forms, including key input, touch input, and voice input. Examples of input modules capable of receiving such user input include traditional keypads, keyboards, and mice; as well as touch sensors that detect user touch; microphones that receive voice signals; cameras that recognize gestures through image recognition; proximity sensors consisting of light or infrared sensors that detect user approach; motion sensors that recognize user movements using accelerometers or gyroscopes; and all other diverse forms of input means that detect or receive various types of user input. This is a comprehensive concept.

[0042] Here, the touch sensor can be implemented as a piezoelectric or capacitive touch sensor that detects touch through a touch panel or touch film attached to the display panel, or as an optical touch sensor that detects touch by an optical method. In addition, the input module may be implemented in the form of an input interface (USB port, PS / 2 port, etc.) that connects an external input device to receive user input, instead of a device that detects user input itself.

[0043] The output module can output various types of information and provide it to the user. The output module is a comprehensive concept that includes a display for outputting video, a speaker for outputting sound (and / or an amplifier connected thereto), a haptic device for generating vibration, and various other forms of output means. In addition, the output module may be implemented in the form of a port-type output interface that connects the individual output means described above.

[0044] The communication module (120) can communicate with an external device. Accordingly, the device can transmit and receive information with an external device through the communication module. For example, the device can communicate with an external device using the communication module so that information stored and generated within the electric vehicle charging management system is shared. The communication module (120) may include, for example, at least one of a wired communication module, a wireless communication module, a short-range communication module, and a location information module.

[0045] Here, communication, that is, the transmission and reception of data, can be performed via wired or wireless means. To this end, the communication module may be composed of a wired communication module that connects to the internet, etc., via a LAN (Local Area Network); a mobile communication module that connects to a mobile communication network via a mobile communication base station to transmit and receive data; a short-range communication module that uses a WLAN (Wireless Local Area Network) family communication method such as Wi-Fi or a WPAN (Wireless Personal Area Network) family communication method such as Bluetooth or Zigbee; a satellite communication module that uses a GNSS (Global Navigation Satellite System) such as GPS (Global Positioning System); or a combination thereof. The wireless communication technology used for communication may include NB-IoT (Narrowband Internet of Things) for low-power communication. In this case, for example, NB-IoT technology may be an example of LPWAN (Low Power Wide Area Network) technology and may be implemented according to standards such as LTE Cat (category) NB1 and / or LTE Cat NB2, but is not limited to the names mentioned above. Additionally, or generally, wireless communication technology implemented in wireless devices according to various embodiments may perform communication based on LTE-M technology. In this case, for example, LTE-M technology may be an example of LPWAN technology and may be referred to by various names such as eMTC (enhanced Machine Type Communication).For example, LTE-M technology may be implemented in at least one of various standards such as 1) LTE CAT 0, 2) LTE Cat M1, 3) LTE Cat M2, 4) LTE non-BL (non-Bandwidth Limited), 5) LTE-MTC, 6) LTE Machine Type Communication, and / or 7) LTE M, and is not limited to the names mentioned above. Additionally or generally, wireless communication technology implemented in wireless devices according to various embodiments may include at least one of ZigBee, Bluetooth, and Low Power Wide Area Network (LPWAN) for low-power communication, and is not limited to the names mentioned above. As an example, ZigBee technology can create personal area networks (PANs) related to small / low-power digital communication based on various standards such as IEEE 802.15.4 and may be referred to by various names.

[0046] The memory (130) can store various types of information. The memory can store data temporarily or semi-permanently. For example, the memory may store an operating system (OS) for operating the first device and / or the second device, data for hosting a website, or data regarding a program or application (e.g., a web application) for generating Braille. In addition, the memory may store modules in the form of computer code as described above.

[0047] Examples of memory (130) may include a hard disk drive (HDD), a solid state drive (SSD), flash memory, ROM (Read-Only Memory), and RAM (Random Access Memory). These memories may be provided as built-in or removable types.

[0048] The processor (140) controls the overall operation of the electronic device (100). To this end, the processor (140) performs computation and processing of various information and can control the operation of the components of the first device and / or the second device.

[0049] The processor (140) may be implemented as a computer or a similar device according to hardware, software, or a combination thereof. Hardware-wise, the processor (140) may be provided in the form of an electronic circuit that processes electrical signals to perform control functions, and software-wise, it may be provided in the form of a program that drives the hardware processor. Meanwhile, unless otherwise specifically mentioned in the following description, the operation of the first device and / or the second device may be interpreted as being performed by the control of the processor (140). That is, the modules may be interpreted as the processor (140) controlling the first device and / or the second device to perform the following operations.

[0050] The processor (140) may be implemented with a memory that stores data for an algorithm or a program that reproduces the algorithm for controlling the operation of components within the device, and at least one sub-processor (not shown) that performs the aforementioned operation using the data stored in the memory. In this case, the memory and the processor may each be implemented as separate chips. Alternatively, the memory and the processor may be implemented as a single chip.

[0051] Additionally, the processor (140) may control a combination of any one or more of the components described above in order to implement various embodiments according to the present disclosure, which will be described in the drawings below, on the device.

[0052] FIG. 2 is a block diagram illustrating the structure between an index database and a key map of an electronic device that performs a key map-based indexing for an index database according to the present disclosure.

[0053] An electronic device (100) according to one embodiment of the present disclosure can compare a comparison key extracted from an input document with an index key stored in an index database (200) for plagiarism check.

[0054] An electronic device (100) according to one embodiment of the present disclosure can pre-query whether there exists an index key matching the comparison key within the index database (200) based on a key map (300) before comparing the comparison key with the index key of the index database (200).

[0055] Specifically, as illustrated in FIG. 2, there may be multiple index databases (200) in which index keys are stored, and key maps (300) may be configured to be connected one-to-one with each index database (100). In other words, each key map (300) corresponds uniquely to a different index database (200), and key map 1 may be connected one-to-one with index database 1, key map 2 with index database 2, and key map 3 with index database 3.

[0056] The comparison key extracted from the input document can be independently queried in each key map (300). According to the present embodiment, the comparison key can first query key map 1 to check whether an index key matching the comparison key exists in index database 1. The same query can be performed for key map 2 and key map 3.

[0057] For example, if the bit value existing at the position corresponding to the comparison key retrieved in Key Map 1 is confirmed to be 0, it is determined that there is no need to retrieve Index Database 1, and the search may be omitted. In other words, there is no need to perform a search (11) on Index Database 1.

[0058] On the other hand, if the bit value existing at the position corresponding to the comparison key retrieved in Key Map 1 is confirmed to be 1, it is determined that there is a possibility that the comparison key exists in Index Database 1, and an inverse index search for Index Database 1 can be performed. In other words, it is necessary to perform a search (11) on Inverse Index Database 1.

[0059] For keymap 2 and keymap 3, it is possible to determine whether to perform index search (13, 15) for index database 2 and index database 3 after keymap querying in the same way.

[0060] When a comparison key extracted from an input document is found to be non-existent in Key Map 1 and determined to be possible to exist in Key Map 2, the electronic device (100) performs an inverse index search only on Index Database 2, thereby reducing unnecessary search requests within the entire system and reducing search delay.

[0061] According to the present embodiment, each index database (200) often contains different topics, document groups, or specific ranges of text, and since the key map (300) is also generated individually based on the set of index keys of the index database (200), the existence of comparison keys can be determined independently in each key map (300).

[0062] According to one embodiment of the present disclosure, this structure can be expanded in the same way even if the number of index databases (200) increases, and maintenance can be easy because the key map corresponding to each index database (200) is maintained as a static (read-only) structure.

[0063] Accordingly, at least one processor (140) of an electronic device (100) according to one embodiment of the present disclosure may be configured to generate and store a key map (300) that uniquely corresponds to each index database (200), and to independently query a comparison key extracted from an input document input for plagiarism check in each of the key maps.

[0064] FIG. 3 is a flowchart illustrating a method for generating a keymap of an electronic device that performs a keymap-based indexing for an index database according to the present disclosure.

[0065] An electronic device (100) according to one embodiment of the present disclosure may include a memory (130) storing at least one process for performing a key map generation operation mapping the presence or absence of any index key in an index database, and at least one processor (140) that performs the operation according to said process.

[0066] As illustrated in FIG. 3, the at least one processor (140) may perform the steps of generating a hash value for each of the multiple index keys stored in the index database (200) (S310), calculating a bit index within a predetermined hash table size based on the hash value (S320), and creating a key map (300) for the index keys by setting a bit value corresponding to the calculated bit index (S330).

[0067] In step (S310), the hash value can be calculated by applying a bit-based hash function, such as the MD5 hash function, to the index key, and a different hash value can be generated for each index key. The generated hash value can be used as an input value to determine a bit index within the hash table size in a subsequent step.

[0068] In step (S320), the hash value can be separated into upper bits and lower bits, and a single bit sequence can be generated by performing an XOR operation on the two separated values. The generated bit sequence is subjected to a modulo operation as a value representing the hash table size, and the result can be used as a bit index within the key map (300). According to the present embodiment, the hash table size represents the total size of the bit map of the key map (300) and can be set independently per file depending on the size of the index database (200).

[0069] In step (S330), a key map (300) can be generated by setting the bit pointed to by the calculated bit index to 1. The bit indicates that the corresponding index key exists in the index database (200). If the existing value of the bit index position is already 1, the bit value can be maintained as is, which ensures that the key map is maintained stably even in a collision situation where multiple index keys sharing the same hash index occur.

[0070] By processing the entire set of index keys sequentially, a key map (300) for the entire index database (200) can finally be completed.

[0071] FIG. 4 is a flowchart illustrating a specific keymap generation process of an electronic device that performs a keymap-based indexing for an index database according to the present disclosure.

[0072] At least one processor (140) of an electronic device (100) according to one embodiment of the present disclosure may be configured to apply an MD5 hash function to an index key stored in an index database (200) to calculate a plurality of bit-based pre-hash values, separate the pre-hash values ​​into upper bits and lower bits, perform an XOR logical operation between the upper bits and the lower bits, generate a final hash value by performing a modulo operation on the result of the logical operation with the size of the hash table, calculate a byte index within the size of the hash table and a bit index within the byte index based on the final hash value, and perform an OR logical operation to set the bit corresponding to the bit index to 1 to change the bit value of the bit.

[0073] As illustrated in FIG. 4, at least one processor (140) can query the bit position for an index key or a comparison key of an input document in an index database (200) by performing an MD5 hash function application (S410), an XOR operation between the upper bit and the lower bit (S420), a modulo operation with the hash table size (S430), a byte index and bit index operation (S440), and a bit setting (S450).

[0074] Specifically, in the application of the MD5 hash function (S410), the MD5 hash function can generate a fixed-length hash value of 128 bits for the index key or comparison key, and the hash value can be used as base data for subsequent index operations. In other words, a 128-bit checksum can be generated by applying the MD5 hash function to the index key or comparison key.

[0075] In the XOR operation between the upper bits and lower bits (S420), the generated 128-bit hash value can be separated into an upper bit region and a lower bit region to perform an XOR operation between the two values. For example, the upper 64 bits can be divided into upper bits and the lower 64 bits into lower bits, and each 64 bit can be XORed. The XOR operation has the effect of mitigating bit bias in the hash value to reduce the possibility of collisions, and the result of the operation can be reduced to a single 64-bit value. By reducing the index key or comparison key to a single 64-bit value, memory usage can be reduced and speed improved during subsequent modulo operations and index calculations, and computational efficiency can be increased. The above XOR result can be used as an intermediate hash value for subsequent modulo operations.

[0076] In the modulo operation (S430) with the hash table size, the final hash value can be calculated by performing a modulo operation with the predefined hash table size on the result of the XOR operation. The hash table size used here corresponds to the total bitmap size constituting the key map (300) and can be set independently for each index database (200). The final hash value can be used as an index pointing to a specific bit position within the bitmap of the key map (300).

[0077] In the byte index and bit index operation (S440), the byte index and the bit index within the byte can be calculated based on the final hash value. For example, since the hash value acts as an absolute index in bit units, the quotient of the final hash value divided by 8 can be set as the byte index, and the remainder can be set as the bit position within the corresponding byte. Through this index conversion process, location information referring to a specific bit within the bitmap structure of the key map (300) can be obtained from the final hash value.

[0078] In bit setting (S450), an OR logical operation can be performed to set the bit value corresponding to the bit position to 1. The bitmap within the key map (300) is allocated a byte array that is initialized to all 0s, and after accessing the byte index (idx1) determined through steps S410 to S440, only the bit value at the bit index (idx2) position within the byte index (idx1) can be changed to 1.

[0079] According to the present embodiment, since an OR logical operation is performed, the bit value can be maintained as 1 without additional operation even if the corresponding bit is already set to 1. This allows the bit value to be maintained as 1 to indicate the possibility of the index key's existence, even if different index keys have the same hash value when a hash function is applied. The bitmap configured in this way can be used as an information structure indicating that the index key exists in the index database.

[0080] That is, in the bit setting (S450), the bit value can be changed according to the following mathematical formula 1, where Vnew is the new value of the idx1 byte of the bitmap in the keymap (300), Vold is the existing value of the idx1 byte of the bitmap, i is an integer from 0 to 7 as the bit index of idx2, and V is a bitwise OR operator.

[0081] [Mathematical Formula 1]

[0082] V new = V old V2 i

[0083] The steps from S410 to S440 can be performed during the indexing process to perform hash calculation and index conversion for the index key, and can be performed during the search process to perform hash calculation and index conversion for the comparison key. After the hash calculation and index conversion for the index key, the bit value can be changed in the bit setting (S450), and after the hash calculation and index conversion for the comparison key, the bit value of the corresponding bit can be queried to check the possibility of the index key's existence.

[0084] FIG. 5 is a flowchart illustrating a search method of an electronic device that performs a keymap-based indexing for an index database according to the present disclosure.

[0085] At least one processor (140) of an electronic device (100) according to one embodiment of the present disclosure may perform the steps of: generating a hash value for a comparison key extracted from an input document input for plagiarism check (S510); determining a bit index within a key map (300) of an index database (200) to be searched based on the hash value of the comparison key (S520); and determining whether to search the index database according to the bit value of the key map by querying the bit value set in the determined bit index (S530).

[0086] In other words, steps (S510) and (S520) can be performed by repeating the same process as illustrated in FIG. 4 for the comparison key of the input document entered to perform the search.

[0087] Specifically, in step (S510), a comparison key is extracted from an input document for plagiarism check, and then a hash value is generated for the comparison key. The process of generating a hash value for the comparison key is performed in the same manner as the process of generating a hash for an index key. That is, a 128-bit hash value is generated through processes such as the application of an MD5 hash function, separation of upper and lower bits, and XOR operation, and then a modulo operation is applied with the same hash table size as that used for the index key so that hash-based indexing for the comparison key can be performed consistently.

[0088] In step (S520), since each index database (200) has a single key map (300) uniquely corresponding to the index database (200), the comparison key can be queried by independently calculating a hash index for each index database (200). The calculated bit index refers to a specific bit position in the bitmap data of the key map (300) for determining whether the comparison key exists.

[0089] In step (S530), at least one processor (140) may be configured to skip searching the index database when it is determined that the comparison key does not exist in the index database (200) based on the bit value, and to perform a search on the index database when it is determined that there is a possibility that the comparison key exists in the index database (200) based on the bit value.

[0090] According to the present embodiment, the key map (300) calculates in advance whether index keys stored in the index database (200) exist and expresses this in bits, thereby eliminating the inefficiency of the conventional method of having to search the entire index structure every time even if the comparison key does not exist in the index database (200).

[0091] FIG. 6 is a flowchart illustrating a process for determining whether to perform a search based on the bit value of an electronic device that performs a key map-based index for an index database according to the present disclosure.

[0092] In step (S530), it can be determined whether to search for a comparison key in the index database (200) based on the bit value.

[0093] If the bit value at the specific bit position is 0, it is determined that the comparison key does not exist in the corresponding index database (200), so an inverse index search may not be performed. Conversely, if the bit value is 1, it is determined that there is a possibility that the comparison key exists in the corresponding index database (200), so an inverse index search may be performed. The possibility of existence of an index key corresponding to the comparison key can be determined quickly through a single bit lookup.

[0094] Specifically, as illustrated in FIG. 6, a bit value within the corresponding key map (300) can be extracted according to a bit index calculated based on the hash value of the comparison key in step (S610) (S610). According to the present embodiment, the bit value is represented as 0 or 1, and 1 can be configured to mean that there is a possibility that the corresponding comparison key exists in the index database, and 0 can be configured to mean that it does not exist.

[0095] In step (S620), the processor can determine whether the extracted bit value is 1. If the bit value is determined to be 1 (Yes in S620), it is determined that there is a possibility that the comparison key exists in the index database, so a search of the index database (200) can be performed (S630). Conversely, if the bit value is determined to be 0 (No in S620), it is determined that the comparison key does not exist in the index database, and accordingly, a search of the index database can be skipped (S640).

[0096] Searching can be performed by exploring the inverted index structure or document ID mapping structure held by the index database (200) to extract documents that match the comparison key.

[0097] FIG. 7 is a configuration diagram illustrating the keymap configuration of an electronic device that performs a keymap-based indexing for an index database according to the present disclosure.

[0098] At least one processor (140) of an electronic device (100) according to one embodiment of the present disclosure may be configured to store the key map (300) composed of bitmap data including a header comprising a magic value for identifying the key map file, the total size of the key map file, the size of an index database corresponding to the key map, a hash table size, and version information of the key map file, and a bit index corresponding to the hash table size.

[0099] As illustrated in FIG. 7, the keymap file may include headers (710–750) and bitmap data (760).

[0100] The header contains basic metadata necessary for interpreting and verifying the keymap file, and the first field of the header, MAGIC (710), can be used as a unique identifier to identify that the file is a keymap file. The MAGIC value can be used to determine the file format, check integrity, or detect unexpected file corruption.

[0101] MAP_SIZE (720) is information indicating the total size of the key map file, and at least one processor (140) can refer to MAP_SIZE (720) to load the key map (300) into memory or read only the necessary range in a disk-based access method. Integrity verification is possible by performing the role of detecting whether the file is corrupted when there is a discrepancy between the actual file size and MAP_SIZE (720).

[0102] DB_SIZE (730) represents the size of the index database (200) corresponding to the key map (300), and through this, the state of the index database (200) at the time the key map (300) was created can be verified. That is, DB_SIZE (730) plays a role in maintaining file consistency according to the structure in which the key map (300) is mapped one-to-one to a specific index database (200). When a discrepancy occurs between the actual index database (200) file size and DB_SIZE (730), it performs the role of detecting whether the database has changed and whether the key map (300) needs to be regenerated, thereby enabling the management of the validity or reliability of the key map (300).

[0103] HASH_SIZE (740) represents the total hash table size of the bitmap data of the key map (300) and can be used as a reference value for converting the hash values ​​of the index key and comparison key into bit indices. According to the present embodiment, the hash value of the index key is calculated modulo the hash table size to set the corresponding bits, so HASH_SIZE (740) is a key structural information that directly affects the bit collision distribution and search efficiency.

[0104] VERSION (750) indicates keymap file version information and can be used to manage whether the keymap (300) structure is changed or compatibility with the index database (200).

[0105] The header of the key map (300) may include MAGIC (710), MAP_SIZE (720), DB_SIZE (730), HASH_SIZE (740), and VERSION (750).

[0106] Bitmap data (760) consists of a bit array having the same length as the hash table size and represents the existence of an index key in the index database (200) in bit units. For an index key existing in the index database (200), a specific bit position is set to 1 through hash calculation, and if it does not exist, the corresponding bit can be kept as 0.

[0107] At least one processor (140) according to one embodiment of the present disclosure may be configured to query the key map (300) according to a memory-based access mode in which the entire key map (300) is loaded into memory and the key map is queried, or a disk-based access mode in which the header of the key map (300) is queried and the bitmap data (760) of the header is read directly from the disk and the key map is queried.

[0108] FIG. 8 is a conceptual diagram illustrating bit settings according to a byte index and a bit index of an electronic device that performs a keymap-based index for an index database according to the present disclosure.

[0109] The bitmap data (760) of this embodiment has a total bit length corresponding to the hash table size (HASH_SIZE), and the bit units are composed of a byte array to indicate whether each index key exists.

[0110] As illustrated in FIG. 8, with reference to an exemplary bit array from bit position 0 to 31, bitmap data (760) may consist of a number of consecutive bytes, and individual bits within each byte are mapped to one bit index on a hash table.

[0111] For example, “Byte 0”, “Byte 1”, “Byte 2”, and “Byte 3” represent a contiguous memory structure of bitmap data, and an index calculated by a hash value is separated into a byte index and a bit index and can point to a specific bit position within the corresponding byte.

[0112] As shown in FIG. 8, val=25 means that when the hash table size is 32, the final hash value is 25, and accordingly, the byte index is 3, which is the quotient of 25 divided by 8, and the bit index is 1, which is the remainder of 25 divided by 8. Accordingly, the second bit (bit position 1) of the “byte 3” of the bitmap data can be a bit indicating whether a comparison key or index key exists.

[0113] FIG. 9 is a conceptual diagram illustrating the bit collision process of an electronic device performing a keymap-based index for an index database according to the present disclosure.

[0114] The bit value is a binary structure containing 0 or 1, and at least one processor (140) may be configured to maintain the bit value as is if the bit value of the bit index to be changed is already 1.

[0115] Specifically, at least one processor (140) generates a key map (300) by calculating a hash value for each of a plurality of index keys and then setting a bit position corresponding to the hash value to 1, and can perform an OR operation to operate reliably even when a plurality of index keys are hashed to the same bit position.

[0116] As illustrated in FIG. 9, the initial state bitmap data (760) may have a state where each byte is set to 0. Subsequently, if a bit index calculated based on the hash value for the first index key corresponds to a specific position (e.g., the second bit within the third byte), that bit is set to 1.

[0117] Next, even if a hash collision occurs where the hash value of the third index key has the same bit index—that is, hashed to the same bit position—an OR logical operation can be performed on the bit position to maintain the already set bit value. Even if the same bit position is pointed to again, the bit value does not change and remains at 1.

[0118] Meanwhile, a method for performing a key map-based index for an index database, performed by a processor of the device according to the present disclosure, wherein the method may include the steps of generating a hash value for each of a plurality of index keys stored in the index database (200), calculating a bit index within a predetermined hash table size based on the hash value, and setting a bit value corresponding to the calculated bit index to generate a key map (300) for the index keys.

[0119] Content that overlaps with the above is omitted for the sake of brevity in the specification.

[0120] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium that stores instructions executable by a computer. The instructions may be stored in the form of program code and, when executed by a processor, may generate a program module to perform the operation of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0121] Computer-readable recording media include all types of recording media that store instructions that can be decoded by a computer. Examples include ROM (Read Only Memory), RAM (Random Access Memory), magnetic tape, magnetic disk, flash memory, optical data storage devices, etc.

[0122] As described above, the disclosed embodiments have been explained with reference to the attached drawings. Those skilled in the art will understand that the present disclosure may be practiced in forms different from the disclosed embodiments without changing the technical spirit or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be interpreted restrictively. Explanation of the symbols

[0123] 100: Electronic device 110: I / O module 120: Communication module 130: Memory 140: Processor 200: Index database Keymap: 300

Claims

Claim 1 Memory storing at least one process for performing a key map generation operation mapping the existence of an arbitrary index key within an index database; An electronic device for performing keymap-based indexing for index databases, comprising: at least one processor that performs the operation according to the above process; wherein the at least one processor is configured to generate a hash value for each of a plurality of index keys stored in each of the plurality of index databases to generate an independent unique keymap file corresponding one-to-one to each of the plurality of index databases, respectively calculate a byte index within a predefined hash table size and a bit index within the byte index based on the hash value, respectively set a bit value corresponding to the calculated bit index to generate an individual keymap for each of the plurality of index keys, and record consistency verification data including a magic value for identifying the keymap file, the size of the corresponding index database, and the hash table size in the header area of ​​the individual keymap, and independently query each of the plurality of unique keymap files based on a comparison key input for plagiarism check, and control the search path to exclude a specific index database determined not to contain the comparison key from the actual search target. Claim 2 An electronic device for performing a keymap-based indexing of an index database configured such that, in claim 1, at least one processor generates a hash value for a comparison key extracted from an input document input for plagiarism check, determines a bit index within a keymap of an index database to be searched based on the hash value of the comparison key, and queries a bit value set in the determined bit index to determine whether to search the index database according to the bit value of the keymap. Claim 3 An electronic device for performing a key map-based index for an index database, wherein the at least one processor is configured to omit a search for the index database when it is determined that the comparison key does not exist in the index database based on the bit value, and to perform a search for the index database when it is determined that there is a possibility that the comparison key exists in the index database based on the bit value. Claim 4 In claim 2, the at least one processor is an electronic device that performs a keymap-based index for an index database configured to store a keymap comprising a header including a magic value for identifying the keymap file, the total size of the keymap file, the size of an index database corresponding to the keymap, a hash table size, and version information of the keymap file, and bitmap data including a bit index corresponding to the hash table size. Claim 5 An electronic device for performing a keymap-based index for an index database configured such that the at least one processor queries the keymap according to a memory-based access mode in which the entire keymap is loaded into memory, or a disk-based access mode in which the header of the keymap is queried and bitmap data of the header is read directly from the disk to query the keymap. Claim 6 An electronic device according to claim 1, wherein at least one processor generates and stores a key map uniquely corresponding to each of the index databases, and performs a key map-based indexing for an index database configured to independently query a comparison key extracted from an input document input for plagiarism check in each of the key maps. Claim 7 An electronic device for performing a key map-based index for an index database configured such that, in claim 1, at least one processor applies an MD5 hash function to an index key stored in the index database to calculate a plurality of bit-based pre-hash values, separates the pre-hash values ​​into upper bits and lower bits, performs an XOR logical operation between the upper bits and the lower bits, generates a final hash value by performing a modulo operation on the result of the logical operation with the size of the hash table, calculates a byte index within the size of the hash table and a bit index within the byte index based on the final hash value, and performs an OR logical operation to change the bit value of the bit by setting the bit corresponding to the bit index to 1. Claim 8 In claim 7, the bit value is a binary structure including 0 or 1, and the at least one processor is an electronic device that performs a keymap-based index for an index database configured to maintain the bit value as is when the bit value of the bit index to be changed is already 1. Claim 9 delete Claim 10 A method for performing a keymap-based index for an index database, performed by a processor of a device, the method comprising: generating a hash value for each of a plurality of index keys stored in the index database; calculating a bit index within a predetermined hash table size based on the hash value; and creating a keymap for the index keys by setting a bit value corresponding to the calculated bit index. A method for performing keymap-based indexing for an index database, comprising: generating a hash value for each of a plurality of index keys stored in each of the plurality of index databases to create an independent, unique keymap file corresponding one-to-one to each of the plurality of index databases; calculating a byte index within a predefined hash table size and a bit index within the byte index based on the hash value, respectively; setting a bit value corresponding to the calculated bit index to create an individual keymap for each of the plurality of index keys; and configuring to record consistency verification data including a magic value for identifying the keymap file, the size of the corresponding index database, and the hash table size in the header area of ​​the individual keymap, and independently querying each of the plurality of unique keymap files based on a comparison key input for plagiarism check, and controlling the search path to exclude a specific index database determined not to contain the comparison key from the actual search target.

Images