Apparatus and method for generating comparison keys for determining presence or absence of plagiarism in document

Abstract

To provide an apparatus and method for generating comparison keys for determining the presence or absence of plagiarism in a document.SOLUTION: An apparatus for generating comparison keys for determining the presence or absence of plagiarism in a document according to the present disclosure divides a Japanese sentence into syntactic units on the basis of Japanese character forms described in a Japanese document, extracts chunks corresponding to the divided syntactic units, and generates a comparison key for determining the presence or absence of plagiarism by applying a first integration process to a plurality of extracted chunks. The first integration process includes a step of integrating, with a preceding chunk, a chunk corresponding to one hiragana character serving as a particle among the plurality of chunks.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an electronic device and a method for operating the same, and more particularly to a device and method for extracting a Japanese comparison key algorithm for determining whether a Japanese document has been plagiarized. [Background technology]

[0002] Extracting Japanese comparison keys is a method for extracting Japanese comparison keys from documents so that documents written in Japanese can be compared with target documents for plagiarism testing. Plagiarism testing between Japanese documents can be performed using the extracted comparison keys as a reference.

[0003] In the case of Korean and English, comparison keys are extracted based on word division, which divides words into phrase units, and plagiarism checks can be performed on the target documents. However, Japanese does not use phrase units that use word division due to the characteristics of its writing system. Instead, syntactic analysis is performed on the corresponding concept of phrase units, which are not divided by word division but by parts of speech. This means that conventional comparison key extraction techniques based on word division cannot be applied, and new rules that fit Japanese phrase units must be defined.

[0004] In order to extract comparison keys based on Japanese phrases, rules must be defined taking into account parts of speech, but building a part-of-speech dictionary for all Japanese expressions requires considerable human and time resources, and is expensive to build. Furthermore, new expressions are constantly emerging in the language, and there are practical limitations to extracting comparison keys based on phrases by building a part-of-speech dictionary that covers all expressions.

[0005] The issue of plagiarism has long been a problem in Japanese educational institutions, research institutions, and other environments where Japanese documents are produced. In particular, the issue of plagiarism in assignments and graduation theses written by students and academic papers written by researchers at Japanese educational institutions such as universities has become a hot topic in the media, and the need for countermeasures has been raised in Japanese society.

[0006] Retraction Watch, a website that publishes information about retractions of scientific papers from the global scientific journal Science, also points out that Japan accounts for 50% of the top 10 researchers with the most retractions.

[0007] The problem of plagiarism is not only a problem in Japan, but also in countries around the world, including Korea, and in response to this, technology for detecting text plagiarism using plagiarism detection programs is being developed.However, the comparative key extraction method for phrase units based on word segmentation used in Korean and English-speaking countries for plagiarism detection cannot be applied to the Japanese language environment, which does not use word segmentation. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Publication No. 2023-023589 Summary of the Invention [Problem to be solved by the invention]

[0009] The present disclosure has been made in consideration of the above circumstances, and its purpose is to provide an apparatus and method for defining plagiarism techniques for works written in Japanese by extracting comparison keys based on Japanese phrases, which are concepts corresponding to phrases.

[0010] The problems to be solved by the present disclosure are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0011] In one aspect, the present disclosure provides an apparatus for determining whether a Japanese document has been plagiarized, comprising: a memory for storing code data relating to an algorithm for determining whether the Japanese document has been plagiarized; noun dictionary data including noun dictionary information; and a processor configured to determine whether the Japanese document has been plagiarized using the algorithm in the code data. The processor extracts chunks corresponding to individual syntax by dividing Japanese sentences into syntactic units based on the Japanese character forms written in the Japanese document, divides multiple nouns included in the chunks into individual nouns using noun dictionary information in the noun dictionary data, merges chunks containing hiragana characters placed between characters other than hiragana characters that serve as particles in the divided individual nouns with chunks preceding the hiragana characters, generates full-width characters based on the characters written in the Japanese document, replaces repetition signs corresponding to characters repeated in the Japanese document with the characters to be repeated corresponding to the repetition signs, excludes chunks containing symbols from the extracted chunks, merges chunks containing numbers from the extracted chunks with the preceding chunks, and generates the processed chunks as a comparison key to be used in plagiarism checks.

[0012] In another aspect, a method for determining whether a Japanese document has been plagiarized according to the present disclosure includes: extracting chunks corresponding to individual syntax by dividing a Japanese sentence into syntax units based on Japanese character forms described in the Japanese document; dividing a plurality of nouns included in the chunks into individual nouns using noun dictionary information of the noun dictionary data; merging chunks including hiragana characters placed between characters other than hiragana characters that serve as particles in the divided individual nouns with chunks preceding the hiragana characters; generating full-width characters based on characters described in the Japanese document; replacing repetition symbols corresponding to characters repeated in the Japanese document with characters to be repeated corresponding to the repetition symbols; excluding chunks including symbols from the extracted chunks; merging chunks including numbers from the extracted chunks with preceding chunks; and generating the processed chunks as a comparison key to be used in plagiarism checks. [Effects of the Invention]

[0013] The present disclosure has the effects of reducing human and time resources, determining whether plagiarism has occurred more accurately, dramatically reducing the probability of false positives, and enabling rational plagiarism testing.

[0014] The effects of the present disclosure are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 illustrates an apparatus according to the present disclosure. [Figure 2] FIG. 1 illustrates the method of the present disclosure. [Figure 3] FIG. 10 is a diagram illustrating an exemplary embodiment of extracting a Japanese comparison key algorithm for determining whether a Japanese document has been plagiarized according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0016] The same reference numerals refer to the same components throughout this disclosure. This disclosure does not describe all elements of the embodiments, and general content in the technical field to which this disclosure pertains or overlapping content in the embodiments will be omitted. The terms "unit, module, component, block" used in the specification can be realized as software or hardware, and depending on the embodiment, multiple "units, modules, components, blocks" may be realized as one component, or one "unit, module, component, block" may include multiple components.

[0017] Throughout this specification, when a part is said to be "connected" to another part, this includes not only direct connection but also indirect connection, including connection via a wireless communication network.

[0018] Furthermore, when a part is described as "comprising" a certain component, this does not mean that other components are excluded, unless otherwise specified, and means that other components may also be included.

[0019] Throughout this specification, when an element is said to be "on" another element, this includes not only when the element is in contact with the other element, but also when there is another element between the two elements.

[0020] The terms "first," "second," etc. are used to distinguish one component from another, and the components are not limited to the terms described above.

[0021] The singular expression includes the plural expression unless the context clearly indicates otherwise.

[0022] The reference numbers in each step are used for convenience of explanation, and the reference numbers do not describe the order of each step, and each step can be performed in a different order than specified unless the context clearly dictates a specific order.

[0023] The working principle and embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0024] The term "device according to the present disclosure" as used herein includes all of the various devices capable of performing computations and providing results to a user. For example, the device according to the present disclosure may include all of a computer, a server device, and a portable terminal, or may take any one of these forms.

[0025] Here, the computer may include, for example, a notebook computer, a desktop computer, a laptop computer, a tablet PC, a slate PC, etc., equipped with a web browser.

[0026] The server device is a server that communicates with external devices and processes information, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0027] The portable terminal is, for example, a wireless communication device that ensures portability and mobility, and may include all kinds of handheld-based wireless communication devices such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smartphones, etc., as well as wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMDs).

[0028] The artificial intelligence functions according to the present disclosure are operated by a processor and memory. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as a CPU, AP, or DSP (Digital Signal Processor), dedicated graphics processors such as a GPU or VPU (Vision Processing Unit), or dedicated artificial intelligence processors such as an NPU. The one or more processors control the processing of input data according to predefined operating rules or artificial intelligence models stored in memory. Alternatively, if the one or more processors are dedicated artificial intelligence processors, the dedicated artificial intelligence processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0029] FIG. 1 is a diagram illustrating the device of the present disclosure.

[0030] Referring to FIG. 1 , the device 100 of the present disclosure can generate a comparison key for determining whether a Japanese document contains plagiarism and calculate the plagiarism rate of the Japanese document. The comparison key may be used for plagiarism testing. The device 100 can extract a comparison key based on word segments, which are phrase division points, using a comparison key extraction method for plagiarism testing. Word segments may be used to separate meaningful sentence structures in Korean and English. In contrast, Japanese does not use word segments and instead uses a mixture of hiragana, katakana, and kanji to easily grasp meaning. Although Japanese uses a mixture of hiragana, katakana, and kanji to separate meaningful sentence structures, the character shapes alone may not fully meet the semantic segmentation criteria of Japanese. Therefore, the device 100 can extract a comparison key based on Japanese phrases and / or Japanese character shapes, taking into account the characteristics of the Japanese language, and then apply separate comparison key extraction conditions to define rules for approximating the Japanese phrase concept to extract a comparison key.

[0031] The device 100 may include an input unit 110, a communication unit 120, a control unit 130, an audio output unit 140, a display unit 150, and the like.

[0032] The input unit 110 is for inputting video information (or signals), audio information (or signals), data, or information input by a user, and may include at least one of at least one camera, at least one microphone, and a user input unit. The audio data and image data collected by the input unit 110 can be analyzed and processed according to a user's control command.

[0033] The communication unit 120 may include one or more components that enable communication with an external device, and may include, for example, at least one of a wired communication module 121 and a wireless communication module 122.

[0034] The wired communication module 121 may include various wired communication modules such as a local area network (LAN) module, a wide area network (WAN) module, or a value added network (VAN) module, as well as various cable communication modules such as a universal serial bus (USB), a high definition multimedia interface (HDMI) (registered trademark), a digital visual interface (DVI), a recommended standard 232 (RS-232), a power line communication, or a plain old telephone service (POTS).

[0035] The wireless communication module 122 may include a Wifi (registered trademark) module, a WiBro (Wireless broadband) module, as well as wireless communication modules 122 that support various wireless communication methods such as GSM (Global System for Mobile Communication), CDMA (registered trademark) (Code Division Multiple Access), WCDMA (registered trademark) (Wideband Code Division Multiple Access), UMTS (universal mobile telecommunications system), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, and 6G.

[0036] The wireless communication module 122 may include an antenna and a wireless communication interface, and may further include a signal conversion module for modulating a digital control signal output from the controller via the wireless communication interface into an analog wireless signal under the control of the controller and / or demodulating an analog wireless signal received via the wireless communication interface into a digital control signal.

[0037] The control unit 130 may include a memory 131 that stores data for an algorithm or a program that reproduces the algorithm for controlling the operation of components in the device 100, and at least one processor 132 that performs the above-mentioned operations using the data stored in the memory 131. In this case, the memory 131 and the processor 132 may be implemented as separate chips, or the memory 131 and the processor 132 may be implemented as a single chip.

[0038] The memory 131 may store data supporting various functions of the device 100 and programs for the operation of the control unit 130, may store input / output data (e.g., music files, still images, videos, etc.), may store a plurality of application programs (or applications) run by the device 100, and data and commands for the operation of the device 100. At least some of these application programs may be downloaded from an external server via wireless communication. Such memory 131 may include at least one type of storage medium selected from the group consisting of a flash memory type, a hard disk type, a solid state disk type (SSD type), a silicon disk drive type (SDD type), a micro multimedia card type, a card-type memory (such as an SD or XD memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. Furthermore, memory 131 may be a database separate from device 100 but connected thereto by wire or wirelessly.

[0039] The memory 131 can store code data related to an algorithm for determining whether a Japanese document has been plagiarized. The memory 131 can also store noun dictionary data including noun dictionary information. The noun dictionary information can include information related to a noun dictionary.

[0040] The processor 132 can be configured to use an algorithm of the code data to determine whether the Japanese document is plagiarized.

[0041] The processor 132 may extract a comparison key based on the character form of Japanese characters. For example, the processor 132 may divide a Japanese document into chunk units for each of hiragana, katakana, kanji, symbols, and numbers. As another example, the processor 132 may extract chunks corresponding to individual syntax by dividing a Japanese sentence into syntactic units based on the character form of Japanese characters written in the Japanese document.

[0042] The processor 132 can apply a noun dictionary. For example, when a chunk unit includes multiple nouns, the processor 132 can subdivide the chunk into individual nouns. As another example, the processor 132 can use noun dictionary information in the noun dictionary data to divide multiple nouns included in the chunk into individual nouns.

[0043] Processor 132 can process comparison keys for single hiragana characters in an integrated manner. For example, processor 132 can group and segment a single hiragana character that serves as a particle with a preceding construction. As another example, processor 132 can merge a chunk that includes a hiragana character that is placed between characters other than a hiragana character that serves as a particle in a segmented individual noun with a chunk that precedes the hiragana character.

[0044] The processor 132 can replace half-width characters with full-width characters. For example, the processor 132 can generate full-width characters based on characters written in the Japanese document. As another example, the processor 132 can replace identical characters (full-width characters and / or half-width characters) or characters with different widths with full-width characters. For example, if the Japanese document contains the character "saito" written in half-width katakana, the half-width katakana character "saito" can be replaced with the full-width character "saito." However, the present invention is not limited to this, and non-Japanese symbols, numbers, etc. can also be included as targets for replacement with full-width characters.

[0045] The processor 132 can replace a character corresponding to a repeat symbol with a character corresponding to its meaning. For example, the processor 132 can replace a repeat symbol indicating a repeated character with the same character from the previous paragraph. As another example, the processor 132 can replace a repeat symbol corresponding to a character repeated in a Japanese document with the character to be repeated corresponding to the repeat symbol.

[0046] The processor 132 can perform processing operations to exclude comparison keys for symbols. For example, if a sentence contains symbols, the processor 132 can exclude the symbols from the extracted chunks by not recognizing the symbols contained in the sentence as a single chunk.

[0047] Processor 132 may process numeric comparison keys in an integrated manner. For example, if a sentence contains numbers, processor 132 may integrate extracted chunks containing numbers with the preceding chunks by including the extracted chunks containing numbers in the chunks preceding the numbers.

[0048] The processor 132 can generate chunks processed by the above operations as a comparison key to be utilized in plagiarism checking.

[0049] The audio output unit 140 may output audio data received via the communication unit 120 or stored in the memory 131, or may output an audio signal related to a function performed by the device 100. The audio output unit 140 may include an amplifier 141, a speaker 142, etc.

[0050] The display unit 150 displays (outputs) information processed by the device 100. For example, the display unit 150 may display execution screen information of an application program (e.g., an application) running on the device 100, or UI (User Interface) or GUI (Graphical User Interface) information based on the execution screen information. The display unit 150 may include a display driver 151 that controls display operations and a display panel 152 that displays images.

[0051] FIG. 2 is a diagram illustrating the method of the present disclosure.

[0052] Referring to FIGS. 1 and 2, the method for determining whether a Japanese document has been plagiarized may include steps S100, S200, S300, S400, S500, S600, S700, and S800.

[0053] In step S100, the control unit 130 divides a Japanese sentence into syntax units based on the character forms of Japanese written in a Japanese document, thereby extracting chunks corresponding to individual syntaxes.

[0054] In step S200, the control unit 130 can divide the plurality of nouns included in the chunk into individual nouns using the noun dictionary information of the noun dictionary data.

[0055] In step S300, the control unit 130 can merge chunks containing hiragana characters that are placed between characters other than hiragana characters that act as particles in the divided individual nouns with chunks that precede the hiragana characters.

[0056] In step S400, the control unit 130 can generate full-width characters based on characters other than Japanese characters written in the Japanese document. In step S400 and other embodiments, the control unit 130 can also generate full-width characters based on all characters, including Japanese characters, written in the Japanese document. For example, if the Japanese document contains "site" written in half-width katakana, the half-width katakana "site" can be replaced with the full-width character "site."

[0057] In step S500, the control unit 130 may replace a repeat code corresponding to a character repeated in a Japanese document with a character to be repeated corresponding to the repeat code.

[0058] In operation S600, the control unit 130 may exclude chunks containing symbols from the extracted chunks.

[0059] In operation S700, the control unit 130 may merge a chunk containing a number among the extracted chunks with a preceding chunk.

[0060] In step S800, the control unit 130 may generate the processed chunk as a comparison key to be utilized in plagiarism checking.

[0061] FIG. 3 is a diagram illustrating an exemplary embodiment of extracting a Japanese comparison key algorithm for determining whether a Japanese document has been plagiarized according to the present disclosure.

[0062] 1, 2, and 3, the embodiment shown in FIG. 3 can be seen in which an arbitrary Japanese sentence (EXAMPLE SENTENCE) is processed through steps 1 to 7 (STEP 1, STEP 2, STEP 3, STEP 4, STEP 5, STEP 6, STEP 7) to extract a comparison key for the Japanese sentence (EXAMPLE SENTENCE). The arbitrary Japanese sentence (EXAMPLE SENTENCE) in FIG. 3 can be referred to as a "sentence from which a comparison key is extracted." It can be seen how a comparison key for such an arbitrary Japanese sentence (EXAMPLE SENTENCE) is extracted at each step by following the arrow direction shown in FIG. 3.

[0063] The first to seventh steps (STEP 1, STEP 2, STEP 3, STEP 4, STEP 5, STEP 6, and STEP 7) may correspond to steps S100 to S700 in FIG.

[0064] In the first step (STEP 1), the processor 132 may extract a comparison key based on the Japanese character form. That is, the processor 132 extracts a comparison key by dividing a Japanese document into chunk units based on hiragana, katakana, kanji, symbols, and numbers. For example, the processor 132 may extract the chunks by dividing a Japanese sentence into individual syntax units based on the Japanese character forms of the Japanese language written in the Japanese document: hiragana, katakana, kanji, symbols, and numbers. Specifically, a Japanese sentence (example sentence) may be divided into small syntax units based on the Japanese character form. Japanese character forms may include, for example, hiragana, katakana, kanji, symbols, and numbers. The individual syntax units thus divided may be referred to as chunks. After each chunk goes through the following seven steps (STEP 1, STEP 2, STEP 3, STEP 4, STEP 5, STEP 6, STEP 7), a comparison key for each chunk can be generated. In the first step (STEP 1), the part of the Japanese sentence (EXAMPLE SENTENCE) "There are many world-class scientific journals" can be divided into "There are many world-class scientific journals" using the " / " symbol as the criterion.

[0065] In the second step (STEP 2), processor 132 can segment compound nouns contained in the Japanese document into individual noun units. That is, processor 132 can apply a noun dictionary. Specifically, the chunks extracted in the first step (STEP 1) are segmented only based on the Japanese character morphology, which differs from the Japanese phrase morphology. For example, based on a portion of the example Japanese sentence (EXAMPLE SENTENCE) ("There are numerous world-class scientific journals"), a compound noun such as "science journal" can be segmented into individual noun units such as "science" and "academic journal." When a compound noun is segmented into individual noun units in this way, this can serve as a segmentation criterion consistent with the Japanese phrase concept. Therefore, processor 132 applies a noun dictionary to chunk units segmented only based on the Japanese character morphology. If a chunk unit contains multiple nouns, processor 132 can subdivide the chunk by segmenting the compound noun into individual nouns. In the second stage (STEP 2), the sentence "There are many world-class scientific journals," which is part of the Japanese sentence (EXAMPLE SENTENCE), can be divided into "There are many world-class scientific journals" using the " / " symbol as a criterion.

[0066] In the third step (STEP 3), processor 132 may integrate the comparison key of a single hiragana character. For example, when a chunk including a single hiragana character serving as a particle is separated, processor 132 may integrate the chunks by including the single chunk in the chunk for the non-hiragana character preceding the single hiragana character. Specifically, after the first and second steps (STEP 1 and STEP 2) are performed, a single hiragana character serving as a particle may be separated into a single chunk. This may be the case when a single hiragana character serving as a particle is located between character forms other than hiragana and is separated into a single chunk. Therefore, according to the concept of phrases in Japanese, a single hiragana character serving as a particle is not separated into a single syntax, but must be grouped with the previous syntax and separated. Therefore, when a single hiragana character is between other character forms, processor 132 can integrate the single hiragana character chunk into the chunk preceding it. When the rules described above are applied in STEP 3, the "sekaiteki" and "na" in the previous example sentence (e.g., "There are many world-class scientific journals") can be integrated into "sekaiteki," and the "academic journals" and "ga" can be integrated into "academic journals ga." That is, the sentence "There are many world-class scientific journals," which is part of the Japanese sentence (EXAMPLE SENTENCE), can be divided into "there are many world-class scientific journals" using the " / " symbol as a criterion, as "there are many world-class / science / academic journals."

[0067] In step 4 (STEP 4), processor 132 may replace half-width characters with full-width characters. For example, processor 132 may replace half-width spaces, katakana, symbols, numbers, and alphabetic characters in the Japanese document with full-width characters. Specifically, not only Japanese but also English, numbers, and other characters are divided into half-width and full-width characters based on the width of the characters. Since these characters are the same but differ only in width, they must be recognized as the same character in a plagiarism testing environment. Therefore, half-width spaces, katakana, symbols, numbers, and alphabetic characters may be replaced with full-width characters. In step 4 (STEP 4), the half-width character "site" in "This is the site to be published," which is part of the Japanese sentence (EXAMPLE SENTENCE), may be replaced with the full-width character "site."

[0068] In step 5 (STEP 5), processor 132 may replace a character corresponding to a repeat symbol (dance character) with the same character. For example, if two or more repeat symbols are included in the Japanese document, processor 132 may replace all of the two or more repeat symbols with the character to be repeated that precedes the two or more repeat symbols, and combine the replaced character and the preceding character to be repeated into one chunk. Specifically, when a repeated character is created in Japanese, the same character is not created twice, and the repeated character is replaced with a repeat symbol that indicates that it is a repeated character. The repeat symbol may include at least one of the following: When a repetition code is used, the repetition code can be replaced with the character to be repeated before the repetition code. Appropriate repetition codes can be represented according to the situation. However, there are cases where such a repetition code is used, and there are also cases where the same character is created by duplication without using the repetition code. Whether the repetition code is used or not, there is only a difference in the notation method, and they mean the same character. Therefore, in a plagiarism detection environment, it is necessary to recognize the word with the repetition code used or the word without the repetition code used as the same character. Accordingly, when the repetition code is used, the processor 132 can replace the repetition code with the character to be repeated before. For example, in the case of "人々", the repetition code "々" means the previous character "人", so the processor 132 can replace "人々" with "人人". Even when two or more repetition codes are used, two or more repetition codes can be replaced with the characters that actually have meaning as described above. For example, in the case of "部分々々", the repetition code "々々" means the previous characters "部分" respectively, so "部分々々" can be replaced with "部分部分". Finally, the processor 132 can recognize the case where the repetition code is used and the case where the repetition code is not used as the same character. According to the upper rules (for example, the first to fourth steps (STEP1, STEP2, STEP3, STEP4)), the repetition code is divided into an independent chunk, so the repetition code (or the chunk of the repetition code) does not match the Japanese clause concept and can be a factor in excessively increasing the number of chunks. Since such a repetition code can be replaced with the previous character that actually has meaning, the character that actually has meaning and the repetition code can be integrally processed in one chunk. This has the effect of enabling the chunks to be divided so as to match the Japanese clause concept even when the repetition code is used. In the fifth step (STEP5), in the case of "数々ある『Scince』の" which is a part of the Japanese sentence (EXAMPLE SENTENCE), the repetition code-containing "数々" can be replaced with "数数".

[0069] In step 6 (STEP 6), processor 132 may exclude a chunk containing a symbol or a comparison key for the symbol. For example, processor 132 may combine a chunk containing the hiragana character with a chunk preceding the hiragana character to generate the full-width character, replace the repeating symbol with the character, and then exclude the chunk containing the symbol. Specifically, if a sentence contains a symbol, the symbol may also be classified as a single chunk in step 1 (STEP 1). However, even if the type of symbol used in the plagiarism detection environment is different or its availability differs, it is necessary to detect whether the text matches based on the content of the main text. If a chunk classified in steps 1 through 5 (STEP 1, STEP 2, STEP 3, STEP 4, and STEP 5) contains a symbol, processor 132 may not recognize the chunk containing the symbol as a single chunk and may exclude it. In the sixth stage (STEP 6), the part of the Japanese sentence (EXAMPLE SENTENCE) "numerous 'Sciences'" can be divided into "numerous / aru / Sciences / " based on the " / " symbol.

[0070] In step 7 (STEP 7), the processor 132 may merge the chunk containing the number with the preceding chunk after excluding the chunk containing the symbol. Specifically, if a sentence contains numbers, the numbers may be classified into a single chunk in step 1 (STEP 1). However, if numbers are frequently used in a sentence, if all numbers are classified into a single chunk, excessive plagiarism may be detected in a plagiarism detection environment. Because the same number is often used, classifying numbers into a separate chunk may unnecessarily increase the number of chunks. Therefore, if numbers are included in the chunks classified in steps 1 through 6 (STEP 1, STEP 2, STEP 3, STEP 4, STEP 5, STEP 6), the chunk containing the number may be merged into a single chunk by being included in the chunk preceding the chunk containing the number. In STEP 7, part of the Japanese sentence (EXAMPLE SENTENCE), "Top 10 people with the most paper retractions," can be divided into "Top 10 people with the most paper retractions" using the " / " symbol as a criterion.

[0071] As shown in the example of FIG. 3, a Japanese sentence can be divided into multiple chunk units through steps 1 to 7 (STEP 1, STEP 2, STEP 3, STEP 4, STEP 5, STEP 6, and STEP 7). The chunk states generated through steps 1 to 7 (STEP 1, STEP 2, STEP 3, STEP 4, STEP 5, STEP 6, and STEP 7) can correspond to the Japanese concept of phrases. The extracted chunks can then be used as comparison keys for plagiarism detection. When comparing a specific Japanese document with multiple Japanese documents, there are human and time limitations on visually comparing the contents of the documents to determine the possibility of plagiarism. To overcome these limitations, documents must be compared mechanically. To do this, documents must be divided into smaller comparison key units from document units so that they can be compared with each other. The comparison key extraction method disclosed herein can be used in this case. When extracting comparison keys from languages ​​that do not use word segments, the typically preferred method is to simply extract comparison keys based on characters. That is, there is a method for classifying a single character using a single comparison key. However, this method of extracting a character-based comparison key can detect suspected plagiarism simply by matching the characters, ignoring cases where multiple characters form a word with a single meaning. As a result, the plagiarism test results are likely to result in unconvincing false positives, suspecting plagiarism when it is not. The method for extracting a comparison key according to the present disclosure can extract a comparison key that matches the Japanese phrase concept using steps 1 through 7 (STEP 1, STEP 2, STEP 3, STEP 4, STEP 5, STEP 6, and STEP 7) as shown in Figure 3. By applying this method of extracting a comparison key to multiple Japanese documents and conducting a plagiarism test using the comparison keys, it is possible to determine which documents and content are suspected of plagiarism by comparison key. If the method of extracting a comparison key disclosed herein can be applied to a large number of plagiarism checks that are difficult to compare one by one with the naked eye, the comparison key can be extracted mechanically to perform plagiarism checks, thereby overcoming the limitations of human resources and time.Furthermore, the method of extracting comparison keys disclosed herein extracts comparison keys in units that are extremely similar to the Japanese concept of phrases, so that comparison keys can be compared in units such as words that have meaning during plagiarism testing, which has the effect of dramatically reducing the probability of false positives compared to character-by-character testing, thereby enabling more rational plagiarism testing.

[0072] Meanwhile, the disclosed embodiments may be realized in the form of a recording medium storing computer-executable instructions. The instructions may be stored in the form of program code, which, when executed by a processor, generates program modules to perform the operations of the disclosed embodiments. The recording medium may be realized as a computer-readable recording medium.

[0073] Computer-readable recording media include any type of recording media that stores computer-readable instructions, such as ROM (Read Only Memory), RAM (Random Access Memory), magnetic tape, magnetic disk, flash memory, and optical data storage devices.

[0074] As described above, the disclosed embodiments have been described with reference to the accompanying drawings. Those skilled in the art will understand that the present disclosure may be embodied in forms different from the disclosed embodiments without changing the technical idea or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be construed as limiting.

Claims

1. A comparison key generation device for determining whether a document has been plagiarized, comprising: A memory that stores an algorithm for determining whether a Japanese document has been plagiarized and noun dictionary information; a processor that uses the algorithm to determine whether the Japanese document has been plagiarized; The processor: Dividing the Japanese sentence into syntax units based on the Japanese character forms written in the Japanese document, extracting chunks corresponding to the divided syntaxes, and applying a first integration process to the extracted chunks to generate a comparison key for determining whether or not there is plagiarism; The first integration step includes: A comparison key generation device for determining whether a document has been plagiarized, comprising a step of integrating a chunk corresponding to one hiragana character that serves as a particle among the plurality of chunks with a preceding chunk.

2. The processor: The comparison key generating device for determining whether a document has been plagiarized according to claim 1 , further applying a substitution process to the plurality of chunks to generate the comparison key for determining whether a document has been plagiarized.

3. The comparison key generation device for determining whether a document has been plagiarized as described in claim 2, characterized in that the replacement process includes a process of replacing half-width characters with full-width characters for chunks among the plurality of chunks that contain half-width characters.

4. 3. The comparison key generation device for determining whether a document has been plagiarized, as described in claim 2, characterized in that the replacement step includes a step of replacing, for chunks among the plurality of chunks that contain repetitive codes, the repetitive codes with characters to be repeated that correspond to the repetitive codes.

5. The repeat code is The comparison key generating device for determining whether a document has been plagiarized according to claim 4, comprising at least one of the following:

6. The processor: Further applying an exclusion process to the plurality of chunks to generate a comparison key for determining whether or not the plagiarism has occurred; The comparison key generation device for determining whether a document has been plagiarized according to claim 1 , wherein the excluding step includes a step of excluding chunks that include symbols from the plurality of chunks.

7. The processor: A second integration process is further applied to the plurality of chunks to generate a comparison key for determining whether or not the plagiarism has occurred; 2. The comparison key generation device for determining whether a document has been plagiarized according to claim 1, wherein the second integrating step includes a step of integrating a chunk containing numbers among the plurality of chunks with a preceding chunk.

8. 2. The comparison key generation device for determining whether a document has been plagiarized according to claim 1, wherein for chunks containing multiple nouns among the multiple chunks extracted by category, the noun dictionary information is used to divide the multiple nouns into individual nouns.

9. The processor:

2. The comparison key generation device for determining whether a document is plagiarized according to claim 1, wherein the chunks are extracted by dividing the Japanese sentence into individual sentence structures based on the character forms of the Japanese language written in the Japanese document, which are hiragana, katakana, kanji, symbols, and numbers.

10. A comparison key generation method for determining whether a document has been plagiarized, which is performed by a device, comprising: A step of dividing a Japanese sentence into syntactic units based on the Japanese character forms written in the Japanese document; extracting chunks corresponding to the divided syntaxes; and applying a first integration process to the extracted chunks to generate a comparison key for determining whether the Japanese document has been plagiarized; The first integration step includes: A comparison key generation method for determining whether a document has been plagiarized, comprising a step of integrating, among the plurality of chunks, a chunk corresponding to one hiragana character that serves as a particle with a preceding chunk.

Images