Apparatus and method for determining plagiarism of japanese document

Abstract

To provide an apparatus and a method for extracting a Japanese comparison key algorithm for determining plagiarism of a Japanese document.SOLUTION: A method includes: Step S100 of extracting chunks corresponding to individual structured sentences from a Japanese text based on Japanese character forms; Step S200 of dividing a plurality of nouns included in the chunks into individual nouns using noun dictionary information; Step S300 of integrating a chunk including Hiragana (Japanese writing system) characters arranged between characters other than Hiragana characters that serve as particles with the individual nouns, with a chunk preceding Hiragana characters; Step S400 generating full-size characters on the basis of the characters; Step S500 of replacing repeated signs by characters for repetition target corresponding to the repeated signs; Step S700 of excluding chunks including symbols out of the extracted chunks, and integrating a chunk including numeric characters, out of the extracted chunks, with the previous chunk; and Step S800 of generating processed chunks, as comparison keys which are used in a plagiarism test.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an electronic device and a method for operating the same, and more particularly, to a device and method for extracting a Japanese comparison key algorithm for determining whether a Japanese document has been plagiarized. [Background technology]

[0002] Extraction of Japanese comparison keys is a method to extract Japanese comparison keys from documents written in Japanese so that they can be used for plagiarism checks with other documents. Using the extracted comparison keys as a reference, plagiarism checks can be performed between Japanese documents.

[0003] In the case of Korean and English, comparison keys are extracted based on word segments, which are divisions into phrase units, and plagiarism checks can be performed on the documents to be compared. However, Japanese does not use phrase units using word segments due to the nature of its writing system. Instead, syntactic analysis is performed on the corresponding concept of phrase units, which are not divided by word segments but by parts of speech. This means that conventional comparison key extraction techniques based on word segments cannot be applied, and new rules must be defined to suit Japanese phrase units.

[0004] In order to extract comparison keys based on Japanese phrases, rules must be defined taking into account parts of speech, but there is a problem in that building a part-of-speech dictionary for all Japanese expressions requires considerable human and time resources and is expensive to build. In addition, new expressions are constantly emerging in the language, and there is a practical limit to extracting comparison keys based on phrases through the construction of a part-of-speech dictionary that covers all expressions.

[0005] The problem of plagiarism has been a continuing issue in Japanese educational institutions, research institutes, and other environments where Japanese documents are created. In particular, the problem of plagiarism in assignments and graduation theses written by students and academic papers written by researchers at Japanese educational institutions such as universities has become a hot topic in the media, and the need for countermeasures has been raised in Japanese society.

[0006] Retraction Watch, a website that publishes information about the retraction of scientific papers from the global scientific journal Science, also points out that Japan accounts for 50% of the top 10 researchers with the most retractions of papers.

[0007] The problem of plagiarism is not only a problem in Japan, but also in many countries around the world, including Korea, and in response to this, text plagiarism detection technology using plagiarism detection programs is being developed. However, the comparative key extraction method for phrase units based on word segments used in Korean and English-speaking countries for plagiarism detection cannot be applied to the Japanese language environment, which does not use word segments. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] JP 2023-023589 A Summary of the Invention [Problem to be solved by the invention]

[0009] The present disclosure has been made in consideration of the above circumstances, and its purpose is to provide an apparatus and method for setting a definition of plagiarism techniques for works written in Japanese by extracting comparison keys based on Japanese phrases, which are concepts corresponding to phrases.

[0010] The problems to be solved by the present disclosure are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0011] In one aspect, the present disclosure provides an apparatus for determining whether a Japanese document has been plagiarized, the apparatus comprising: a memory for storing code data relating to an algorithm for determining whether the Japanese document has been plagiarized, noun dictionary data including noun dictionary information; and a processor configured to determine whether the Japanese document has been plagiarized using the algorithm of the code data. The processor extracts chunks corresponding to individual syntax by dividing a Japanese sentence into syntactic units based on the character forms of Japanese written in the Japanese document, divides a plurality of nouns included in the chunks into individual nouns using noun dictionary information of the noun dictionary data, merges chunks containing hiragana characters arranged between characters other than hiragana characters that act as particles in the divided individual nouns with chunks preceding the hiragana characters, generates full-width characters based on characters written in the Japanese document, replaces repetition signs corresponding to characters repeated in the Japanese document with characters to be repeated corresponding to the repetition signs, excludes chunks containing symbols from the extracted chunks, merges chunks containing numbers from the extracted chunks with the preceding chunk, and generates the processed chunks as a comparison key to be used in plagiarism checks.

[0012] In another aspect, a method for determining whether a Japanese document has been plagiarized according to the present disclosure includes: extracting chunks corresponding to individual syntax by dividing a Japanese sentence into syntax units based on Japanese character forms described in the Japanese document; dividing a plurality of nouns included in the chunks into individual nouns using noun dictionary information of the noun dictionary data; merging a chunk including a hiragana character arranged between characters other than hiragana characters that serve as particles in the divided individual nouns with a chunk preceding the hiragana character; generating full-width characters based on characters described in the Japanese document; replacing a repetition code corresponding to a character repeated in the Japanese document with a character to be repeated corresponding to the repetition code; excluding chunks including symbols from the extracted chunks; merging chunks including numbers from the extracted chunks with a preceding chunk; and generating the processed chunks as a comparison key to be used in a plagiarism check. Effect of the Invention

[0013] The present disclosure has the effects of reducing human and time resources, determining whether or not plagiarism has occurred more accurately, dramatically reducing the probability of false positives, and enabling rational plagiarism testing.

[0014] The effects of the present disclosure are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 illustrates an apparatus according to the present disclosure. [Diagram 2] FIG. 1 illustrates the method of the present disclosure. [Diagram 3] FIG. 13 is a diagram illustrating an exemplary embodiment of the present disclosure for extracting a Japanese comparison key algorithm for determining whether a Japanese document has been plagiarized. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] The same reference numerals refer to the same components throughout this disclosure. This disclosure does not describe all the components of the embodiments, and general content in the technical field to which this disclosure pertains or content that overlaps in the embodiments will be omitted. The term "part, module, member, block" used in the specification can be realized as software or hardware, and depending on the embodiment, multiple "parts, modules, members, blocks" can be realized as one component, or one "part, module, member, block" can include multiple components.

[0017] Throughout the specification, when a part is said to be "connected" to another part, this includes not only direct connection but also indirect connection, including connection via a wireless communication network.

[0018] Furthermore, when a part is described as "comprising" a certain component, this does not mean that other components are excluded, unless specifically stated to the contrary, but rather that the part may further include other components.

[0019] Throughout this specification, when an element is referred to as being "on" another element, this includes not only when the element is in contact with the other element, but also when there is another element between the two elements.

[0020] The terms "first", "second", etc. are used to distinguish one component from another component, and the components are not limited to the terms described above.

[0021] The singular expression includes the plural expression unless there is a clear exception in the context.

[0022] The reference numbers in each step are used for convenience of description, and do not describe the order of each step; each step may be performed in a different order than specified, unless the context clearly dictates a specific order.

[0023] Hereinafter, the working principle and embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0024] In this specification, the term "device according to the present disclosure" includes all of the various devices that can perform computation and provide a result to a user. For example, the device according to the present disclosure may include all of a computer, a server device, and a portable terminal, or may be in any one of the forms.

[0025] Here, the computer may include, for example, a notebook computer, a desktop computer, a laptop computer, a tablet PC, a slate PC, etc. equipped with a web browser.

[0026] The server device is a server that communicates with external devices and processes information, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, a web server, etc.

[0027] The portable terminal may be, for example, a wireless communication device that ensures portability and mobility, and may include any kind of handheld-based wireless communication device such as a personal communication system (PCS), a global system for mobile communications (GSM), a personal digital cellular (PDC), a personal handyphone system (PHS), a personal digital assistant (PDA), an international mobile telecommunication (IMT)-2000, a code division multiple access (CDMA)-2000, a W-code division multiple access (W-CDMA), a wireless broadband internet (WiBro) terminal, a smartphone, etc., and a wearable device such as a watch, a ring, a bracelet, an anklet, a necklace, glasses, a contact lens, or a head-mounted device (HMD).

[0028] The artificial intelligence function according to the present disclosure is operated by a processor and a memory. The processor may be composed of one or more processors. In this case, the one or more processors may be a general-purpose processor such as a CPU, AP, DSP (Digital Signal Processor), etc., a processor dedicated to graphics such as a GPU, VPU (Vision Processing Unit), or a processor dedicated to artificial intelligence such as an NPU. The one or more processors control the processing of input data according to predefined operating rules or artificial intelligence models stored in the memory. Alternatively, when the one or more processors are processors dedicated to artificial intelligence, the processor dedicated to artificial intelligence may be designed into a hardware structure specialized for processing a specific artificial intelligence model.

[0029] FIG. 1 is a diagram illustrating the device of the present disclosure.

[0030] Referring to FIG. 1, the device 100 of the present disclosure may generate a comparison key for determining whether a Japanese document is plagiarized and calculate a plagiarism rate of the Japanese document. The comparison key may be used for plagiarism inspection. The device 100 may extract a comparison key based on a word segment, which is a division point of a phrase, using a comparison key extraction method for plagiarism inspection. Word segments may be used to divide meaningful sentence structures in Korean and English. In contrast, word segments are not used in Japanese, and hiragana, katakana, and kanji are mixed to make the meaning easily comprehensible. Although hiragana, katakana, and kanji are mixed in Japanese to divide meaningful sentence structures, the character form alone may not completely match the semantic segmentation criteria of Japanese. Therefore, the device 100 may extract a comparison key based on a Japanese phrase and / or a Japanese character form in consideration of the characteristics of Japanese, and apply a separate comparison key extraction condition to define a rule that is close to the Japanese phrase concept to extract a comparison key.

[0031] The device 100 may include an input unit 110, a communication unit 120, a control unit 130, an audio output unit 140, a display unit 150, and the like.

[0032] The input unit 110 is for inputting video information (or signals), audio information (or signals), data, or information input by a user, and may include at least one of at least one camera, at least one microphone, and a user input unit. The audio data and image data collected by the input unit 110 may be analyzed and processed according to a user's control command.

[0033] The communication unit 120 may include one or more components that enable communication with an external device, and may include, for example, at least one of a wired communication module 121 and a wireless communication module 122.

[0034] The wired communication module 121 may include various wired communication modules such as a local area network (LAN) module, a wide area network (WAN) module, or a value added network (VAN) module, as well as various cable communication modules such as a universal serial bus (USB), a high definition multimedia interface (HDMI) (registered trademark), a digital visual interface (DVI), recommended standard 232 (RS-232), power line communication, or plain old telephone service (POTS).

[0035] The wireless communication module 122 may include a wireless communication module 122 that supports various wireless communication methods such as a Wifi (registered trademark) module, a WiBro (wireless broadband) module, a Global System for Mobile Communication (GSM), a Code Division Multiple Access (CDMA), a Wideband Code Division Multiple Access (WCDMA), a universal mobile telecommunications system (UMTS), a Time Division Multiple Access (TDMA), a Long Term Evolution (LTE), 4G, 5G, and 6G.

[0036] The wireless communication module 122 may include an antenna and a wireless communication interface, and may further include a signal conversion module for modulating a digital control signal output from the controller via the wireless communication interface into an analog wireless signal under the control of the controller and / or demodulating an analog wireless signal received via the wireless communication interface into a digital control signal.

[0037] The control unit 130 may include a memory 131 that stores data for an algorithm or a program that reproduces the algorithm for controlling the operation of components in the device 100, and at least one processor 132 that performs the above-mentioned operations using the data stored in the memory 131. In this case, the memory 131 and the processor 132 may be realized as separate chips, or the memory 131 and the processor 132 may be realized as a single chip.

[0038] The memory 131 may store data supporting various functions of the device 100 and programs for the operation of the control unit 130, may store input / output data (e.g., music files, still images, videos, etc.), may store a plurality of application programs (or applications) run by the device 100, and may store data and commands for the operation of the device 100. At least some of these application programs may be downloaded from an external server via wireless communication. Such memory 131 may include at least one of the following types of storage media: flash memory type, hard disk type, solid state disk type, silicon disk drive type, multimedia card micro type, card-type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, and optical disk. Also, memory 131 may be a database separate from device 100 but connected by wire or wirelessly.

[0039] The memory 131 can store code data related to an algorithm for determining whether a Japanese document has been plagiarized. The memory 131 can also store noun dictionary data including noun dictionary information. The noun dictionary information can include information related to a noun dictionary.

[0040] The processor 132 can be configured to use an algorithm of the code data to determine whether the Japanese document has been plagiarized.

[0041] The processor 132 may extract a comparison key based on the character form of Japanese. For example, the processor 132 may divide a Japanese document into chunk units for each of hiragana, katakana, kanji, symbols, and numbers. As another example, the processor 132 may extract chunks corresponding to individual syntax by dividing a Japanese sentence into syntax units based on the character form of Japanese written in the Japanese document.

[0042] The processor 132 can apply a noun dictionary. For example, when a chunk unit includes multiple nouns, the processor 132 can subdivide the chunk into individual nouns. As another example, the processor 132 can use noun dictionary information of the noun dictionary data to divide multiple nouns included in the chunk into individual nouns.

[0043] Processor 132 can process comparison keys of one hiragana character in an integrated manner. For example, processor 132 can group and segment a one hiragana character that serves as a particle with a preceding construction. As another example, processor 132 can merge a chunk that includes a hiragana character that is located between characters other than a hiragana character that serves as a particle in a segmented individual noun with a chunk that precedes the hiragana character.

[0044] The processor 132 can replace half-width characters with full-width characters. For example, the processor 132 can generate full-width characters based on characters written in the Japanese document. As another example, the processor 132 can replace identical characters (full-width characters and / or half-width characters) or characters with full-width characters, since the widths of the characters may differ. For example, if the Japanese document contains "site" in half-width katakana, the half-width katakana "site" can be replaced with the full-width character "site". However, the present invention is not limited to this, and symbols and numbers other than Japanese can also be included as targets for replacement with full-width characters.

[0045] The processor 132 can replace a character corresponding to a repeat code with a character corresponding to its meaning. For example, the processor 132 can replace a repeat code that indicates a repeated character with the same character in the previous paragraph. As another example, the processor 132 can replace a repeat code that indicates a character that is repeated in a Japanese document with the character to be repeated that corresponds to the repeat code.

[0046] The processor 132 can perform processing operations that exclude comparison keys for symbols. For example, if a sentence contains symbols, the processor 132 can exclude chunks that contain symbols from among the extracted chunks, rather than recognize the symbols contained in the sentence as one chunk.

[0047] Processor 132 may process numeric comparison keys in an integrated manner. For example, if a sentence contains numbers, processor 132 may integrate extracted chunks containing numbers with the preceding chunks by including the extracted chunks containing numbers in the chunks preceding the numbers.

[0048] The processor 132 may generate chunks processed by the above-described operations as a comparison key to be utilized in plagiarism checking.

[0049] The audio output unit 140 may output audio data received via the communication unit 120 or stored in the memory 131, or may output an audio signal related to a function performed by the device 100. The audio output unit 140 may include an amplifier 141, a speaker 142, and the like.

[0050] The display unit 150 displays (outputs) information processed by the device 100. For example, the display unit 150 may display execution screen information of an application program (for example, an application) run by the device 100, or UI (User Interface) or GUI (Graphic User Interface) information based on the execution screen information. The display unit 150 may include a display driver 151 that controls a display operation and a display panel 152 that displays an image.

[0051] FIG. 2 is a diagram illustrating the method of the present disclosure.

[0052] Referring to FIG. 1 and FIG. 2, a method for determining whether a Japanese document has been plagiarized may include steps S100, S200, S300, S400, S500, S600, S700, and S800.

[0053] In step S100, the control unit 130 may extract chunks corresponding to individual sentences by dividing a Japanese sentence into sentence units based on the character forms of Japanese written in a Japanese document.

[0054] In step S200, the control unit 130 may divide the nouns included in the chunk into individual nouns using noun dictionary information of the noun dictionary data.

[0055] In step S300, the control unit 130 may merge a chunk including a hiragana character that is placed between characters other than a hiragana character that serves as a particle in a divided individual noun with a chunk preceding the hiragana character.

[0056] In step S400, the control unit 130 may generate full-width characters based on non-Japanese characters written in the Japanese document. In step S400 and other embodiments, the control unit 130 may generate full-width characters based on all characters, including Japanese, written in the Japanese document. For example, if the Japanese document includes "site" in half-width katakana, the half-width katakana "site" may be replaced with the full-width character "site."

[0057] In step S500, the control unit 130 may replace a repeat code corresponding to a character repeated in a Japanese document with a character to be repeated corresponding to the repeat code.

[0058] In operation S600, the control unit 130 may exclude chunks including symbols from among the extracted chunks.

[0059] In operation S700, the control unit 130 may merge a chunk including a number among the extracted chunks with a preceding chunk.

[0060] In operation S800, the control unit 130 may generate the processed chunk as a comparison key to be utilized in plagiarism checking.

[0061] FIG. 3 is a diagram illustrating an exemplary embodiment of the present disclosure for extracting a Japanese comparison key algorithm for determining whether a Japanese document has been plagiarized.

[0062] 1, 2 and 3, the embodiment shown in FIG. 3 shows that an arbitrary Japanese sentence (EXAMPLE SENTENCE) is processed through steps 1 to 7 (STEP 1, STEP 2, STEP 3, STEP 4, STEP 5, STEP 6, STEP 7) to extract a comparison key for the Japanese sentence (EXAMPLE SENTENCE). The arbitrary Japanese sentence (EXAMPLE SENTENCE) in FIG. 3 can be called a "sentence from which a comparison key is extracted." It can be seen how a comparison key for such an arbitrary Japanese sentence (EXAMPLE SENTENCE) is extracted for each step along the arrow direction shown in FIG. 3.

[0063] The first to seventh steps (STEP 1, STEP 2, STEP 3, STEP 4, STEP 5, STEP 6, and STEP 7) may correspond to steps S100 to S700 in FIG.

[0064] In the first step (STEP 1), the processor 132 may extract a comparison key based on the character form of Japanese. That is, the processor 132 may extract a comparison key by dividing a Japanese document into chunk units based on hiragana, katakana, Chinese characters, symbols, and numbers. For example, the processor 132 may extract the chunks by dividing the Japanese sentence into individual syntaxes based on the character forms of Japanese written in the Japanese document, that is, hiragana, katakana, Chinese characters, symbols, and numbers. To be more specific, a Japanese sentence (EXAMPLE SENTENCE) may be divided into small syntax units based on the character form of Japanese. The character forms of Japanese may include, for example, hiragana, katakana, Chinese characters, symbols, and numbers. The individual syntaxes divided in this way may be called chunks. After each chunk goes through the first through seventh steps (STEP1, STEP2, STEP3, STEP4, STEP5, STEP6, STEP7), a comparison key for each chunk can be generated. In the first step (STEP1), a part of a Japanese sentence (EXAMPLE SENTENCE), "There are many world-class scientific journals," can be classified as "There are many world-class scientific journals" using the " / " symbol as the criterion.

[0065] In the second step (STEP 2), the processor 132 can classify compound nouns written in the Japanese document into individual noun units. That is, the processor 132 can apply a noun dictionary. More specifically, the chunks extracted in the first step (STEP 1) are classified only based on the character form of Japanese, which is different from the phrase criterion of Japanese. For example, based on a part of the example Japanese sentence (EXAMPLE SENTENCE) ("There are many world-class scientific journals"), a compound noun such as "science academic journal" can be classified into individual noun units such as "science" and "academic journal". When a compound noun is classified into individual noun units in this way, it can be a classification criterion that matches the phrase concept of Japanese. Therefore, the processor 132 applies a noun dictionary to the chunk units classified only based on the character form of Japanese, and when one chunk unit includes multiple nouns, the processor 132 can subdivide the chunk by dividing the compound noun into individual nouns. In the second stage (STEP 2), the sentence "There are many world-class scientific journals," which is part of a Japanese sentence (EXAMPLE SENTENCE), can be categorized as "There are many world-class scientific journals" based on the " / " symbol.

[0066] In the third step (STEP 3), the processor 132 may integrate the comparison key of one hiragana character. For example, when a chunk including one hiragana character serving as a particle is divided, the processor 132 may integrate the chunks by including the one chunk in a chunk for a character other than the hiragana character preceding the one hiragana character. To be more specific, after the first and second steps (STEP 1, STEP 2) are performed, a case may occur in which one hiragana character serving as a particle is divided into one chunk. This may correspond to a case in which the one hiragana character serving as a particle is between character forms other than hiragana and is divided into one chunk. Therefore, according to the phrase concept in Japanese, the one hiragana character serving as a particle is not divided into one syntax, but must be grouped together with the previous syntax and divided. Therefore, when a single hiragana character is between other character forms, processor 132 can process the chunk of the single hiragana character by including the chunk of the single hiragana character in the chunk preceding the chunk of the single hiragana character and integrating them into one chunk. When the above-mentioned rules are applied in the third stage (STEP 3), "sekai-teki" and "na" in the previous example sentence (e.g., "There are many world-class scientific journals") can be integrated into "sekai-teki", and "academic journals" and "ga" can be integrated into "academic journals ga". In other words, "There are many world-class scientific journals", which is part of a Japanese sentence (EXAMPLE SENTENCE), can be classified as "There are many world-class / science / academic journals" based on the " / " symbol.

[0067] In the fourth step (STEP 4), the processor 132 can replace half-width characters with full-width characters. For example, the processor 132 can replace half-width characters such as spaces, katakana, symbols, numbers, and alphabets written in the Japanese document with full-width characters. To be more specific, not only Japanese but also English, numbers, etc. are divided into half-width characters and full-width characters based on the width of the characters. Since such characters are the same character but differ only in the width of the characters, such characters need to be recognized as the same character in the plagiarism inspection environment. Therefore, half-width spaces, katakana, symbols, numbers, and alphabets can be replaced with full-width characters. In the fourth step (STEP 4), the half-width character "site" in "This is a site to be published," which is part of the Japanese sentence (EXAMPLE SENTENCE), can be replaced with the full-width character "site."

[0068] In the fifth step (STEP5), the processor 132 can replace the character corresponding to the repeat code (dance character) with the same character. For example, when two or more repeat codes are described in the Japanese document, the processor 132 can replace all of the two or more repeat codes with the character to be repeated that precedes the two or more repeat codes, and integrate the replaced character and the preceding character to be repeated into one chunk. To be more specific, when a repeated character is created in Japanese, the same character is not created in duplicate, and the repeated character is created by replacing it with a repeat code that means it is a repeated character. The repeat code can include at least one of the following: When a repetition code is used, the repetition code can be replaced with the character to be repeated before the repetition code through a replacement process. Appropriate repetition codes can be represented according to the situation. However, there are cases where such repetition codes are used, and there are also cases where the same character is created by duplication without using the repetition code. Whether the repetition code is used or not, there is only a difference in the notation method, but they mean the same character. Therefore, in a plagiarism detection environment, it is necessary to recognize words with repetition codes and words without repetition codes as the same character. Accordingly, when a repetition code is used, the processor 132 can replace the repetition code with the character to be repeated before. For example, in the word "人々" (people), the repetition code "々" means the previous character "人" (person), so the processor 132 can replace "人々" with "人人". Even when two or more repetition codes are used, the two or more repetition codes can be replaced with the characters that actually have meaning as described above. For example, in the word "部分々々" (parts), the repetition code "々々" means the previous characters "部分" (part) respectively, so "部分々々" can be replaced with "部分部分". Finally, the processor 132 can recognize the case where the repetition code is used and the case where the repetition code is not used as the same character. According to the upper rules (for example, the first to fourth steps (STEP1, STEP2, STEP3, STEP4)), the repetition code is divided into an independent chunk. Therefore, the repetition code (or the chunk of the repetition code) does not conform to the Japanese clause concept and may be a factor in excessively increasing the number of chunks. Since such a repetition code can be replaced with the previous character that actually has meaning, the character that actually has meaning and the repetition code can be integrally processed in one chunk. This has the effect of enabling the chunks to be divided so as to conform to the Japanese clause concept even when the repetition code is used. In the fifth step (STEP5), in the part of the Japanese sentence (EXAMPLE SENTENCE) "数々ある『Scince』の" (numerous "Scince"), the repetition code-containing "数々" (numerous) can be replaced with "数数".

[0069] In the sixth step (STEP6), the processor 132 may exclude a comparison key of a symbol or a chunk of a symbol. For example, the processor 132 may combine the chunk including the hiragana character with the chunk preceding the hiragana character to generate the full-width character, replace the repeating code with the character, and then exclude the chunk including the symbol. To be more specific, if a text includes a symbol, the symbol may also be classified into one chunk in the first step (STEP1). However, even if the types of symbols used in the plagiarism inspection environment are different or the availability of the symbols is different, it is necessary to detect whether the text matches with the content of the main text. If a symbol is included in the chunk classified in the first to fifth steps (STEP1, STEP2, STEP3, STEP4, STEP5), the processor 132 may not recognize the chunk including the symbol as one chunk and may exclude it. In the sixth stage (STEP 6), the sentence "There are many 'Sciences'", which is part of a Japanese sentence (EXAMPLE SENTENCE), can be divided into "There are many Sciences" based on the " / " symbol.

[0070] In the seventh step (STEP 7), the processor 132 may integrate the chunk containing the number with the preceding chunk after excluding the chunk containing the symbol. Specifically, if the sentence contains numbers, the numbers may be classified into one chunk in the first step (STEP 1). However, if numbers are frequently used in a sentence and all numbers are classified into one chunk, there is a possibility that excessive plagiarism may be recognized in a plagiarism inspection environment. Since the same numbers are often used, if the numbers are classified into a single chunk, the number of chunks may increase unnecessarily. Therefore, if numbers are included in the chunks classified in the first to sixth steps (STEP 1, STEP 2, STEP 3, STEP 4, STEP 5, STEP 6), the chunk containing the number may be integrated into one chunk by being included in the chunk preceding the chunk containing the number. In STEP 7, part of the Japanese sentence (EXAMPLE SENTENCE), "Top 10 people with the most paper retractions," can be categorized as "Top 10 people with the most paper retractions" using the " / " symbol as the criterion.

[0071] As shown in the embodiment of FIG. 3, a Japanese text can be divided into a plurality of chunk units by the first to seventh steps (STEP1, STEP2, STEP3, STEP4, STEP5, STEP6, STEP7). The state of the chunks generated by the first to seventh steps (STEP1, STEP2, STEP3, STEP4, STEP5, STEP6, STEP7) can correspond to the Japanese phrase concept. The extracted chunks can be used as comparison keys to be utilized in plagiarism inspection. When comparing a specific Japanese document with a plurality of Japanese documents to be compared, there are human and time limitations in comparing the contents of the documents with the naked eye and determining the possibility of plagiarism. In order to solve such limitations, it is necessary to mechanically compare the documents. To do so, it is necessary to divide the documents into smaller comparison key units from document units so that they can be compared with each other. In this case, the comparison key extraction method disclosed herein can be utilized. When extracting a comparison key from a language that does not use word segments, the method that is usually considered to be prioritized is to simply extract a comparison key based on characters. That is, there is a method of classifying one character with one comparison key. However, such a method of extracting a character-based comparison key can detect suspected plagiarism if the characters match, ignoring cases where multiple characters form a word with one meaning. As a result, there is a high possibility that the result of the plagiarism test will be an unconvincing over-detection of plagiarism, where plagiarism is suspected when it is not. The method of extracting a comparison key according to the present disclosure can extract a comparison key that matches the Japanese phrase concept by using the first to seventh steps (STEP 1, STEP 2, STEP 3, STEP 4, STEP 5, STEP 6, STEP 7) as shown in FIG. 3. By applying such a method of extracting a comparison key to multiple Japanese documents and performing a plagiarism test using the comparison keys, it is possible to confirm which documents and contents are suspected of plagiarism on a comparison key basis. If the method of extracting a comparison key disclosed in the present invention is applied to a large amount of plagiarism checking that is difficult to compare one by one with the naked eye, the comparison key can be extracted mechanically to perform plagiarism checking, thereby resolving the human and time limitations.In addition, the method of extracting comparison keys disclosed herein extracts comparison keys in units that are extremely similar to the concept of phrases in Japanese, so that comparison keys can be compared in units of words or other meaningful words during plagiarism checking, which has the effect of dramatically reducing the probability of false positives compared to character-by-character checking, thereby enabling rational plagiarism checking.

[0072] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing instructions executable by a computer. The instructions may be stored in the form of program code, which, when executed by a processor, may generate program modules to perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0073] The computer-readable recording medium includes any type of recording medium in which computer-readable instructions are stored, such as a read only memory (ROM), a random access memory (RAM), a magnetic tape, a magnetic disk, a flash memory, an optical data storage device, and the like.

[0074] As described above, the disclosed embodiments have been described with reference to the accompanying drawings. Those skilled in the art will understand that the present disclosure may be embodied in a form different from the disclosed embodiments without changing the technical idea or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be interpreted as being limited.

Claims

1. A device for determining whether a Japanese document has been plagiarized, comprising: A memory for storing code data relating to an algorithm for determining whether or not the Japanese document has been plagiarized and noun dictionary data including noun dictionary information; A processor configured to determine whether the Japanese document has been plagiarized using the algorithm of the code data, The processor, Dividing the Japanese sentence into syntactic units based on the character forms of Japanese written in the Japanese document, and extracting chunks corresponding to individual syntactic units; Dividing the plurality of nouns included in the chunk into individual nouns using noun dictionary information of the noun dictionary data; a chunk including a hiragana character that is located between characters other than a hiragana character that serves as a particle in the divided individual noun is merged with a chunk preceding the hiragana character; generating full-width characters based on characters written in the Japanese document; Replace a repeat code corresponding to a character repeated in the Japanese document with a character to be repeated that corresponds to the repeat code; Among the extracted chunks, chunks containing symbols are excluded. Among the extracted chunks, chunks that contain numbers are merged with the preceding chunk, The apparatus generates the processed chunks as a comparison key to be utilized in plagiarism checking.

2. The processor, The device of claim 1, wherein the chunks are extracted by dividing the Japanese document into separate syntax structures based on the character forms of the Japanese language written in the Japanese document, namely hiragana, katakana, kanji, symbols, and numbers.

3. The processor, 3. The apparatus according to claim 2, wherein compound nouns contained in said Japanese document are divided into individual noun units.

4. The processor, The device described in claim 3, characterized in that when dividing a chunk containing one hiragana character that serves as the particle, the chunk is integrated by including the one chunk in a chunk for characters other than the hiragana character that precede the one hiragana character.

5. The processor, 5. The device according to claim 4, wherein half-width spaces, katakana, symbols, numbers, and alphabets written in the Japanese document are replaced with the full-width characters.

6. The processor, When two or more repeating symbols are described in the Japanese document, the two or more repeating symbols are all replaced with the character to be repeated that precedes the two or more repeating symbols; 6. The apparatus of claim 5, further comprising merging the replaced character and the preceding repeated character into one chunk.

7. The repeat code is and The processor, 7. The apparatus of claim 6, wherein when the repeat code is used, the repeat code is replaced with the character to be repeated before the repeat code.

8. The processor, The device according to claim 7, characterized in that the chunk containing the hiragana character is merged with a chunk preceding the hiragana character, the full-width character is generated, the repeating code is replaced with the character, and then the chunk containing the symbol is excluded.

9. The processor, The apparatus according to claim 8, characterized in that after the chunk containing the symbol is removed, the chunk containing the number is merged with the preceding chunk.

10. A method for determining whether a Japanese document has been plagiarized, comprising: extracting chunks corresponding to individual syntax by dividing a Japanese sentence into syntax units based on Japanese character forms written in the Japanese document; dividing the plurality of nouns included in the chunk into individual nouns using noun dictionary information of the noun dictionary data; a step of merging a chunk including a hiragana character that is located between characters other than the hiragana character that serves as a particle in the divided individual noun with a chunk preceding the hiragana character; generating full-width characters based on characters written in the Japanese document; replacing a repeat code corresponding to a character repeated in the Japanese document with a character to be repeated corresponding to the repeat code; removing chunks including symbols from the extracted chunks; merging the extracted chunks containing numbers with the preceding chunk; and generating the processed chunks as a comparison key for use in plagiarism checking.

Images