Method and system for extracting product review keywords based on language models

Abstract

This disclosure provides a language model-based product review keyword extraction method, which is performed by at least one processor. The method includes the steps of: collecting product-related review data based on information for identifying the product; generating at least one response data from at least a portion of the review data based on a predetermined set of questions using a language model; and extracting product-related review keywords based on the at least one response data.

Description

Technical Field

[0005] ,

[0001] The present disclosure relates to a method and system for extracting product review keywords. Specifically, it relates to a method and system for generating answer data based on a plurality of predetermined questions from review data related to a product using a language model, and extracting keywords based on the answer data.

Background Art

[0002] Recently, as online transactions of products increase, the types of products traded online are also diversifying. Therefore, before purchasing a product, potential buyers often refer to the reviews of buyers who have purchased the product. If there are no reviews for the product, potential buyers may hesitate to purchase even if the price is lower compared to similar products. Thus, when purchasing products online, product reviews have a great impact on the purchase decision.

[0003] Buyers can access product reviews through blogs, Internet cafes, product review comments on online shopping malls (or smart stores), etc. However, not all product reviews are reliable. Reviews exaggerating the merits of products or promotional reviews are also posted online for the purpose of advertising products. Also, as the amount of product reviews that can be searched online becomes huge, sellers and potential buyers may spend a lot of time and effort searching for product information that meets their needs.

Summary of the Invention

Problems to be Solved by the Invention

[0004] [[ID=​​​​​​​This disclosure can be implemented in various ways, including methods, apparatus (systems), or computer programs recorded on a readable storage medium.

[0006] According to one embodiment of the present disclosure, a language model-based product review keyword extraction method, performed by at least one processor, may include the steps of: collecting product-related review data based on information for identifying a product; generating at least one response data from at least a portion of the review data based on a predetermined set of questions using a language model; and extracting product-related review keywords based on the at least one response data.

[0007] This disclosure provides a computer-readable, non-temporary recording medium that stores instruction words for executing a product review keyword extraction method according to one embodiment of this disclosure.

[0008] A system according to one embodiment of the present disclosure includes a communication module, a memory, and at least one processor connected to the memory and configured to execute at least one computer-readable program contained in the memory, wherein the at least one program may include instructions for collecting product-related review data based on information for identifying the product, generating at least one answer data from at least a portion of the review data based on a set of predetermined questions using a language model, and extracting product-related review keywords based on the at least one answer data. [Effects of the Invention]

[0009] According to various embodiments of this disclosure, users can easily find review keywords for a product simply by entering information to identify that product. This allows users to easily understand and analyze information or characteristics of relevant brands and products based on product reviews from other users who have purchased or used the product. Furthermore, providing review keywords can help users decide whether or not to purchase a product.

[0010] According to various embodiments of this disclosure, spam / promotional review data can be removed from the review data using a passage-type classifier, thereby simplifying the nature of the review data when extracting review keywords. In other words, only informational review data can be extracted from the review data and used in the subsequent review keyword extraction step. This reduces the uncertainty of the review keywords extracted in the review keyword extraction step and improves the reliability of the review keywords.

[0011] According to various embodiments of this disclosure, it is possible to determine whether the review data contains answers corresponding to at least some of a predetermined set of question data. In other words, the characteristics of the information contained in the review data can be determined in advance. This reduces uncertainty in the review keyword extraction step and improves the reliability of the review keywords.

[0012] According to various embodiments of this disclosure, the quality of the output response data can be improved by using an additional generative model in addition to the generative model that extracts response data corresponding to question data from review data. Furthermore, the efficiency of response data acceptance can be improved by accepting only a portion of the response data extracted by the generative model and using that portion to train the additional generative model, rather than accepting all of it.

[0013] According to various embodiments of this disclosure, the accuracy or reliability of the extracted review keywords can be improved by post-processing the response data extracted from the review data in correspondence with the question data. Furthermore, the reliability of the final extracted review keywords can be increased by reflecting at least a portion of the sentences from the original review data in the review keywords.

[0014] According to various embodiments of this disclosure, users can easily view review keywords associated with a product simply by entering product information. This allows potential buyers to use the outputted review keywords as a reference when deciding whether to purchase the product they desire. Furthermore, the outputted review keywords can also be used as a tool for analyzing brand / product reputation.

[0015] The effects of this disclosure are not limited to those described above, and any other effects not mentioned would be clearly understood by a person with ordinary skill in the art to which this disclosure pertains ("ordinary art") from the claims. [Brief explanation of the drawing]

[0016] Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, where similar reference numbers represent similar elements, but are not limited thereto. [Figure 1] An example of a product review keyword extraction method provided by one embodiment of this disclosure is shown. [Figure 2] This is a schematic diagram showing a configuration in which an information processing system is connected to communicate with multiple user terminals in order to extract review keywords for a product according to one embodiment of this disclosure. [Figure 3] This is a block diagram showing the internal configuration of a user terminal and information processing system according to one embodiment of the present disclosure. [Figure 4] This figure shows an example of a procedure for collecting review data according to one embodiment of the present disclosure. [Figure 5]A diagram showing an example of filtering review data according to its nature according to an embodiment of the present disclosure. [Figure 6] A diagram showing an example of a procedure for extracting review keywords from review data according to an embodiment of the present disclosure. [Figure 7] A diagram showing an example of preprocessing review data according to an embodiment of the present disclosure. [Figure 8] A diagram showing an example of learning a language model for generating answer data according to an embodiment of the present disclosure. [Figure 9] A diagram showing an example of postprocessing answer data generated according to an embodiment of the present disclosure. [Figure 10] A diagram showing an example of review keywords according to an embodiment of the present disclosure. [Figure 11] A flowchart showing an example of a method according to an embodiment of the present disclosure.

Embodiments for Carrying Out the Invention

[0017] Hereinafter, specific contents for implementing the present disclosure will be specifically described with reference to the accompanying drawings. However, in the following description, specific descriptions of well-known functions and configurations may be omitted when there is a risk of unnecessarily obscuring the gist of the present disclosure.

[0018] In the accompanying drawings, the same or corresponding components are denoted by the same reference numerals. Also, in the description of the following embodiments, duplicate descriptions of the same or corresponding components can be omitted. However, even if the description of a component is omitted, it is not intended that the component is not included in any embodiment.

[0019] The advantages and features of the disclosed embodiments, and the methods for achieving them, will become clear by referring to the embodiments described below together with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, and can be realized in various different forms. Merely these embodiments make the present disclosure complete, and the present disclosure is provided for a person of ordinary skill in the art to fully understand the scope of the invention.

[0020] The terms used in this specification will be briefly explained, and the disclosed embodiments will be specifically described. The terms used in this specification are, as much as possible, general terms that are currently widely used in consideration of the functions in the present disclosure. However, this may change depending on the intentions and precedents of those skilled in the relevant art, the emergence of new technologies, etc. Also, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be specifically described in the explanatory part of the corresponding invention. Therefore, the terms used in the present disclosure should not be merely the names of the terms, but should be defined based on the meaning of the terms and the entire content of the present disclosure.

[0021] In this specification, singular expressions include plural expressions unless it is specifically specified in the context that they are singular. Also, plural expressions include singular expressions unless it is specifically specified in the context that they are plural. Throughout the specification, when a certain part is described as including a certain component, this means that, unless there is a contrary description, it does not exclude other components, but can further include other components.

[0022] Furthermore, the terms “module” or “part” as used in this specification refer to a component of software or hardware, and “module” or “part” plays a role. However, “module” or “part” is not limited to software or hardware. A “module” or “part” may be configured on an addressable storage medium and may be configured to regenerate one or more processors. Thus, as an example, a “module” or “part” may include components such as software components, object-oriented software components, class components, and task components, and at least one of processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. Components and “modules” or “parts” may be combined into fewer components and “modules” or “parts” based on the functionality they provide internally, or they may be further separated into additional components and “modules” or “parts”.

[0023] According to one embodiment of the present disclosure, a “module” or “part” can be realized by a processor and memory. “Processor” should be broadly interpreted to include general-purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, and the like. Depending on the context, “processor” may also refer to ordered semiconductor devices (ASICs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), and the like. “Processor” may also refer to a combination of processing devices, such as a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors coupled with a DSP core, or any other combination of such configurations. “Memory” should also be broadly interpreted to include any electronic component capable of storing electronic information. The term "memory" can also refer to various types of processor-readable media, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage devices, and registers. Memory is said to be in electronic communication with the processor if the processor can read information from and / or write information to it. Memory integrated into a processor is in electronic communication with the processor.

[0024] In this disclosure, “System” may include, but is not limited to, at least one of a server device and a cloud device. For example, a system may consist of one or more server devices. Another example is that a system may consist of one or more cloud devices. Yet another example is that a system may consist of and operate with both server devices and cloud devices.

[0025] In this disclosure, “Display” means any display device associated with a computing device, for example, any display device that is controlled by or provided by a computing device and capable of displaying any information / data.

[0026] In this disclosure, “each of the A” or “each of the A” may refer to each of all components included in the A, or to each of some of the components included in the A.

[0027] In this disclosure, “machine learning model” can include any model used to infer an answer to a given input. According to one embodiment, a machine learning model can include an artificial neural network model that includes an input layer, a plurality of hidden layers, and an output layer, where each layer can include a plurality of nodes. In this disclosure, multiple machine learning models are described as separate machine learning models, but are not limited to this, and some or all of multiple machine learning models can be implemented as a single machine learning model. Also, a single machine learning model can include multiple machine learning models. In this disclosure, the terms “machine learning model” and “artificial neural network model” can be used interchangeably to refer to the same or similar models. Also, in this disclosure, “language model” may refer to a machine learning model or artificial neural network model configured to calculate probabilities for at least one or more sequences of words or sentences, or to generate sequences of words or sentences.

[0028] Figure 1 shows an example of a method for extracting review keywords 140 for a product 110 provided by one embodiment of the present disclosure. In one embodiment, based on information for identifying the product 110, review data 120 related to the product 110 can be collected from data sources (e.g., blogs, online shopping malls, internet cafes, company homepages related to the product, etc.). Here, the information for identifying the product may be a predetermined product name, product number, or catalog ID related to the product. The review data 120 may also be review data generated within a predetermined period (e.g., the last year).

[0029] In one embodiment, a language model 130 can be used to generate at least one response data from at least a portion of the review data 120 based on a predetermined set of questions. Here, the predetermined set of questions may be related to the intended use of the product, the purchase intent of the product, the advantages of the product, the disadvantages of the product, purchase history / plans, related products / brands mentioned with the product, where the product is purchased, the path to awareness of the product, the ingredients / applied technology of the product, the appearance of the product, the nickname of the product, how to use the product, product collaborations / projects, etc. For example, by inputting question data related to the product awareness path, such as "How did you learn about this product?", and review data 120 into the language model 130, a response data corresponding to that question (for example, "I heard it was spicy and delicious") can be generated.

[0030] In one embodiment, the language model 130 may be a model that generates response data corresponding to question data by inputting document data (e.g., product review data) and question data. Alternatively, the language model 130 may be a sequence-to-sequence model. Furthermore, the language model 130 may be a machine learning model trained using a predetermined training dataset including document data or question data. The process of training the language model 130 will be described in detail below with reference to Figure 8.

[0031] In one embodiment, response data to a predetermined set of questions can be converted into embedding vectors. Furthermore, at least one group can be generated by clustering the embedding vectors based on the distance between them. In this case, a representative keyword can be extracted from each of these at least one group. Here, the representative keyword can be determined based on the frequency of keywords included in at least one group.

[0032] In one embodiment, review keywords 140 related to product 110 can be extracted based on at least one response data. For example, review keywords 140 may include keywords such as "My dad said he wanted to eat it" related to the target use of the product, "I'm going camping" related to how the product is used, "I heard it's spicy and delicious" related to the way the product was discovered, and "The taste isn't too strong" related to the product's advantages.

[0033] This configuration allows users to easily see review keywords for a product simply by entering information to identify that product. This enables users to easily understand and analyze information or characteristics of related brands and products based on product reviews from other users who have purchased or used the product. Furthermore, providing review keywords can help users decide whether or not to purchase the product.

[0034] Figure 2 is a schematic diagram showing a configuration in which an information processing system 230 is connected to a plurality of user terminals 210_1, 210_2, and 210_3 so as to be able to communicate with them in order to extract product review keywords according to one embodiment of the present disclosure. As shown in the figure, the plurality of user terminals 210_1, 210_2, and 210_3 can be connected to an information processing system 230 that can provide a product review keyword extraction service via a network 220. Here, the plurality of user terminals 210_1, 210_2, and 210_3 may include terminals of users to whom the product review keyword extraction service is provided.

[0035] In one embodiment, the information processing system 230 may include one or more server devices and / or databases capable of storing, providing, and executing computer-executable programs (e.g., downloadable applications) and data related to the provision of a product review keyword extraction service, or one or more distributed computing devices and / or distributed databases based on a cloud computing service.

[0036] The product review keyword extraction service provided by the information processing system 230 can be provided to users through a product review keyword extraction service application web browser or web browser extension program installed on each of the user terminals 210_1, 210_2, and 210_3. For example, the information processing system 230 can provide information corresponding to product review keyword extraction requests received from user terminals 210_1, 210_2, and 210_3, or execute corresponding processing, through the product review keyword extraction service application or the like.

[0037] Multiple user terminals 210_1, 210_2, and 210_3 can communicate with the information processing system 230 via the network 220. The network 220 can be configured to enable communication between the multiple user terminals 210_1, 210_2, and 210_3 and the information processing system 230. Depending on the installation environment, the network 220 can consist of wired networks such as Ethernet, Power Line Communication, telephone line communication equipment, and RS-serial communication, as well as wireless networks such as mobile communication networks, WLAN (Wireless LAN), Wi-Fi, Bluetooth, and ZigBee, or a combination thereof. The communication method is not limited and can include not only communication methods that utilize the communication network that the network 220 may include (for example, mobile communication networks, wired internet, wireless internet, broadcasting networks, satellite networks, etc.), but also short-range wireless communication between user terminals 210_1, 210_2, and 210_3.

[0038] Figure 2 shows a mobile phone terminal 210_1, a tablet terminal 210_2, and a PC terminal 210_3 as examples of user terminals, but is not limited to these. User terminals 210_1, 210_2, and 210_3 may be any computing device capable of wired and / or wireless communication, and capable of installing and running a product review keyword extraction service application or a web browser, etc. For example, user terminals may include AI speakers, smartphones, mobile phones, navigation systems, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, game consoles, wearable devices, IoT (Internet of Things) devices, VR (virtual reality) devices, AR (augmented reality) devices, set-top boxes, etc. Furthermore, while Figure 2 shows that three user terminals 210_1, 210_2, and 210_3 communicate with the information processing system 230 via the network 220, the system is not limited to this configuration, and it is also possible to configure it so that a different number of user terminals communicate with the information processing system 230 via the network 220.

[0039] Figure 3 is a block diagram showing the internal configuration of a user terminal 210 and an information processing system 230 according to one embodiment of the present disclosure. The user terminal 210 may refer to any computing device capable of running applications, web browsers, etc., and capable of wired / wireless communication, and may include, for example, the mobile phone terminal 210_1, tablet terminal 210_2, PC terminal 210_3 in Figure 2. As shown, the user terminal 210 may include a memory 312, a processor 314, a communication module 316, and an input / output interface 318. Similarly, the information processing system 230 may include a memory 332, a processor 334, a communication module 336, and an input / output interface 338. As shown in Figure 3, the user terminal 210 and the information processing system 230 may be configured to communicate information and / or data via a network 220 using their respective communication modules 316, 336. Furthermore, the input / output device 320 may be configured to input information and / or data to the user terminal 210 via the input / output interface 318, or to output information and / or data generated from the user terminal 210.

[0040] The memories 312 and 332 may include any non-temporary computer-readable recording medium. According to one embodiment, the memories 312 and 332 may include permanent mass storage devices such as ROM (read-only memory), disk drives, SSDs (solid-state drives), and flash memory. As another example, permanent mass storage devices such as ROM, SSDs, flash memory, and disk drives are separate permanent storage devices distinct from the memories and may be included in the user terminal 210 or the information processing system 230. The memories 312 and 332 may also store an operating system and at least one program code.

[0041] Such software components may be loaded from a computer-readable recording medium separate from the memories 312, 332. Such a computer-readable recording medium may include a recording medium that can be directly connected to such a user terminal 210 and information processing system 230, and may include computer-readable recording media such as floppy drives, disks, tapes, DVD / CD-ROM drives, and memory cards. As another example, the software components may be loaded into the memories 312, 332 via the communication modules 316, 336 instead of from a computer-readable recording medium. For example, at least one program may be loaded into the memories 312, 332 based on a computer program installed by a file provided via the network 220 by developers or a file distribution system that distributes application installation files.

[0042] Processors 314, 334 may be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to processors 314, 334 by memory 312, 332 or by communication modules 316, 336. For example, processors 314, 334 may be configured to execute instructions received according to program code stored in a recording device such as memory 312, 332.

[0043] Communication modules 316 and 336 can provide configurations or functions for the user terminal 210 and the information processing system 230 to communicate with each other via the network 220, and can provide configurations or functions for the user terminal 210 and / or the information processing system 230 to communicate with other user terminals or other systems (for example, another cloud system). For example, a request or data (e.g., a product review keyword extraction request) generated by the processor 314 of the user terminal 210 according to program code stored in a recording device such as memory 312 may be transmitted to the information processing system 230 via the network 220 under the control of the communication module 316. Conversely, control signals or commands provided under the control of the processor 334 of the information processing system 230 may be received by the user terminal 210 via the communication module 336 and the network 220 through the communication module 316 of the user terminal 210.

[0044] The input / output interface 318 may also be a means for interface with the input / output device 320. For example, the input device may include a camera including an audio sensor and / or image sensor, a keyboard, a microphone, a mouse, etc., and the output device may include a display, a speaker, a haptic feedback device, etc. In another example, the input / output interface 318 may be a means for interface with a device that integrates a configuration or function for input and output into one, such as a touchscreen. For example, when the processor 314 of the user terminal 210 processes instructions of a computer program loaded into memory 312, a service screen, etc., which is configured using information and / or data provided by the information processing system 230 or other user terminals, may be displayed on the display via the input / output interface 318. In Figure 3, the input / output device 320 is shown not to be included in the user terminal 210, but is not limited to this, and may be configured as a device integrated with the user terminal 210. Furthermore, the input / output interface 338 of the information processing system 230 may be connected to the information processing system 230 or may be a means for interface with an input or output device (not shown) that the information processing system 230 may include. In Figure 3, the input / output interfaces 318 and 338 are shown as separate components from the processors 314 and 334, but are not limited to this, and the input / output interfaces 318 and 338 may be configured to be included in the processors 314 and 334.

[0045] The user terminal 210 and the information processing system 230 may include more components than those shown in Figure 3. However, it is not necessary to clearly illustrate most of the conventional technical components. In one embodiment, the user terminal 210 may be implemented to include at least a portion of the input / output device 320 described above. The user terminal 210 may also further include other components such as a transceiver, a GPS (Global Positioning system) module, a camera, various sensors, and a database. For example, if the user terminal 210 is a smartphone, it may include components that are generally included in a smartphone, such as an accelerometer, gyroscope, microphone module, camera module, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration.

[0046] While a program for a product review keyword extraction service application is running, the processor 314 can receive input or selected text, images, videos, audio and / or actions through input devices such as a camera, microphone, and an audio sensor connected to the input / output interface 318. The received text, images, videos, audio and / or actions can be stored in the memory 312 or provided to the information processing system 230 via the communication module 316 and the network 220.

[0047] The processor 314 of the user terminal 210 may be configured to manage, process, and / or store information and / or data received from the input / output device 320, other user terminals, the information processing system 230, and / or multiple external systems. The information and / or data processed by the processor 314 can be provided to the information processing system 230 via the communication module 316 and the network 220. The processor 314 of the user terminal 210 can transmit and output information and / or data to the input / output device 320 via the input / output interface 318. For example, the processor 314 can display the received information and / or data on the screen of the user terminal 210.

[0048] The processor 334 of the information processing system 230 may be configured to manage, process, and / or store information and / or data received from multiple user terminals 210 and / or multiple external systems. The information and / or data processed by the processor 334 can be provided to the user terminals 210 via the communication module 336 and the network 220.

[0049] Figure 4 shows an example of a procedure for collecting review data according to one embodiment of the present disclosure. As shown in the figure, review data can be collected from one or more data sources or databases using a database search command (e.g., an SQL query) 420 that includes information for identifying a product (e.g., product name, product number, catalog ID, etc.) 410. Figure 4 shows, but is not limited to, collecting review data from blogs and online shopping malls based on product name (or product number). Review data can also be collected from internet cafes, internet news, company homepages related to the product, and text converted from audio of review videos.

[0050] In one embodiment, the online shopping mall review data 430 collected from the online shopping mall may be preprocessed (432). For example, the online shopping mall review data 430 may be filtered to extract only the review data generated within a predetermined period (e.g., one year). Alternatively, predetermined prohibited words or special characters included in the online shopping mall review data 430 may be removed.

[0051] In one embodiment, the blog review data 440 collected from the blog may be preprocessed (442). For example, the blog review data 440 may be filtered to extract only the review data generated within a predetermined period (e.g., the last year). Also, because the blog review data 440 is very large, it may be divided into any number of chunks. Furthermore, predetermined prohibited words or special characters contained in the blog review data 440 may be removed.

[0052] In one embodiment, pre-processed online shopping mall review data and pre-processed blog review data may be stored in a review database 450. In this case, the review data may be stored in correspondence to at least one of a predetermined set of questions. For example, the review data may be classified into review data related to the purchase intent of the product, review data related to the advantages of the product, review data related to the awareness path of the product, etc., and stored in the review database 450.

[0053] Figure 5 shows an example of filtering review data 510 according to its nature according to one embodiment of the present disclosure. In one embodiment, review data 510 collected from a blog or online shopping mall can be classified into informative review data 530 and spam / promotional review data 540 by a passage-type classifier 520 (or Passage Type Classifier; PTC). Here, informative review data 530 is determined to contain actual user review information on various characteristics of a product and can be stored in a review database 550. Conversely, if the collected review data is spam / promotional, it can be determined to contain promotional phrases (for example, "We may receive a certain commission for this review"). Such spam / promotional review data can be used in a subsequent review keyword extraction step to extract inappropriate review keywords that do not correspond to predetermined questions.

[0054] In one embodiment, the passage-type classifier 520 may be a language model or machine learning model that distinguishes informational review data 530 from review data 510. For example, by training it with a set of training data and ground truth data for filtering abusive documents, the passage-type classifier 520 can determine whether particular review data is informational review data or review data with other properties. Since the passage-type classifier 520 must perform inference on a large amount of review data, it may be, for example, a relatively lightweight form of BERT model, but is not limited thereto.

[0055] This configuration allows for the simplification of the nature of the review data during keyword extraction by using the passage-type classifier 520 to remove spam / promotional review data. In other words, only informational review data can be extracted from the review data and used in the subsequent keyword extraction step. This reduces the uncertainty of the keyword extracted in the keyword extraction step and improves the reliability of the keyword.

[0056] Figure 6 shows an example of a procedure for extracting review keywords from review data according to one embodiment of the present disclosure. In one embodiment, review data can be preprocessed by using a language model to check whether the review data stored in the review database 610 contains answers to at least some of a predetermined set of questions (620). An example of how review data is preprocessed will be described in detail below with reference to Figure 7.

[0057] In one embodiment, if the review data includes answers to at least some of a predetermined set of questions, at least some of the review data can be extracted as answer data corresponding to at least some of the predetermined set of questions (630). Post-processing steps can be performed on the extracted answer data, such as removing duplicate sentences or sentences that are different from the original review data (640). Furthermore, by clustering similar answer data into at least one group, representative keywords from each of the at least one group can be extracted and stored in the keyword database 650 as review keywords.

[0058] Figure 7 shows an example of preprocessing review data according to one embodiment of the present disclosure. In one embodiment, by preprocessing the review data stored in the review database 710 through a Question Semantic Matcher (QSM) 720, it is possible to determine whether or not answer data corresponding to at least some of a predetermined set of questions 730 can be extracted from the review data. If an answer to a question is extracted even when there is no answer data corresponding to a particular question among the predetermined set of questions 730 in the review data, it may generate inappropriate answer data. Therefore, it is possible to prevent the generation of inappropriate answer data through an appropriate preprocessing process.

[0059] In one embodiment, the question semantic matching model 720 may be a machine learning model that determines whether it is possible to extract answer data corresponding to at least some of a predetermined set of questions 730 from review data. The question semantic matching model 720 is trained on a set of document data, question data, and ground truth data, and can determine whether the review data contains answers corresponding to each of the predetermined set of questions 730. For example, through the question semantic matching model 720, it can be determined that the review data contains answers corresponding to questions related to the intended use of the product, the purchase intent of the product, the appearance of the product, and the nickname of the product. In this case, only the answer data corresponding to those questions can be ultimately generated, and the generation of inappropriate answer data corresponding to the remaining questions can be prevented.

[0060] Through this configuration, it is possible to determine whether the review data contains answers corresponding to at least some of a predetermined set of question data. In other words, the characteristics of the information contained in the review data can be understood in advance. This reduces uncertainty in the review keyword extraction step and improves the reliability of the review keywords.

[0061] Figure 8 shows an example of training a language model for generating response data according to one embodiment of the present disclosure. In one embodiment, the language model for generating response data can be trained using a predetermined training dataset. Specifically, a set of question data 810 and document data 820 can be input to a first generative model 830. In this case, the first generative model 830 can pseudo-label at least a portion of the document data 820 as first response data 840 for a particular question among the question data 810. Here, the first response data 840 may contain noise responses that are inappropriate for the particular question. For example, if the particular question is related to the cognitive pathway of a product, the first generative model 830 may generate noise responses related to product advantages that are far removed from the particular question.

[0062] In one embodiment, a second generative model 860 can be further used to reduce such noisy responses. Specifically, a portion of the first response data 840 can be accepted, thereby removing noise. For example, the acceptance of the response data may be performed manually by a system operator. This allows the accepted response data 850 to be considered as ground truth data. Here, for the efficiency of acceptance, it is not necessary to accept all of the first response data 840. Alternatively, the second generative model 860 can be trained using specific questions corresponding to the first response data 840 and the accepted response data 850. In this case, by inputting the remaining unaccepted portion of the first response data 840 into the second generative model 860, the second generative model 860 can label the corresponding portion of the first response data 840 as second response data 870. Using the second generative model 860 thus trained, response data can be generated from the review data.

[0063] This configuration allows for improved quality of output response data by using an additional generative model in addition to the generative model that extracts response data corresponding to question data from review data (or document data). Furthermore, the efficiency of response data acceptance can be improved by accepting only a portion of the response data extracted by the generative model and using that portion to train the additional generative model, rather than accepting all of it.

[0064] Figure 9 shows an example of post-processing of response data generated by one embodiment of the present disclosure. In one embodiment, a language model can be used to generate first response data 920 from input data 910. Here, the input data 910 input to the language model may include information for identifying a product (e.g., product name), review data, question data, etc. For example, the language model can generate first response data 920 corresponding to a question related to the shortcomings of a product from review data related to an identified product (e.g., a collapsible long desk), such as "It makes a sound like it's going to break every time you turn a screw into the wooden part; is the tabletop not very durable; is it not very durable; is it not sturdy; is it not sturdy;".

[0065] In one embodiment, second response data 930 can be generated by post-processing the first response data 920 generated by the language model. Specifically, duplicate sentences can be removed from the first response data 920. For example, since the phrase "not solid" is duplicated in the first response data 920, the second response data 930 can be generated by removing this sentence.

[0066] In one embodiment, third response data 940 can be generated by post-processing the second response data 930. Specifically, hallucination sentences that do not exist in the original review data can be removed from the second response data 930. For example, in the second response data 930, "because it is not solid" does not exist in the original review data, so the third response data 940 can be generated by removing this sentence. In addition, in the second response data 930, sentences that do not exist in the original review data but have undergone slight changes such as word segmentation, sentence punctuation, or grammatical word substitution can be replaced with the corresponding sentences included in the original review data. For example, in the second response data 930, "because it makes a sound like it is about to break every time you turn a screw into the wooden part" is similar to "because it makes a sound like it is about to break every time you turn a screw into the wooden part" in the original review data, so it can be replaced with the corresponding sentence included in the original review data. Here, if the match score between the response data and a portion of the review data is above a predetermined threshold, it can be determined that the response data and the portion of the review data are similar to each other, and that portion of the review data can replace the response data.

[0067] In one embodiment, the fourth response data 950 can be generated by post-processing the third response data 940. Specifically, sentences in a subordination relationship can be removed from the third response data 940. For example, in the third response data 940, "weak durability" is included in "Is the tabletop's durability weak?", so the fourth response data 950 can be generated by removing the sentences in a subordination relationship. Here, the remaining sentences, excluding the longest sentence among the sentences in a subordination relationship, can be removed.

[0068] In one embodiment, review keywords can be extracted based on post-processed response data. Specifically, each of the post-processed response data can be converted into an embedding vector using a model based on an artificial neural network. Furthermore, the embedding vectors can be clustered into at least one group based on the distance between them. For example, multiple embedding vectors may be clustered using the K-means algorithm, but this is not limited to that. In this case, representative keywords can be extracted as review keywords from each of the at least one group. Here, the representative keywords may be determined based on the frequency of keywords included in at least one group.

[0069] This configuration allows for post-processing of response data extracted from review data in correspondence with question data, thereby improving the accuracy or reliability of the extracted review keywords. Furthermore, by reflecting at least a portion of the original review data's sentences in the review keywords, the reliability of the final extracted review keywords can be enhanced.

[0070] Figure 10 shows an example of review keywords 1020-1040 according to one embodiment of the present disclosure. As shown in the figure, review keywords 1020-1040 related to product 1010 can be extracted using a language model. Here, the review keywords 1020-1040 may be output as labels related to a predetermined set of questions.

[0071] For example, if the product is Makguksu, review keywords 1020 related to the target use of the product may include "My husband," "With my younger brother," and "With my 4-year-old baby." Additionally, review keywords 1030 related to how the product was discovered may include "Gogi-ri Makguksu was the first to introduce it to me," "I heard a lot of rumors that it was delicious," and "I've seen it in the media." Furthermore, review keywords 1040 related to how the product is used may include "On top of mixed noodles," "With soba noodles mixed with perilla oil and brewed soy sauce," and "Turning noodle dishes into one-bowl meals."

[0072] Figure 10 shows review keywords according to three labels, but is not limited to these; review keywords according to a wider variety of labels may be output. Furthermore, while it is shown that up to 10 review keywords can be arbitrarily selected and output for each label, this is not limited to this.

[0073] This configuration allows users to easily view review keywords related to a product simply by entering product information. This enables potential buyers to use the generated review keywords as a reference when deciding whether to purchase the desired product. Furthermore, the generated review keywords can also be used as a tool for analyzing brand / product reputation.

[0074] Figure 11 is a flowchart illustrating an example of method 1100 according to one embodiment of the present disclosure. In one embodiment, method 1100 can be performed by at least one processor. Method 1100 can be initiated by the processor collecting product-related review data based on information for identifying the product (S1110). Here, the information for identifying the product may be a predetermined product name, product number, or catalog ID related to the product. The product-related review data may be review data generated within a predetermined period.

[0075] Subsequently, the processor can use a language model to generate at least one response data from at least a portion of the review data based on a predetermined set of questions (S1120). In this case, the processor can use the language model to determine whether the review data contains answers to at least a portion of the predetermined set of questions. Furthermore, if the processor determines that the review data contains answers to at least a portion of the predetermined set of questions, it can determine at least a portion of the review data as the response data corresponding to at least a portion of the predetermined set of questions.

[0076] Subsequently, the processor can extract product-related review keywords based on at least one response data (S1130). Specifically, the processor can post-process at least one response data. Furthermore, the processor can extract at least one product-related review keywords from the post-processed response data.

[0077] In one embodiment, the processor can use a machine learning model to remove spam review data and promotional review data from product-related review data. The processor can also remove predetermined prohibited words or special characters from product-related review data.

[0078] In one embodiment, the processor can train a language model using a predetermined training dataset. Here, the predetermined training dataset may include at least one of document data and question data. Specifically, the processor can use a first generative model to pseudo-label at least a portion of the document data as first answer data for a specific question from the question data, and then train a second generative model using the specific question and a portion of the first answer data. Furthermore, the processor can train the second generative model using the specific question and the remainder of the first answer data. In this case, the remainder of the first answer data may be accepted answer data. The processor can also label a portion of the first answer data as second answer data for a specific question through the second generative model.

[0079] In one embodiment, the processor can remove response data containing duplicate sentences from at least one response data. If there are multiple response data that are inclusion relationships with at least one response data, the processor can remove the remaining response data, excluding one of the multiple response data that are inclusion relationships. Furthermore, the processor can remove the remaining response data, excluding the longest response data among the multiple response data.

[0080] In one embodiment, the processor can determine sentences in the review data that correspond to at least a portion of the response data, based on the match score between at least a portion of the response data and the review data. Subsequently, if the match score is greater than or equal to a predetermined threshold, the processor can replace at least a portion of the response data with sentences from the review data. In this case, if the match score is less than the predetermined threshold, the processor can remove at least a portion of the response data.

[0081] In one embodiment, the processor can convert answer data for a predetermined set of questions into embedding vectors. Furthermore, based on the distance between the embedding vectors, the processor can generate at least one group. In addition, the processor can extract representative keywords from each of these at least one group.

[0082] In one embodiment, review data can be collected from blogs and online shopping malls. In this case, review data related to some of a predetermined set of questions may be collected from blogs, and review data related to the remaining questions may be collected from online shopping malls.

[0083] The above method can be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may continuously store a computer-executable program or temporarily store it for execution or download. The medium may also be a variety of recording or storage means in the form of a combination of one or more hardware components, and is not limited to a medium directly connected to a computer system, but may be distributed on a network. Examples of media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and media such as ROM, RAM, and flash memory configured to store program instructions. Other examples of media include recording or storage media managed by app stores that distribute applications, and sites and servers that supply or distribute various other software.

[0084] The methods, operations, or techniques described herein can be implemented by various means. For example, such techniques can be implemented in hardware, firmware, software, or a combination thereof. A person of ordinary skill would understand that the various exemplary logic blocks, modules, circuits, and algorithmic steps described in conjunction with the disclosures herein can also be implemented in electronic hardware, computer software, or a combination thereof. To illustrate this hardware and software compatibility clearly, various exemplary components, blocks, modules, circuits, and steps have been described in general terms of their functionality. Whether such functionality is implemented in hardware or software depends on the design requirements imposed on the particular application and the overall system. A person of ordinary skill could implement the functionality described in various ways for each specific application, but such implementations should not be construed as exceeding the scope of the disclosures.

[0085] In hardware implementation, the processing unit used to perform the technology may be implemented in one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, computers, or combinations thereof.

[0086] Accordingly, the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure can also be implemented or run on any combination of general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gates and transistor logic, discrete hardware components, or any other devices designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other combination of configurations.

[0087] In implementing firmware and / or software, the technology can also be implemented as instructions stored in computer-readable media such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), or magnetic or optical data storage devices. The instructions may be executable by one or more processors, which may cause the processors(s) to perform specific modes of the functions described herein.

[0088] When implemented in software, the above techniques may be stored as one or more instructions or codes on a computer-readable medium or transmitted over a computer-readable medium. A computer-readable medium includes any medium that facilitates the transmission of computer programs from one location to another, and includes both computer storage media and communication media. The storage medium may be any available medium accessible by a computer. Such computer-readable media may include, but are not limited to, RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, or other magnetic storage devices, or any other medium accessible by a computer that can be used to transmit or store desired program code in the form of instructions or data structures. Furthermore, any connection is appropriately referred to as a computer-readable medium.

[0089] For example, when software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, these are included within the definition of a medium. The terms "disk" and "disc" as used in this application include CDs, laserdiscs, optical discs, DVDs (digital versatile discs), floppy disks, and Blu-ray discs, where "disks" typically reproduce data magnetically, while "discs" reproduce data optically using a laser. Any combination of the above should also be included within the scope of a computer-readable medium.

[0090] Software modules may also be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other known form of storage medium. Examples of storage mediums include those connected to the processor so that the processor can read information from or write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and storage medium may reside within an ASIC. The ASIC may reside within a user terminal. Alternatively, the processor and storage medium may exist as separate components in the user terminal.

[0091] While the embodiments described above have been explained as utilizing the aspects of the subject matter currently disclosed in one or more independent computer systems, the disclosure is not limited thereto and can also be implemented in conjunction with any computing environment, such as a network or a distributed computing environment. Furthermore, aspects of the subject matter in the disclosure can be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include PCs, network servers, and portable devices.

[0092] In this specification, although the disclosure has been described in relation to some embodiments, various modifications and alterations may be made without departing from the scope of the disclosure as understandable to a person of the ordinary skill in the art to which the invention of this disclosure pertains. Such modifications and alterations should be deemed to fall within the scope of the claims appended to this specification.

Claims

1. A method for extracting product review keywords based on a language model, which is executed by at least one processor, The steps include: collecting review data related to the product based on information for identifying the product; A step of using a language model to generate at least one response data from at least a portion of the review data based on a predetermined set of questions, A step of extracting review keywords related to the product based on at least one of the aforementioned response data, A method for extracting product review keywords, including those mentioned above.

2. The product review keyword extraction method according to claim 1, further comprising the step of removing spam review data and promotional review data from review data related to the product using a machine learning model.

3. The step of generating at least one response data from at least a portion of the review data based on a predetermined set of questions using a language model is: The steps include using the language model to determine whether the review data contains answers to at least some of the predetermined set of questions, If it is determined that the review data contains answers to at least some of the predetermined multiple questions, the step of determining at least some of the review data as answer data corresponding to at least some of the predetermined multiple questions, A method for extracting product review keywords according to claim 1, including the method described in claim 1.

4. The process further includes the step of training the language model using a predetermined training dataset, The product review keyword extraction method according to claim 1, wherein the predetermined learning dataset includes at least one of document data and question data.

5. The step of training the language model using a predetermined training dataset is: The steps include: using a first generative model to pseudo-label at least a portion of the document data as first answer data for a specific question among the question data; A step of training a second generative model using the aforementioned specific questions and a portion of the first answer data, A method for extracting product review keywords according to claim 4, including the method described in claim 4.

6. The remaining portion of the aforementioned first response data consists of the accepted response data. The step of training the language model using a predetermined dataset is: A step of training the second generative model using the aforementioned specific questions and the remaining first answer data, The steps include labeling a portion of the first response data as second response data for a specific question through the second generation model, The method for extracting product review keywords according to claim 5, further comprising:

7. The step of extracting review keywords related to the product based on at least one of the aforementioned response data is: The steps include: post-processing at least one of the aforementioned response data, The steps include extracting at least one review keyword related to the product from the post-processed response data, A method for extracting product review keywords according to claim 1, including the method described in claim 1.

8. The step of post-processing at least one of the aforementioned response data is: The product review keyword extraction method according to claim 7, further comprising the step of removing response data containing duplicate sentences from at least one of the response data.

9. The step of post-processing at least one of the aforementioned response data is: The steps include determining a sentence in the review data that corresponds to at least a portion of the response data, based on a match score between at least a portion of the response data and the review data, If the match score is equal to or greater than a predetermined threshold, the step of replacing at least a portion of the response data with sentences from the review data, A method for extracting product review keywords according to claim 7, including the method described in claim 7.

10. The step of post-processing at least one of the response data is: The product review keyword extraction method according to claim 9, further comprising the step of removing at least a portion of the response data if the match score is less than the predetermined threshold.

11. The step of post-processing at least one of the response data is: The product review keyword extraction method according to claim 7, further comprising the step of removing the remaining response data, excluding one of the multiple response data that are inclusion relationships with at least one of the response data.

12. The step of post-processing at least one of the aforementioned response data is: The product review keyword extraction method according to claim 11, further comprising the step of removing the remaining response data from the plurality of response data except for the longest response data.

13. The step of extracting review keywords related to the product based on at least one of the aforementioned response data is: The steps include converting the answer data to the predetermined set of questions into an embedding vector, A step of generating at least one group based on the distance between the aforementioned embedding vectors, A method for extracting product review keywords according to claim 1, including the method described in claim 1.

14. The step of extracting review keywords related to the product based on at least one of the aforementioned response data is: The product review keyword extraction method according to claim 13, further comprising the step of extracting representative keywords from each of the at least one group.

15. The product review keyword extraction method according to claim 1, wherein the information for identifying the aforementioned product is at least one of a predetermined product name, product number, or catalog ID related to the product.

16. The aforementioned review data was collected from blogs and online shopping malls. Review data related to some of the predetermined questions mentioned above was collected from the blog mentioned above. The product review keyword extraction method according to claim 1, wherein the review data related to the remaining of the predetermined set of questions is collected from the online shopping mall.

17. The product review keyword extraction method according to claim 1, wherein the review data related to the aforementioned product is review data generated within a predetermined period.

18. A method for extracting product review keywords according to claim 1, further comprising the step of collecting review data related to the product based on information for identifying the product, and then removing predetermined prohibited words or special characters from the review data related to the product.

19. A program for executing the method described in claim 1 on a computer.

20. Communication module and Memory and The system includes at least one processor connected to the memory and configured to execute at least one computer-readable program contained in the memory, The at least one program is Based on information to identify the product, we collect review data related to the said product. Using a language model, at least one response data is generated from at least a portion of the review data based on a predetermined set of questions. A system including a command for extracting review keywords related to the product based on at least one of the aforementioned response data.

Images