The Method And System For Regulatory Search And Violation Analysis For Product Using Large Language Model

Abstract

The present invention relates to a method and system for detecting and analyzing violations of regulations on a product using a big language model, wherein the method and system receive a product technical document from a user terminal to extract one or more regulatory keywords, search for text data matching the regulatory keywords from a database to derive a first relevant regulatory data, derive a second relevant regulatory data based on the embedding similarity between a query embedding for a chunk of the product technical document and embedding data stored in the database, and input a prompt including the product technical document, the first relevant regulatory data, the second relevant regulatory data, and user request information into the big language model to derive violations and legal grounds regarding the regulations.

Description

Technology Field

[0001] The present invention relates to a method and system for detecting and analyzing violations of regulations on a product using a big language model, wherein the method and system receive a product technical document from a user terminal to extract one or more regulatory keywords, search for text data matching the regulatory keywords from a database to derive a first relevant regulatory data, derive a second relevant regulatory data based on the embedding similarity between a query embedding for a chunk of the product technical document and embedding data stored in the database, and input a prompt including the product technical document, the first relevant regulatory data, the second relevant regulatory data, and user request information into the big language model to derive violations and legal grounds regarding the regulations. Background Technology

[0003] In modern society, as the development cycle of products incorporating new technologies accelerates, the legal regulations and standards that products must comply with are becoming more complex and subject to frequent changes. Accordingly, it is essential to review whether relevant technical documentation complies with extensive regulatory requirements during the product development phase.

[0004] Conventional document analysis and review technologies include a method and program for searching for citation phrases within a document for inspection, as disclosed in Korean Registered Patent No. 10-1634681. This method and program disclose a technology that searches for identifiable targets, such as footnotes or formatting, included within the document for inspection, determines the cited document based on this, and identifies exclusion areas during plagiarism checks. While such conventional technology is effective for mechanically matching and analyzing specific formats or phrases within a document, it has limitations in understanding the context of the document or identifying the same meaning expressed in different terms.

[0005] As a result, determining whether specific product specifications violate particular regulatory provisions or obtaining specific guidance to resolve them still requires reliance on manual reviews by highly specialized personnel, which causes significant time and cost.

[0006] Therefore, there is a need to develop a system capable of providing highly reliable review results by going beyond simple keyword matching to grasp the contextual meaning between documents and precisely analyzing the correlations between vast regulatory data and product information. Prior art literature

[0008] Republic of Korea Registered Patent No. 10-1634681 (June 23, 2016) The problem to be solved

[0009] The present invention aims to provide a method and system for detecting and analyzing regulations and violations of a product using a big language model, wherein the method and system receive a product technical document from a user terminal to extract one or more regulatory keywords, search for text data matching the regulatory keywords from a database to derive a first relevant regulatory data, derive a second relevant regulatory data based on the embedding similarity between a query embedding for a chunk of the product technical document and embedding data stored in the database, and input a prompt including the product technical document, the first relevant regulatory data, the second relevant regulatory data, and user request information into the big language model to derive violations and legal grounds regarding the regulations. means of solving the problem

[0011] In order to solve the above-mentioned problem, one embodiment of the present invention provides a method for regulatory search and violation analysis of a product using a large language model, which is performed by a server system comprising one or more processors and one or more memories, comprising: a document reception step of receiving a product technical document regarding a product from a user terminal; a keyword extraction step of inputting a first prompt, which includes the received product technical document and a intent to extract keywords related to regulation from the product technical document, into an internal or external large language model to extract one or more regulatory keywords; a keyword search step of deriving first relevant regulatory data by searching for text data matching the regulatory keywords from a database in which current regulatory data and regulatory data scheduled for future implementation are respectively stored as text data and embedding data; and an embedding generation step of dividing the product technical document into text and image chunks, and inputting each divided chunk into a trained artificial neural network-based embedding model to generate a plurality of query embeddings. The present invention provides a method for regulatory search and violation analysis, comprising: an embedding search step of calculating embedding similarity between embedding data stored in the database and the query embedding, and deriving second related regulatory data associated with the product technical document based on the calculated embedding similarity; and a violation analysis step of inputting a second prompt into the big language model, which includes the product technical document; the first related regulatory data; the second related regulatory data; and the intent to determine whether the contents described in the product technical document violate the judgment criteria based on the judgment criteria, thereby deriving violations of regulations and legal grounds corresponding to the violations.

[0012] In one embodiment of the present invention, the regulatory search and violation analysis method further includes a request receiving step of receiving user request information including query text entered by a user regarding the review direction or concerns of the product technical document from the user terminal; and the violation analysis step may include a request reflection step of deriving violations and legal grounds reflecting the user request information by inputting the user request information and a intent to perform analysis by considering the user's intent included in the user request information into the big language model, thereby further including the user request information and a intent to perform analysis by considering the user's intent included in the user request information when determining whether the judgment criteria are violated.

[0013] In one embodiment of the present invention, the regulatory search and violation analysis method further includes a visualization step after the violation analysis step, in which a marking is displayed on an area identified as a violation on the original text of the product technical document to visualize it; and the visualization step may include a violation layer popup step, in which a violation layer displaying the violation and legal grounds is popped up on the original text when an input pointer is detected to be located on a specific marking area.

[0014] In one embodiment of the present invention, the visualization step can visually distinguish and provide the type of violation according to the timing of regulatory application in the original text by displaying a first marking element in the area identified as the violation when the violation derived in the violation analysis step is an item based on current regulatory data, and displaying a second marking element in the area identified as the violation when the violation is an item based on regulatory data scheduled to be implemented in the future.

[0015] In one embodiment of the present invention, the violation analysis step further includes a recommendation derivation step for deriving a recommendation corresponding to the violation by inputting into the big language model a second prompt with the intent to derive a recommendation for resolving the violation; and the visualization step may further include a recommendation layer overlay step for overlaying a recommendation layer displaying the violation and a recommendation corresponding to the violation on one side of the original text when a selection input for the specific marking area is received.

[0016] In order to solve the above-mentioned problem, one embodiment of the present invention comprises a server system that performs a method for regulatory search and violation analysis for a product using a big language model, the server system comprising one or more processors and one or more memories, and includes: a document receiving unit that receives a product technical document for a product from a user terminal; a keyword extraction unit that extracts one or more regulatory keywords by inputting a first prompt, which includes the received product technical document and a intent to extract keywords related to regulations from the product technical document, into an internal or external big language model; a keyword search unit that derives first related regulatory data by searching for text data matching the regulatory keywords from a database in which current regulatory data and regulatory data scheduled for future implementation are respectively stored as text data and embedding data; and an embedding generation unit that divides the product technical document into text and image chunks, and inputs each divided chunk into a trained artificial neural network-based embedding model to generate a plurality of query embeddings. The present invention provides a server system comprising: an embedding search unit that calculates embedding similarity between embedding data stored in the database and the query embedding, and derives second related regulatory data associated with the product technical document based on the calculated embedding similarity; and a violation analysis unit that inputs a second prompt into the big language model, which includes the product technical document; the first related regulatory data; the second related regulatory data; and the intent to determine whether the contents described in the product technical document violate the judgment criteria based on the judgment criteria of the first related regulatory data and the second related regulatory data, thereby deriving violations of regulations and legal grounds corresponding to the violations. Effects of the invention

[0018] In one embodiment of the present invention, by applying a hybrid search method that combines text data search using regulatory keywords and embedding data search using artificial neural network-based query embeddings, it is possible to precisely derive contextually related regulatory data without omission even if the terms do not exactly match, thereby achieving the effect of dramatically increasing the accuracy of regulatory search.

[0019] In one embodiment of the present invention, by reflecting user request information, such as review directions or concerns entered by the user, into the analysis prompt, it is possible to achieve the effect of providing customized regulatory violation analysis results that match the specific intentions of the user.

[0020] In one embodiment of the present invention, a marking is displayed on an area identified as a violation in the original text of a product technical document to visualize it, and the violation and legal grounds are provided in the form of a violation layer pop-up according to the position of an input pointer, thereby enabling the user to intuitively identify the location of regulatory violations within a vast document.

[0021] In one embodiment of the present invention, violations according to current regulatory data and regulatory data scheduled for future implementation are visually distinguished and displayed using different first marking elements and second marking elements, thereby enabling the user to proactively respond to changes in the regulatory environment or product launch time.

[0022] In one embodiment of the present invention, recommendations for resolving violations are derived together through a large language model and provided as a recommendation layer, thereby enabling the effect of suggesting practical design modifications and solutions beyond simple problem identification.

[0023] In one embodiment of the present invention, by converting non-standard terms within a query text entered by a user into standard terms to generate expanded text and utilizing it for regulatory data search, it is possible to achieve the effect of ensuring standardized and accurate regulatory search results at all times, regardless of the user's query level or term selection. Brief explanation of the drawing

[0025] FIG. 1 schematically illustrates the steps of a method for regulatory search and violation analysis for a product using a big language model according to one embodiment of the present invention. FIG. 2 schematically illustrates the detailed components of a server system that performs a method for regulatory search and violation analysis of a product using a big language model according to one embodiment of the present invention. FIG. 3 schematically illustrates the process of performing a keyword extraction step and a keyword search step according to an embodiment of the present invention. FIG. 4 schematically illustrates the process of performing the embedding generation step and the embedding search step according to one embodiment of the present invention. FIG. 5 illustrates, in an exemplary manner, a process for comparing embedding similarity between embedding data and query embeddings according to an embodiment of the present invention. FIG. 6 schematically illustrates the process of performing a violation analysis step according to one embodiment of the present invention. FIG. 7 schematically illustrates the process of performing the request reception step and the request reflection step according to another embodiment of the present invention. FIG. 8 illustrates, exemplarily, the process of performing a term normalization step according to one embodiment of the present invention. FIG. 9 schematically illustrates the process of performing a visualization step according to one embodiment of the present invention. FIG. 10 schematically illustrates a first marking element and a second marking element according to an embodiment of the present invention. FIG. 11 illustrates an exemplary recommendation layer overlaid on one side of a text according to one embodiment of the present invention. FIG. 12 illustrates, in an exemplary manner, the internal configuration of a computing device according to one embodiment of the present invention. Specific details for implementing the invention

[0026] Hereinafter, various embodiments and / or aspects are disclosed with reference to the drawings. For illustrative purposes, numerous specific details are disclosed in the following description to aid in a general understanding of one or more aspects. However, it will also be recognized by those skilled in the art that these aspects may be practiced without such specific details. The following description and the accompanying drawings describe specific exemplary aspects of one or more aspects in detail. However, these aspects are exemplary, and some of the various methods in the principles of the various aspects may be used, and the description is intended to include all such aspects and their equivalents.

[0028] In addition, various aspects and features will be presented by a system that may include multiple devices, components and / or modules, etc. It should also be understood and recognized that various systems may include additional devices, components and / or modules, etc., and / or may not include all of the devices, components, modules, etc. discussed in relation to the drawings.

[0029] Terms such as “embodiment,” “example,” “aspect,” “example,” etc. as used herein may not be interpreted as implying that any aspect or design described is superior or more advantageous than other aspects or designs. Terms used below, such as “part,” “component,” “module,” “system,” “interface,” etc., generally refer to computer-related entities and may, for example, refer to hardware, a combination of hardware and software, or software.

[0030] Additionally, the terms “comprising” and / or “comprising” should be understood to mean that the relevant feature and / or component is present, but not to exclude the presence or addition of one or more other features, components and / or groups thereof.

[0031] Additionally, terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0032] The “user terminal” mentioned below may be implemented as a computer or portable terminal capable of connecting to a server or other terminal via a network. Here, the computer includes, for example, a notebook, desktop, or laptop equipped with a web browser, and the portable terminal may include, for example, all types of handheld-based wireless communication devices that ensure portability and mobility, such as smartphones, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), and Wibro (Wireless Broadband Internet) terminals. In addition, the “network” can be implemented as a wired network such as a Local Area Network (LAN), Wide Area Network (WAN), or Value Added Network (VAN), or as any type of wireless network such as a mobile radio communication network or a satellite communication network.

[0033] Furthermore, in the embodiments of the present invention, all terms used herein, including technical or scientific terms, unless otherwise defined, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in the embodiments of the present invention.

[0035] FIG. 1 schematically illustrates the steps of a method for detecting regulations and analyzing violations of a product using a big language model according to one embodiment of the present invention, and FIG. 2 schematically illustrates the detailed components of a server system for detecting regulations and analyzing violations of a product using a big language model according to one embodiment of the present invention.

[0037] As illustrated in FIGS. 1 and 2, a method for regulatory search and violation analysis of a product using a large language model (3000), performed by a server system (1000) comprising one or more processors and one or more memories, comprises: a document reception step (S100) of receiving a product technical document for a product from a user terminal (2000); a keyword extraction step (S200) of inputting a first prompt, which includes the received product technical document and a intent to extract keywords related to regulation from the product technical document, into an internal or external large language model (3000) to extract one or more regulatory keywords; and a keyword search step (S300) of deriving first related regulatory data by searching for text data matching the regulatory keywords from a database in which current regulatory data and regulatory data scheduled for future implementation are respectively stored as text data and embedding data. An embedding generation step (S400) for dividing the above product technical document into text and image chunks, and inputting each divided chunk into a learned artificial neural network-based embedding model to generate a plurality of query embeddings; an embedding search step (S500) for calculating embedding similarity between the embedding data previously stored in the database and the query embeddings, and deriving second related regulatory data associated with the above product technical document based on the calculated embedding similarity; and the above product technical document; the above first related regulatory data; the above second related regulatory data; A method for regulatory search and violation analysis is provided, comprising: a violation analysis step (S600) for deriving violations of a regulation and legal grounds corresponding to the violations by inputting a second prompt into the big language model (3000), which includes the intent to determine whether the contents described in the product technical document violate the judgment criteria based on the first relevant regulatory data and the second relevant regulatory data; and inputting the second prompt, which includes the intent to determine whether the contents described in the product technical document violate the judgment criteria.

[0039] Specifically, the method for regulatory search and violation analysis according to one embodiment of the present invention begins by receiving product technical documents, such as specifications, design drawings, and manuals, from a user terminal (2000) through the document reception step (S100).

[0040] Subsequently, in the keyword extraction step (S200) and the keyword search step (S300), the context of the document is identified and key regulatory keywords are extracted using a large language model (3000), and then text data within the database is searched based on this to secure the first relevant regulatory data.

[0041] In parallel with this, in the embedding generation step (S400) and the embedding search step (S500), the product technical document is divided into chunks of semantic units, and a query embedding is generated by vectorizing them. The generated query embedding is searched by comparing vector similarity with regulatory embedding data in the database, thereby deriving second related regulatory data that is contextually similar even if the keywords do not exactly match.

[0042] Finally, in the violation analysis step (S600), all collected information is input into the large language model (3000), and a precise violation analysis based on actual laws is performed through search augmented generation (RAG) technology. This suppresses the hallucinatory phenomenon of the large language model (3000) and provides highly reliable analysis results.

[0044] The above server system (1000) may be physically implemented as a computing device including a processor for data processing, a memory for storing instructions, and a network interface for communication with an external device.

[0045] The above document receiving unit (1100) communicates with the user terminal (2000) to receive a product technical document corresponding to the document to be analyzed.

[0046] The keyword extraction unit (1200) and the violation analysis unit (1600) perform natural language processing tasks by utilizing an external large language model (3000) connected via API or within the server system (1000).

[0047] The keyword search unit (1300) and the embedding search unit (1500) access a database storing a vast amount of statutes, notices, and standard data to perform a search. The database includes not only text-based data but also vector-based embedding data for semantic search, and maintains the latest regulatory information through periodic updates.

[0048] The above embedding generation unit (1400) runs an embedding model that vectorizes the content of a document, which can be performed within the server system (1000) or by calling an external embedding API.

[0049] Through the organic combination of these components, the server system (1000) can provide accurate and rapid regulatory violation analysis services regardless of the user's level of expertise.

[0051] In one embodiment of the present invention, by applying a hybrid search method that combines text data search using regulatory keywords and embedding data search using artificial neural network-based query embeddings, it is possible to precisely derive contextually related regulatory data without omission even if the terms do not exactly match, thereby achieving the effect of dramatically increasing the accuracy of regulatory search.

[0053] FIG. 3 schematically illustrates the process of performing a keyword extraction step and a keyword search step according to an embodiment of the present invention.

[0055] As illustrated in FIG. 3, the regulatory search and violation analysis method comprises: a keyword extraction step (S200) for extracting one or more regulatory keywords by inputting a first prompt, which includes a received product technical document and a purpose to extract keywords related to regulation from the product technical document, into an internal or external large language model (3000); and a keyword search step (S300) for deriving one or more first related regulatory data by searching for text data that matches each of the one or more regulatory keywords from a database in which current regulatory data and regulatory data scheduled for future implementation are respectively stored as text data and embedding data.

[0057] Specifically, as illustrated in FIG. 3(a), the keyword extraction unit (1200) generates a first prompt containing the intent to extract keywords related to regulation from the product technical document and the product technical document in order to analyze the received product technical document, and inputs this into the large language model (3000). This is intended to induce the large language model (3000) to select only core words that are highly likely to be directly related to laws or regulatory standards from among the numerous technical terms in the product technical document and output one or more regulatory keywords.

[0059] Next, as illustrated in FIG. 3(b), the keyword search unit (1300) searches for text data in the database using the extracted one or more regulatory keywords. The database is constructed with text data and embedding data that includes not only current laws and regulations but also notices and standard information scheduled to be implemented in the future, enabling comprehensive searching.

[0060] The keyword search unit (1300) compares the extracted regulatory keywords with text data in the database to derive regulatory data that exactly or partially matches words or codes as first related regulatory data.

[0061] In one embodiment of the present invention, the effect of quickly and accurately identifying relevant regulations based on clear legal terms or standard numbers can be achieved, and in addition, if there are multiple search results, all of them are included to derive first relevant regulatory data, thereby enabling a broad review of potential regulatory risks related to the product.

[0063] FIG. 4 schematically illustrates the process of performing the embedding generation step and the embedding search step according to one embodiment of the present invention, and FIG. 5 exemplarily illustrates the process of comparing embedding similarity between embedding data and query embeddings according to one embodiment of the present invention.

[0065] As illustrated in FIGS. 4 and 5, the regulatory search and violation analysis method comprises: an embedding generation step (S400) of dividing the product technical document into text and image chunks and inputting each divided chunk into a learned artificial neural network-based embedding model to generate a plurality of query embeddings; and an embedding search step (S500) of calculating the embedding similarity between the embedding data stored in the database and the query embeddings, and deriving second related regulatory data associated with the product technical document based on the calculated embedding similarity.

[0067] Specifically, as illustrated in FIG. 4(a), the embedding generation step (S400) begins with the process of dividing the received product technical document into chunks of a preset size or semantic unit. Since the received product technical document may include not only text but also images such as drawings and graphs, the embedding generation unit (1400) can identify and process the text chunks and image chunks, respectively.

[0068] Each divided chunk (Chunk #1 to Chunk #3) is input into a trained artificial neural network-based embedding model. The embedding model converts the input unstructured data into query embeddings (V), which are coordinate values ​​in a high-dimensional vector space.Q1 to V Q3 It is converted into ). The embedding model used at this time may be a pre-trained model specialized in legal and technical terms, or a model fine-tuned through a regulatory dataset.

[0070] The subsequent embedding search step (S500) is performed by the embedding search unit (1500), and searches for regulatory information within the database using the generated query embedding. The database contains current regulatory data and regulatory data scheduled for future implementation, not only as text data but also as pre-vectorized embedding data (V T1 , V T2 , V T3 It is stored in the form of , 쪋).

[0071] As shown in FIG. 5, the above embedding search unit (1500) is a query embedding (V) generated from a product technical document. Q ) and multiple regulatory embedding data (V) stored in the database T Calculate the embedding similarity between ). Cosine similarity, Euclidean distance, etc., can be used as similarity calculation methods.

[0073] As an embodiment of the present invention illustrated in FIG. 5, a query embedding (V Q1 ) and the first regulation embedding data (V T1 The similarity between ) is 0.103, and the second regularization embedding data (V T2 The similarity with ) is 0.985, and the third regularization embedding data (V T3 The similarity with ) can be calculated as 0.215.

[0074] The above embedding search unit (1500) selects regulatory data that has a calculated similarity greater than or equal to a preset threshold (e.g., 0.75) and has the highest similarity ranking, and derives it as second related regulatory data. For example, regulatory data with the highest similarity of 0.985 (VT2 Regulations corresponding to ) are derived as second-related regulatory data.

[0076] In one embodiment of the present invention, through this embedding-based search, cases where the contextual meaning is similar even if the forms of the words are different can be determined with a high degree of similarity, thereby compensating for the limitations of the keyword matching method and increasing the recall rate of the search. The derived second relevant regulatory data is subsequently provided as a judgment criterion for the big language model (3000) together with the first relevant regulatory data in the violation analysis step (S600).

[0078] FIG. 6 schematically illustrates the process of performing a violation analysis step according to one embodiment of the present invention.

[0080] As illustrated in FIG. 6, the regulatory search and violation analysis method comprises a violation analysis step (S600) of deriving violations of regulations and legal grounds corresponding to violations by inputting a second prompt into the big language model (3000), which includes the product technical document; the first relevant regulatory data; the second relevant regulatory data; and the intent to determine whether the contents described in the product technical document violate the judgment criteria based on the first relevant regulatory data and the second relevant regulatory data.

[0082] Specifically, the violation analysis unit (1600) according to one embodiment of the present invention integrates all information obtained in the aforementioned steps and performs an in-depth analysis.

[0083] The violation analysis unit (1600) collects the original or summary of the product technical document received in the document reception step (S100), the first relevant regulatory data derived in the keyword search step (S300), and the second relevant regulatory data derived in the embedding search step (S500).

[0085] Subsequently, the violation analysis unit (1600) organizes the collected information into a single integrated context to generate a second prompt. The second prompt does not merely list information but includes a clear instruction to the effect that "determine whether the contents described in the product technical document violate the judgment criteria by using the first relevant regulatory data and the second relevant regulatory data as judgment criteria." This assigns the specific role of a 'regulatory compliance determiner' to the large language model (3000).

[0087] When the generated second prompt is input into the above-mentioned macro language model (3000), the above-mentioned macro language model (3000) compares and analyzes the product specifications (Spec) and regulatory standards (Regulatory Standards) item by item. For example, if the product technical document states "battery capacity 5000mAh" and the collected regulatory data contains a clause stating "the battery capacity of a portable device cannot exceed 4000mAh," the above-mentioned macro language model (3000) identifies this as a violation.

[0089] Finally, the above-mentioned large language model (3000) derives and outputs pairs of identified violations and legal grounds serving as the basis for the judgment. In one embodiment of the present invention, by utilizing such search augmented generation (RAG) technology, the large language model (3000) can perform accurate analysis even on the latest regulations or confidential internal corporate documents that it has not learned, and can effectively suppress the hallucination phenomenon in which the large language model (3000) generates content different from the facts.

[0091] FIG. 7 schematically illustrates the process of performing a request reception step and a request reflection step according to another embodiment of the present invention, and FIG. 8 exemplarily illustrates the process of performing a term normalization step according to one embodiment of the present invention.

[0093] As illustrated in FIGS. 7 and 8, the regulatory search and violation analysis method further includes a request reception step of receiving user request information including query text entered by a user regarding the review direction or concerns of the product technical document from the user terminal (2000); and the violation analysis step (S600) further includes a request reflection step of deriving violations and legal grounds reflected in the user request information by inputting the user request information and the intent to perform analysis by considering the user's intent included in the user request information into the big language model (3000) in the second prompt, thereby including the user request information; and a statement to perform analysis by considering the user's intent included in the user request information when determining whether the judgment criteria are violated.

[0094] Additionally, the violation analysis step (S600) further includes a term normalization step, which, after the request reception step, analyzes the query text included in the user request information and converts non-standard terms, including non-technical terms and field terms within the query text, into standard terms, including standard technical terms and legal terms that match the regulatory data stored in the database, thereby generating expanded text; and the request reflection step further includes the expanded text when inputting the user request information into the second prompt, thereby causing the large language model (3000) to search for specific regulatory items that correspond to the user's intention.

[0096] Specifically, the request reception step is a step of receiving input in text form from the user terminal (2000) regarding directions or concerns that the user specifically wishes to review, in addition to product technical documents. For example, if the user inputs a query text such as "Does this product violate any waterproofing regulations?", the server system (1000) receives this as user request information.

[0098] The subsequent term normalization step is the process of converting non-technical or field terms included in the query text entered by the user into standard terms suitable for searching regulatory databases.

[0099] As an embodiment of the present invention illustrated in FIG. 8, when a user uses the general term "waterproof," the server system (1000) generates expanded text by mapping it to specific and specialized standard terms such as "IP rating," "waterproof standard," "water tightness," and "IEC 60529 standard." This can be done by utilizing an ontology or a thesaurus of the relevant technical field, going beyond simple synonym substitution.

[0101] In the above request reflection step, the refined user request information and extended text are included in the second prompt of the above violation analysis step (S600). That is, instead of simply giving a command to the giant language model (3000) to "find regulatory violations," specific instructions can be given to the effect that "since the user is concerned about 'waterproofing,' focus on reviewing regulatory provisions related to 'IP rating' and 'water tightness' to determine whether there is a violation."

[0102] Through this, the above-mentioned large language model (3000) can accurately identify the user's intention and provide clear analysis results regarding the part the user is most curious about among numerous regulatory data.

[0104] In one embodiment of the present invention, by reflecting user request information, such as review directions or concerns entered by the user, into the analysis prompt, it is possible to achieve the effect of providing customized regulatory violation analysis results that match the specific intentions of the user.

[0105] In addition, in one embodiment of the present invention, by converting non-standard terms within a query text entered by a user into standard terms to generate expanded text and utilizing it for regulatory data search, it is possible to achieve the effect of ensuring standardized and accurate regulatory search results at all times, regardless of the user's query level or term selection.

[0107] FIG. 9 schematically illustrates the process of performing a visualization step according to one embodiment of the present invention, and FIG. 10 schematically illustrates a first marking element and a second marking element according to one embodiment of the present invention.

[0109] As illustrated in FIGS. 9 and 10, the regulatory search and violation analysis method further includes a visualization step after the violation analysis step (S600) of displaying a marking on an area identified as a violation on the original text of the product technical document to visualize it; and the visualization step includes a violation layer popup step of popping up a violation layer (L1) on the original text that displays the violation and legal grounds when an input pointer is detected to be located on a specific marking area.

[0110] In addition, the visualization step provides a visual distinction of the types of violations according to the timing of regulatory application in the original text by displaying a first marking element (E1) in the area identified as the violation when the violation derived in the violation analysis step (S600) is an item based on current regulatory data, and displaying a second marking element (E2) in the area identified as the violation when the violation is an item based on regulatory data scheduled for future implementation.

[0112] Specifically, when it is determined through the violation analysis unit (1600) that a specific phrase in the product technical document violates regulations, the server system (1000) does not merely provide the user with a result report in text form, but visually displays the problematic part directly on the original document file uploaded by the user to enhance intuitiveness.

[0113] As illustrated in FIG. 9(a), a highlight, underline, or box-shaped marking is created in the problematic text or image area within the product technical document. When a user places an input pointer, such as a mouse cursor, over the corresponding marking area as illustrated in FIG. 9(b), the server system (1000) detects this and pops up a violation layer (L1).

[0114] The above violation layer (L1) includes a summary of the violation, such as "use of advertisement for efficacy outside of approval," and specific legal information serving as the basis for the judgment, such as "Article 68 of the Pharmaceutical Affairs Act (Prohibition of Exaggerated Advertising) and pharmaceuticals, etc...", so that the user can immediately identify the reason for the violation without having to search for a separate document.

[0116] Furthermore, since product development takes time from planning to launch, it is essential to review not only currently enforced regulations but also future regulations scheduled to be implemented by the time of product launch.

[0117] As illustrated in FIG. 10, the present invention applies different forms of marking depending on the nature of the violation. More specifically, a first marking element (E1) is applied to matters that violate current regulatory data that are currently in effect, and a second marking element (E2) is applied to matters that are not currently in violation but have the potential to violate regulatory data scheduled to be implemented in the future due to amendments to laws, etc.

[0118] As an embodiment of the present invention, the first marking element (E1) is marked with a red color to convey to the user a warning that immediate correction is required, and the second marking element (E2) is marked with a blue color to convey a caution that future risk management is required, thereby enabling the user to visually distinguish the timing of regulation application and the urgency of response through color.

[0120] In one embodiment of the present invention, a marking is displayed on an area identified as a violation in the original text of a product technical document to visualize it, and the violation and legal grounds are provided in the form of a violation layer pop-up according to the position of an input pointer, thereby enabling the user to intuitively identify the location of regulatory violations within a vast document.

[0121] In addition, in one embodiment of the present invention, violations according to current regulatory data and regulatory data scheduled for future implementation are visually distinguished and displayed using different first and second marking elements, thereby enabling the user to proactively respond to changes in the regulatory environment or product launch time.

[0123] FIG. 11 illustrates an exemplary recommendation layer overlaid on one side of a text according to one embodiment of the present invention.

[0125] As illustrated in FIG. 11, the violation analysis step (S600) further includes a recommendation derivation step for deriving recommendations corresponding to the violation by inputting into the large language model (3000) a second prompt with the intent to derive recommendations for resolving the violation; and the visualization step further includes a recommendation layer overlay step for overlaying a recommendation layer (L2) that displays the violation and recommendations corresponding to the violation on one side of the original text when a selection input for the specific marking area is received.

[0127] Specifically, the present invention goes beyond merely notifying the fact of a regulatory violation and provides a specific solution on how the user should rectify it.

[0128] When the violation analysis unit (1600) generates the second prompt in the violation analysis step (S600), it adds an instruction to the effect of, "provide specific recommendations based on relevant regulations on how to modify product specifications or document contents to resolve the identified violations."

[0129] Accordingly, the above-mentioned large language model (3000) traces back the legal standards included in the first relevant regulatory data and the second relevant regulatory data to generate a guide for converting a violation state into a compliant state. For example, regarding a violation of "discrepancy in labeling" as illustrated in FIG. 11, it derives specific action guidelines such as "modifying the efficacy and effect statements on the outer packaging and accompanying documents to be identical to the wording in the product approval." In addition, regarding a violation of "insufficient evidence for efficacy testing," it can provide not only simple text modifications but also procedural advice requiring additional experiments or data acquisition, such as "changing the test cycle to a 24-month stability re-evaluation cycle and submitting supporting data."

[0131] This information is conveyed to the user through the recommendation layer overlay stage of the visualization stage. When the user performs a selection input, such as clicking the marking area described in FIG. 9 or FIG. 10, a recommendation layer (L2) can be overlaid on one side of the document in a sliding manner, as shown in FIG. 11. The recommendation layer (L2) is configured so that the user can simultaneously compare and review the product technical document and the recommendations.

[0132] Inside the above recommendation layer (L2), current regulatory violations and regulatory violations scheduled for future implementation are displayed separately, and for each item, a correction guide generated by the above-mentioned giant language model (3000) is provided in the form of a list.

[0133] Furthermore, according to another embodiment of the present invention, a 'Reflect Recommendation' button is provided at the bottom of the recommendation layer (L2), so that when a user selects it, the macro language model (3000) can automatically modify the product technical document according to the recommended content or generate a modified draft, and this can be linked to a generative function.

[0135] In one embodiment of the present invention, recommendations for resolving violations are derived together through a large language model and provided as a recommendation layer, thereby enabling the effect of suggesting practical design modifications and solutions beyond simple problem identification.

[0137] FIG. 12 illustrates the internal configuration of a computing device (11000) according to one embodiment of the present invention.

[0139] The server system (1000) mentioned in the description of FIG. 1 may include components of the computing device (11000) illustrated in FIG. 12, which will be described later.

[0141] As illustrated in FIG. 12, the computing device (11000) may include at least one processor (11100), memory (11200), peripheral interface (11300), input / output subsystem (I / O subsystem) (11400), power circuit (11500), and communication circuit (11600).

[0143] Specifically, the memory (11200) may include, for example, high-speed random access memory, magnetic disk, SRAM, DRAM, ROM, flash memory, or non-volatile memory. The memory (11200) may include software modules, instruction sets, or various other data required for the operation of the computing device (11000).

[0144] At this time, access to the memory (11200) from other components, such as the processor (11100) or the peripheral device interface (11300), can be controlled by the processor (11100). The processor (11100) may be composed of a single or multiple units and may include processors in the form of GPUs and TPUs to improve computational processing speed.

[0145] The above peripheral device interface (11300) can connect input and / or output peripheral devices of the computing device (11000) to the processor (11100) and the memory (11200). The processor (11100) can perform various functions for the computing device (11000) and process data by executing a software module or instruction set stored in the memory (11200).

[0146] The input / output subsystem (11400) may connect various input / output peripheral devices to the peripheral device interface (11300). For example, the input / output subsystem (11400) may include a controller for connecting peripheral devices such as a monitor, keyboard, mouse, printer, or, if necessary, a touchscreen or sensor to the peripheral device interface (11300). According to another aspect, the input / output peripheral devices may be connected to the peripheral device interface (11300) without passing through the input / output subsystem (11400).

[0147] The power circuit (11500) may supply power to all or part of the components of the terminal. For example, the power circuit (11500) may include one or more power sources such as a power management system, a battery or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator, or any other components for power generation, management, and distribution.

[0148] The communication circuit (11600) may enable communication with another computing device using at least one external port. Alternatively, as described above, the communication circuit (11600) may enable communication with another computing device by including an RF circuit and transmitting and receiving an RF signal, also known as an electromagnetic signal, as needed.

[0150] The embodiment of FIG. 12 is merely an example of the computing device (11000), and the computing device (11000) may have some components shown in FIG. 12 omitted, additional components not shown in FIG. 12 added, or a configuration or arrangement that combines two or more components. For example, a computing device for a communication terminal in a mobile environment may include a touchscreen or sensors in addition to the components shown in FIG. 12, and the communication circuit (1160) may include a circuit for RF communication of various communication methods (Wi-Fi, 3G, LTE, 5G, 6G, Bluetooth, NFC, Zigbee, etc.). The components that can be included in the computing device (11000) may be implemented as hardware, software, or a combination of both hardware and software, including one or more integrated circuits specialized for signal processing or applications.

[0151] Methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded on a computer-readable medium. In particular, the program according to the present embodiment may be configured as a PC-based program or an application dedicated to a mobile terminal. An application to which the present invention is applied may be installed on a user terminal through a file provided by a file distribution system. For example, the file distribution system may include a file transmission unit (not shown) that transmits the file upon a request from the user terminal.

[0153] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0154] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave in order to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be standardized and stored or executed in a standardized manner on a networked computing device. Software and data may be stored on one or more computer-readable recording media.

[0155] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0157] In one embodiment of the present invention, by applying a hybrid search method that combines text data search using regulatory keywords and embedding data search using artificial neural network-based query embeddings, it is possible to precisely derive contextually related regulatory data without omission even if the terms do not exactly match, thereby achieving the effect of dramatically increasing the accuracy of regulatory search.

[0158] In one embodiment of the present invention, by reflecting user request information, such as review directions or concerns entered by the user, into the analysis prompt, it is possible to achieve the effect of providing customized regulatory violation analysis results that match the specific intentions of the user.

[0159] In one embodiment of the present invention, a marking is displayed on an area identified as a violation in the original text of a product technical document to visualize it, and the violation and legal grounds are provided in the form of a violation layer pop-up according to the position of an input pointer, thereby enabling the user to intuitively identify the location of regulatory violations within a vast document.

[0160] In one embodiment of the present invention, violations according to current regulatory data and regulatory data scheduled for future implementation are visually distinguished and displayed using different first marking elements and second marking elements, thereby enabling the user to proactively respond to changes in the regulatory environment or product launch time.

[0161] In one embodiment of the present invention, recommendations for resolving violations are derived together through a large language model and provided as a recommendation layer, thereby enabling the effect of suggesting practical design modifications and solutions beyond simple problem identification.

[0162] In one embodiment of the present invention, by converting non-standard terms within a query text entered by a user into standard terms to generate expanded text and utilizing it for regulatory data search, it is possible to achieve the effect of ensuring standardized and accurate regulatory search results at all times, regardless of the user's query level or term selection.

[0164] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents. Therefore, other implementations, other embodiments, and equivalents to the claims below are also within the scope of the claims.

Claims

Claim 1 A method for regulatory search and violation analysis of a product using a large language model, performed by a server system comprising one or more processors and one or more memories, comprising: a document reception step of receiving a product technical document regarding a product from a user terminal; and the received product technical document; A keyword extraction step of extracting one or more regulatory keywords by inputting a first prompt, including the intent to extract regulatory-related keywords from the product technical document, into an internal or external big language model; a keyword search step of deriving first relevant regulatory data by searching for text data matching the regulatory keywords from a database in which current regulatory data and regulatory data scheduled for future implementation are respectively stored as text data and embedding data; an embedding generation step of dividing the product technical document into text and image chunks and inputting each divided chunk into a trained artificial neural network-based embedding model to generate a plurality of query embeddings; an embedding search step of calculating embedding similarity between the embedding data stored in the database and the query embeddings, and deriving second relevant regulatory data associated with the product technical document based on the calculated embedding similarity; and the product technical document; the first relevant regulatory data; the second relevant regulatory data; The method for regulatory search and violation analysis includes a second prompt input into the big language model, which includes the intent to determine whether the contents described in the product technical document violate the judgment criteria based on the first relevant regulatory data and the second relevant regulatory data; and derives violations of regulations and legal grounds corresponding to the violations. The method further includes a request reception step of receiving user request information from the user terminal, which includes query text entered by the user regarding the review direction or concerns of the product technical document. The violation analysis step includes the user request information in the second prompt; and the intent to perform analysis by considering the user's intent included in the user request information when determining whether the judgment criteria are violated.The method further includes a request reflection step for deriving violations and legal grounds reflecting the user request information by inputting the user request information into the aforementioned large language model, wherein the violation analysis step further includes a term normalization step, wherein, after the request reception step, the user request information is analyzed to generate expanded text by converting non-standard terms, including non-technical and field terms within the user request information, into standard terms, including standard technical and legal terms that match regulatory data stored in the database, wherein the user request information is input into the second prompt, the user request information is input with the expanded text included therein, thereby causing the large language model to search for specific regulatory items corresponding to the user's intent, and the method for regulatory search and violation analysis further includes a visualization step, wherein, after the violation analysis step, the user request information is visualized by marking an area identified as a violation on the original text of the product technical document, and the visualization step, when it detects that an input pointer is located on a specific marked area, the corresponding violation and A method for regulatory search and violation analysis, comprising: a violation layer popup step of popping up a violation layer indicating a legal basis on the original text; Claim 2 delete Claim 3 delete Claim 4 A method for regulatory search and violation analysis according to claim 1, wherein the visualization step displays a first marking element in an area identified as a violation when the violation derived in the violation analysis step is an item based on current regulatory data, and displays a second marking element in an area identified as a violation when the violation is an item based on regulatory data scheduled for future implementation, thereby visually distinguishing and providing the type of violation according to the timing of regulatory application in the original text. Claim 5 A method for regulatory search and violation analysis according to claim 1, wherein the violation analysis step further includes a recommendation derivation step for deriving recommendations corresponding to the violation by inputting into the big language model a second prompt with the intent to derive recommendations for resolving the violation; and the visualization step further includes a recommendation layer overlay step for overlaying a recommendation layer displaying the corresponding violation and recommendations corresponding to the violation on one side of the original text when receiving a selection input for the specific marking area. Claim 6 A server system comprising one or more processors and one or more memories, and performing a method for regulatory search and violation analysis of a product using a large language model, comprising: a document receiving unit that receives a product technical document regarding a product from a user terminal; and the received product technical document; A keyword extraction unit that extracts one or more regulatory keywords by inputting a first prompt, including the intent to extract regulatory-related keywords from the product technical document, into an internal or external big language model; a keyword search unit that derives first related regulatory data by searching for text data matching the regulatory keywords from a database in which current regulatory data and regulatory data scheduled for future implementation are respectively stored as text data and embedding data; an embedding generation unit that generates a plurality of query embeddings by dividing the product technical document into text and image chunks and inputting each divided chunk into a trained artificial neural network-based embedding model; an embedding search unit that calculates the embedding similarity between the embedding data stored in the database and the query embeddings, and derives second related regulatory data associated with the product technical document based on the calculated embedding similarity; and the product technical document; the first related regulatory data; the second related regulatory data; The violation analysis unit includes a second prompt, which inputs the second prompt into the big language model to determine whether the contents described in the product technical document violate the judgment criteria based on the first relevant regulatory data and the second relevant regulatory data, and derives violations of regulations and legal grounds corresponding to the violations; and the regulatory search and violation analysis method includes a request reception step of receiving user request information including query text entered by the user regarding the review direction or concerns of the product technical document from the user terminal; and the violation analysis unit includes the user request information in the second prompt; and a notice to perform analysis by considering the user's intent included in the user request information when determining whether the judgment criteria are violated.A request reflection step is performed to derive violations and legal grounds reflecting the user request information by further including and inputting into the aforementioned large language model; and the violation analysis unit further performs a term normalization step after the request reception step, which analyzes the query text included in the user request information and converts non-standard terms, including non-technical and field terms within the query text, into standard terms, including standard technical and legal terms that match the regulatory data stored in the database, to generate expanded text; and the request reflection step enables the large language model to search for specific regulatory items corresponding to the user's intent by further including and inputting the expanded text when inputting the user request information into the second prompt; and the regulatory search and violation analysis method further includes a visualization step that visualizes by marking the area identified as the violation on the original text of the product technical document based on the violations derived by the violation analysis unit; and the visualization step indicates that an input pointer is located on a specific marked area A server system comprising: a violation layer popup step that, upon detection, pops up a violation layer on the original text indicating the relevant violation and legal grounds;

Images