Method, computing device and computer program for providing search results for target companies for insurance sales based on artificial intelligence

Abstract

A method for providing search results for insurance sales target companies based on artificial intelligence, a computing device, and a computer program are provided. A method for providing search results for insurance sales target companies based on artificial intelligence according to various embodiments of the present disclosure, wherein the method is performed by a computing device, comprises the steps of: obtaining a natural language query including search conditions regarding an insurance sales target company; generating a query in the form of a structured query based on the obtained natural language query through a previously trained artificial intelligence model; and providing search results for said insurance sales target companies derived by querying a corporate information database using said generated query.

Description

Technology Field

[0001] Various embodiments of the present disclosure relate to a method, apparatus, and computer program for providing search results for companies targeted for insurance sales based on artificial intelligence. Background Technology

[0002] Recently, the importance of corporate sales in the insurance industry has been continuously increasing, and group insurance sales, in particular, have established themselves as a core business area for securing stable profits and maintaining long-term customer relationships. The success of such corporate sales largely depends on how effectively appropriate target companies are identified and selected; in particular, securing a list of target companies for sales activities is recognized as the first and most critical step in the sales process.

[0003] Traditionally, the identification of target companies for insurance sales has been carried out primarily through manual database processing. For instance, when an insurance consultant required a list of potential sales targets, it was common practice to request a so-called "advanced database" from headquarters or a support department; in this case, personnel would manually search and filter corporate information databases to extract a list of companies meeting specific criteria, which would then be provided in the form of an Excel file.

[0004] The problem of excessively long data processing times in these conventional methods has been continuously pointed out. In practice, it typically took two to three days to create a single advanced database, which acted as a factor making timely responses difficult in a rapidly changing business environment. Furthermore, the structure in which data processing tasks were concentrated on specific personnel led to increased workload, and processing delays were inevitable when multiple requests occurred simultaneously.

[0005] In particular, for entry-level insurance consultants lacking sales experience, database-based customer acquisition is often virtually their only means of sales activity, as they have not established networks or sales markets. However, under conventional methods, long waiting times for database provision and repeated requests when the provided data is unsuitable for actual sales situations have become factors hindering the establishment of new entrants in the sales industry.

[0006] Furthermore, with the recent advancement of sales strategies, search conditions are becoming increasingly complex. There is a growing demand for multi-layered filtering that goes beyond simple criteria such as region, industry, and sales revenue, including financial ratio calculations, logical operations (AND, OR) on multiple conditions, and the possession of specific certifications. However, processing these complex conditions manually using Excel requires significant technical proficiency, and there are limitations where processing may become impossible or errors are likely to occur depending on the combination of conditions.

[0007] As an attempt to address these issues, some systems are being introduced that provide web-based search interfaces for corporate information databases or allow users to directly select search conditions through predefined filter options. For example, a drop-down menu method is used to view company lists by selecting conditions such as region, industry, and size; however, there are still limitations in expressing complex combinations of conditions, and it remains difficult to accurately convey the user's search intent to the system.

[0008] Furthermore, one of the critical factors in searching for target companies for insurance sales is the structural complexity of the corporate information database. Generally, corporate information databases consist of dozens to hundreds of columns, and it is difficult to formulate appropriate search queries without accurately understanding the names and meanings of each column. In conventional technology, performing searches without a sufficient understanding of this database structure frequently resulted in problems such as failure to obtain desired results or system performance degradation caused by inappropriate search conditions.

[0009] As such, while the need for automation and intelligent technologies is increasing to enhance the speed, accuracy, and usability of searching for insurance sales target companies, existing methods suffer from limitations in processing speed, difficulties in handling complex conditions, and insufficient understanding of user intent. Consequently, a more advanced technological approach is required in the field of insurance sales support.

[0010] The aforementioned background technology is one that the inventor possessed or acquired in the process of deriving the content of the present disclosure, and it cannot be considered as prior art disclosed to the general public prior to the filing of this application. The problem to be solved

[0011] The problem that the present disclosure aims to solve is to provide an AI-based method for providing search results for insurance sales target companies, a computing device, and a computer program, which, for the purpose of resolving the aforementioned conventional problems, automatically convert a natural language query entered by a user into a query in the form of a structured query language through a pre-trained AI model and query a corporate information database, thereby enabling real-time processing of advanced database operations that previously took several days, and thereby significantly improving the efficiency and accessibility of searching for insurance sales target companies.

[0012] Another problem that the present disclosure aims to solve is to provide an AI-based method for providing search results for insurance sales target companies, a computing device, and a computer program that utilizes embedding models and semantic search technology to precisely extract relevant database column information from a user's natural language query, selects similar reference queries, and uses them to generate queries, thereby enabling general users to easily search without specifying complex search conditions and effectively supporting the sales activities of new insurance consultants, particularly those lacking sales experience.

[0013] Another problem that the present disclosure aims to solve is to provide an AI-based method for providing search results for insurance sales target companies, a computing device, and a computer program that can ensure the accuracy and quality of a query and prevent database query errors and performance issues in advance by performing multi-stage validation using a large-scale language model on a generated query to systematically check feasibility, search field suitability, the presence of performance degradation factors, and the presence of unnecessary conditions.

[0014] Another problem that the present disclosure aims to solve is to provide an AI-based method for providing search results for insurance sales target companies, a computing device, and a computer program that can continuously support the user until they obtain the correct search results, rather than simply ending in a search failure, by improving the query through an automatic correction mechanism when the query fails to pass validity verification, or by providing a re-entry request message including a specific correction guide when correction is difficult, and further provide detailed information about the searched companies to comprehensively support the entire process of sales activities.

[0015] The problems that this disclosure aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below. means of solving the problem

[0016] The present disclosure may be implemented in various ways, including a method, an apparatus (system), or a computer program stored on a readable storage medium.

[0017] A method for providing search results for an insurance sales target company based on artificial intelligence according to one embodiment of the present disclosure for solving the above-mentioned problem, wherein the method for providing search results for an insurance sales target company based on artificial intelligence is performed by a computing device, may include the steps of: obtaining a natural language query including search conditions regarding an insurance sales target company; generating a query in the form of a structured query based on the obtained natural language query through a previously trained artificial intelligence model; and providing search results for an insurance sales target company derived by querying an enterprise information database using the generated query.

[0018] In various embodiments, the pre-trained artificial intelligence model includes a Large Language Model (LLM) configured to generate a query in the form of a structured query and to validate the generated query, and the step of generating a query may include: a step of extracting mapping column information corresponding to an acquired natural language query among a plurality of column information for an enterprise information database; a step of selecting at least one reference query among a plurality of reference queries prepared in advance based on the extracted mapping column information; a step of generating a preliminary query by inputting the acquired natural language query, the extracted mapping column information, the selected reference query, and a preliminary basic prompt into the Large Language Model as input data; and a step of performing validation on the generated preliminary query and generating a query that passes the validation as a final query.

[0019] In various embodiments, the pre-trained artificial intelligence model further includes an embedding model configured to output a result of converting input text data into a vector, which is pre-trained based on training data including insurance-related term information, industry classification information, and corporate information database information, and the step of extracting mapping column information corresponding to an acquired natural language query includes the step of converting the acquired natural language query into a query vector using the embedding model, the step of acquiring a plurality of previously converted column vectors for each of the plurality of column information, the step of calculating a similarity between the converted query vector and each of the acquired plurality of column vectors, and the step of extracting column information corresponding to a column vector among the plurality of column vectors whose calculated similarity is greater than or equal to a reference value as mapping column information, and the mapping column information may include a column name and descriptive information regarding the column name.

[0020] In various embodiments, the step of selecting at least one reference query may include: obtaining a previously transformed mapping column vector for extracted mapping column information; obtaining a previously transformed plurality of reference query vectors for each of a plurality of reference queries prepared in advance; calculating a similarity between the obtained mapping column vector and each of the obtained plurality of reference query vectors; and selecting a predetermined number of reference query vectors in order starting from the reference query vector with the highest calculated similarity among the obtained plurality of reference query vectors, and selecting them as reference queries corresponding to the selected reference query vectors.

[0021] In various embodiments, a pre-prepared basic prompt includes work instructions for a large-scale language model to generate a query in the form of a structured query, a description of an enterprise information database, and query writing rules, and the step of generating a preliminary query may include the step of configuring a query generation prompt by combining a natural language query obtained from the pre-prepared basic prompt, extracted mapping column information, and a selected reference query, and the step of inputting the configured query generation prompt into a large-scale language model to generate a preliminary query in the form of a structured query.

[0022] In various embodiments, the step of generating a final query includes the step of outputting a validation result for a generated preliminary query as input data into a large-scale language model using the generated preliminary query and a pre-prepared validation prompt, and the step of confirming a query with a passing validation result as the final query. The pre-prepared validation prompt may include work instructions and validation criteria for the large-scale language model to perform validation of the input query.

[0023] In various embodiments, the step of outputting a validation result may include: a first validation step of deriving a first validation result by verifying whether a preliminary query generated using a large-scale language model is an executable query in an enterprise information database; a second validation step of deriving a second validation result by verifying whether a search field included in a preliminary query generated using a large-scale language model corresponds to a search condition regarding an insurance sales target enterprise; a third validation step of deriving a third validation result by verifying whether a preliminary query generated using a large-scale language model includes a performance degradation factor of the enterprise information database; a fourth validation step of deriving a fourth validation result by verifying whether a preliminary query generated using a large-scale language model includes unnecessary conditions or unnecessary search fields; and a step of outputting a validation result as "passed" when the derived first validation result, the derived second validation result, the derived third validation result, and the derived fourth validation result are all "passed."

[0024] In various embodiments, the step of generating a final query may further include: when at least one of the derived first verification result, derived second verification result, derived third verification result, and derived fourth verification result is not a pass, generating correction data for a preliminary query generated to correspond to the verification step where the non-pass verification result was derived using a large-scale language model; calculating a correction difficulty for the generated correction data; when the calculated correction difficulty is less than a threshold value, correcting the generated preliminary query based on the generated correction data to generate a correction query; and re-performing the first verification step, second verification step, third verification step, and fourth verification step for the generated correction query to confirm the correction query that has passed each of the respective verification steps as the final query.

[0025] In various embodiments, the step of generating correction data includes: using a large-scale language model to identify at least one part requiring correction that indicates a target requiring correction in a preliminary query generated based on the type of verification step from which a non-pass verification result is derived; identifying, for the identified part requiring correction, any one of deletion correction, addition correction, and modification correction; generating a correction element corresponding to any one of the identified correction types; and applying the generated correction element to the identified part requiring correction to generate correction data. The step of calculating the correction difficulty may include assigning a score to each of the generated correction elements included in the generated correction data and determining the correction difficulty based on the assigned score.

[0026] In various embodiments, the step of determining the correction difficulty may include: assigning a preset base score to each of the generated correction elements included in the generated correction data, corresponding to one of the correction types of deletion, addition, and change; calculating a weighted score for each of the generated correction elements by applying different weights to the base score assigned according to the type of verification step that was not passed; and determining the final correction difficulty based on the final score obtained by summing each of the calculated weighted scores.

[0027] In various embodiments, the step of generating a query may further include the step of requesting a user terminal to re-enter a natural language query if the generated preliminary query fails to pass the validation performed.

[0028] In various embodiments, the step of requesting a user terminal to re-enter a natural language query may include the step of analyzing the cause of failure to pass validation using a large-scale language model and the step of transmitting a natural language query re-entry request message to the user terminal, which includes a correction guide for the generated preliminary query that failed validation based on the analyzed cause.

[0029] In various embodiments, the step of providing search results may include the step of providing a list of companies targeted for insurance sales in the form of a list, and, when any one of the provided list of companies is selected, the step of providing corporate management summary information including at least one of financial status, industry characteristics, and management status for the selected company.

[0030] A computing device according to another embodiment of the present disclosure for solving the above-described problem comprises a memory and at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, and the at least one program may include instructions for performing a method for providing search results for an artificial intelligence-based insurance sales target company when executed by at least one processor.

[0031] A computer program according to another embodiment of the present disclosure for solving the above-described problem may be stored on a computer-readable recording medium to execute a method for providing search results for an artificial intelligence-based insurance sales target company on a computing device.

[0032] Other specific details of the present disclosure are included in the detailed description and drawings. Effects of the invention

[0033] According to various embodiments of the present disclosure, by converting a natural language query entered by a user into a query in the form of a structured query language through a pre-trained artificial intelligence model and querying a corporate information database, advanced database operations that previously took several days can be processed in real time, thereby offering the advantage of significantly improving the efficiency and accessibility of searching for companies targeted for insurance sales.

[0034] In addition, by utilizing embedding models and semantic search technology to precisely extract relevant database column information from users' natural language queries and selecting similar reference queries to use in query generation, general users can easily search without specifying complex search conditions. This offers the advantage of effectively supporting the sales activities of new insurance consultants, particularly those lacking sales experience.

[0035] In addition, by performing multi-stage validation using a large-scale language model on the generated queries to systematically check feasibility, search field suitability, the presence of performance degradation factors, and the inclusion of unnecessary conditions, there is an advantage in guaranteeing the accuracy and quality of the queries and preventing database query errors and performance issues in advance.

[0036] In addition, if a query fails validation, it can be improved through a correction mechanism; if correction is difficult, a re-entry request message including specific modification guides is provided. This allows for continuous support until the user obtains the correct search results, rather than simply ending in a search failure, and furthermore, provides detailed information about the found companies, offering the advantage of providing integrated support for the entire sales process.

[0037] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0038] The following drawings attached to this specification illustrate preferred embodiments of the present disclosure and serve to further enhance understanding of the technical concept of the present disclosure together with the detailed description of the invention; therefore, the present disclosure should not be interpreted as being limited only to the matters described in such drawings. FIG. 1 is a diagram illustrating an artificial intelligence-based insurance sales target company search result provision system according to one embodiment of the present disclosure. FIG. 2 is a diagram illustrating the hardware configuration of a computing device according to another embodiment of the present disclosure. FIG. 3 is a flowchart of a method for providing search results for companies targeted for insurance sales based on artificial intelligence according to another embodiment of the present disclosure. FIG. 4 is a flowchart of a method for generating a query in the form of a structured query in various embodiments. FIG. 5 is a diagram illustrating the process of generating a query in the form of a structured query in various embodiments. FIG. 6 is a flowchart of a method for generating a final query from a query that has passed validation in various embodiments. FIG. 7 is a flowchart of a method for outputting validation results in various embodiments. FIG. 8 is a flowchart of a method for generating a final query when the validation result fails in various embodiments. FIG. 9 is a flowchart of a method for requesting re-entry of a natural language query when the validation result fails in various embodiments. FIGS. 10 to 12 are drawings illustrating an exemplary user interface (UI) of an artificial intelligence-based insurance sales target company search result provision service applicable to various embodiments. Specific details for implementing the invention

[0039] The advantages and features of the present disclosure and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure is complete and to fully inform those skilled in the art of the scope of the present disclosure, and the present disclosure is defined only by the scope of the claims.

[0040] The terms used herein are for describing the embodiments and are not intended to limit the disclosure. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. As used herein, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the components mentioned.

[0041] Throughout this specification, the same reference numerals refer to the same components, and "and / or" includes each of the mentioned components and all combinations of one or more thereof. Although terms such as "first," "second," etc., are used to describe various components, they are not limited by these terms. These terms are used merely to distinguish one component from another. Accordingly, the first component mentioned below may be the second component within the technical scope of this disclosure.

[0042] As used herein, the terms “part” or “module” refer to hardware components such as software, FPGAs, or ASICs, and the “part” or “module” performs certain roles. However, the “part” or “module” is not limited to software or hardware. The “part” or “module” may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, by example, the “part” or “module” includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules,” or further separated into additional components and “parts” or “modules.”

[0043] Spatially relative terms such as "below," "beneath," "lower," "above," and "upper" may be used to facilitate the description of the relationship between one component and other components as illustrated in the drawings. Spatially relative terms should be understood as encompassing different orientations of components during use or operation, in addition to the orientations depicted in the drawings. For example, if a component depicted in a drawing is inverted, a component described as "below" or "beneath" of another component may be placed "above" of that component. Therefore, the exemplary term "below" may encompass both the lower and upper directions. Components may also be oriented in other directions, and accordingly, spatially relative terms may be interpreted according to the orientation.

[0044] Expressions such as "first," "second," or "first," "second" as used in this specification are used to distinguish one object from another when referring to a plurality of objects of the same kind, unless otherwise indicated by the context, and do not limit the order or importance of said objects.

[0045] Expressions used herein such as “A, B, and C,” “A, B, or C,” “A, B, and / or C,” or “at least one of A, B, and C,” “at least one of A, B, or C,” “at least one of A, B, and / or C,” “at least one selected from A, B, and C,” “at least one selected from A, B, or C,” “at least one selected from A, B, and / or C,” etc., may mean each of the listed items or all possible combinations of the listed items. For example, “at least one selected from A and B” may refer to (1) A, (2) at least one of A, (3) B, (4) at least one of B, (5) at least one of A and at least one of B, (6) at least one of A and B, (7) at least one of B and A, and (8) all of A and B.

[0046] As used herein, the expression “based on” is used to describe one or more factors affecting an act or action of a decision or judgment described in the phrase or sentence containing such expression, and such expression does not exclude additional factors affecting said act or action of a decision or judgment.

[0047] As used in this specification, the expression that a component (e.g., a first component) is "connected" or "connected" to another component (e.g., a second component) may mean not only that a component is directly connected or connected to another component, but also that it is connected or connected through a new component (e.g., a third component).

[0048] As used herein, the expression "configured to" may have meanings such as "set to," "capable of," "modified to," "made to," or "capable of." Such expression is not limited to the meaning of "specifically designed in hardware," and, for example, a processor configured to perform a specific operation may mean a generic-purpose processor capable of performing that specific operation by executing software.

[0049] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which this disclosure pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0050] In this specification, the term "computer" refers to any type of hardware device comprising at least one processor, and may be understood to include software configurations operating on said hardware device according to the embodiments. For example, the term "computer" may be understood to include smartphones, tablet PCs, desktops, laptops, and user clients and applications running on each of these devices, but is not limited thereto.

[0051] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0052] Each step described in this specification is described as being performed by a computer, but the subject of each step is not limited thereto, and depending on the embodiment, at least some of each step may be performed on different devices.

[0054] FIG. 1 is a diagram illustrating an artificial intelligence-based insurance sales target company search result provision system according to one embodiment of the present disclosure.

[0055] Referring to FIG. 1, an artificial intelligence-based insurance sales target company search result providing system according to one embodiment of the present disclosure may include a computing device (100), a user terminal (200), an external server (300), and a network (400).

[0056] Here, the artificial intelligence-based insurance sales target company search result provision system illustrated in FIG. 1 is according to one embodiment, and its components are not limited to the embodiment illustrated in FIG. 1 and may be added, changed, or deleted as needed.

[0057] In one embodiment, the computing device (100) can perform a process of providing search results for companies targeted for insurance sales based on artificial intelligence.

[0058] In various embodiments, when the computing device (100) obtains a natural language query including search conditions regarding an insurance sales target company, it can generate a query in the form of a Structured Query Language (SQL) based on the obtained natural language query and provide search results for an insurance sales target company derived by querying an enterprise information database.

[0059] In various embodiments, the computing device (100) can provide search results for insurance sales target companies derived by generating a query in the form of a structured query based on a natural language query obtained through a pre-trained artificial intelligence model and querying a corporate information database.

[0060] Here, the pre-trained artificial intelligence model may include a Large Language Model (LLM) and an Embedding Model.

[0061] Large-scale language models are generally a type of artificial neural network specialized for natural language processing (NLP), possessing hundreds of millions to hundreds of billions of parameters or more, and may be models pre-trained based on vast amounts of text data.

[0062] Large-scale language models can be composed of multiple network functions, and network functions can generally be made up of multi-layered transformer blocks, each transformer block consisting of multiple computational units, and each unit may mainly include an attention mechanism and a position-wise feedforward network.

[0063] In transformer-based large-scale language models, input is processed as a sequence of tokens; input tokens are converted into embedding vectors, and after positional encoding is added, they can be input into transformer layers. Input and output embeddings are typically represented as high-dimensional computational units, and the representation of each token can be progressively updated by passing through multiple layers.

[0064] The Transformer layer may include a self-attention operation that enables the dynamic learning of interrelationships between each token in the input sequence. Self-attention calculates weights by considering the association of each token with all other tokens in the input sequence and can generate new representations based on this. In this case, the attention operation is performed in parallel in the form of multi-head attention, allowing the model to understand the input from various perspectives.

[0065] A large-scale language model according to various embodiments of the present disclosure may be configured to receive a natural language query as input, generate a query statement in the form of a structured query language (SQL), and verify the validity of the generated query statement. Specifically, the large-scale language model may learn terms specialized in the insurance sales domain, schema information of an enterprise information database, SQL writing rules, etc., to generate a query statement that accurately reflects the user's search intent, and may verify whether the generated query statement is appropriate in terms of executableability, field suitability, performance degradation factors, unnecessary conditions, etc.

[0066] An embedding model is an artificial neural network model that transforms text data into a fixed-dimensional vector space and can be trained so that semantically similar texts are placed in close positions within the vector space. Embedding models generally have an encoder structure and can tokenize input text and then convert it into a high-dimensional vector through a transformer layer, etc.

[0067] An embedding model according to various embodiments of the present disclosure may be pre-trained based on training data including insurance-related term information, industry classification information, and corporate information database information, and configured to output a result of converting input text data into a vector. During the training process of the embedding model, the semantic relationships between terms specialized in the insurance sales domain may be optimized to be reflected in a vector space, thereby enabling precise measurement of semantic similarity between natural language queries and database column information.

[0068] Embedding models can be utilized to perform semantic search; specifically, they convert natural language queries into vectors and database column information into vectors, then calculate similarity between vectors (e.g., cosine similarity) to extract columns highly relevant to the query or select similar reference queries. This enables users to construct appropriate search conditions using only natural language expressions, even if they do not have precise knowledge of the database structure.

[0069] Artificial intelligence models according to various embodiments of the present disclosure can be implemented as large-scale language models and embedding models based on a transformer structure, and thereby can provide the function of generating accurate structured query terms based on contextual understanding of input natural language queries and performing intelligent search based on semantic similarity.

[0070] In various embodiments, the computing device (100) may be connected to a user terminal (200) via a network (400) and may provide a user interface (UI) (e.g., FIGS. 10 to 12) from the user terminal (200). When a natural language query including search conditions regarding an insurance sales target company is obtained through the user interface (UI), a query in the form of a structured query is generated based on the obtained natural language query, and the search results regarding the insurance sales target company derived by querying the corporate information database may be provided through the user interface (UI).

[0071] Here, the user terminal (200) may refer to any form of entity(s) in a system having a mechanism for communicating with a computing device (100). For example, such a user terminal (200) may include a PC (personal computer), a notebook, a mobile terminal, a smartphone, a tablet PC, and a wearable device, and may include any type of terminal capable of connecting to a wired or wireless network. Additionally, the user terminal (200) may include any computing device implemented by at least one of an agent, an API (Application Programming Interface), and a plug-in. Additionally, the user terminal (200) may include an application source and / or a client application.

[0072] Additionally, the network (400) may refer to a connection structure capable of exchanging information between each node, such as multiple terminals and servers. For example, the network (400) may include a Local Area Network (LAN), a Wide Area Network (WAN), the World Wide Web (WWW), a wired / wireless data network, a telephone network, a wired / wireless television network, a Controller Area Network (CAN), and Ethernet.

[0073] Wireless data communication networks may include, but are not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth network, NFC (Near-Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc.

[0074] In one embodiment, an external server (300) may be connected to a computing device (100) via a network (400) and may store and manage various information and data (e.g., corporate information database, reference query, column information, basic prompt, verification prompt, etc.) necessary for the computing device (100) to perform the method of providing search results for companies targeted for insurance sales based on artificial intelligence, or may collect, store, and manage various information and data (e.g., search history, query generation history, validity verification results, etc.) derived as the computing device (100) performs the method of providing search results for companies targeted for insurance sales based on artificial intelligence. For example, the external server (300) may be a storage server separately provided outside the computing device (100), but is not limited thereto. Hereinafter, the hardware configuration of the computing device (100) will be described with reference to FIG. 2.

[0076] FIG. 2 is a diagram illustrating the hardware configuration of a computing device according to another embodiment of the present disclosure.

[0077] Referring to FIG. 2, a computing device (100) according to another embodiment of the present disclosure may include one or more processors (110), a memory (120) for loading a computer program (151) executed by the processor (110), a bus (130), a communication interface (140), and a storage (150) for storing the computer program (151). Here, FIG. 2 illustrates only the components related to the embodiments of the present disclosure. Accordingly, a person skilled in the art to which the present disclosure pertains will understand that other general-purpose components may be included in addition to the components illustrated in FIG. 2.

[0078] The processor (110) can control the overall operation of each component of the computing device (100). The processor (110) may be configured to include a CPU (Central Processing Unit), an MPU (Micro Processor Unit), an MCU (Micro Controller Unit), a GPU (Graphic Processing Unit), or any form of processor well known in the art of the present disclosure.

[0079] Additionally, the processor (110) may perform operations for at least one application or program for executing the method according to the embodiments of the present disclosure, and the computing device (100) may have one or more processors.

[0080] In various embodiments, the processor (110) may further include Random Access Memory (RAM) (not shown) and Read-Only Memory (ROM) (not shown) for temporarily and / or permanently storing signals (or data) processed within the processor (110). Additionally, the processor (110) may be implemented in the form of a System on Chip (SoC) comprising at least one of a graphics processing unit, RAM, and ROM.

[0081] Memory (120) stores various data, instructions and / or information. Memory (120) may load a computer program (151) from storage (150) to execute a method / operation according to various embodiments of the present disclosure. When a computer program (151) is loaded into memory (120), the processor (110) may perform the method / operation by executing one or more instructions constituting the computer program (151). Memory (120) may be implemented as a volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

[0082] The bus (130) provides communication functions between components of the computing device (100). The bus (130) can be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

[0083] The communication interface (140) supports wired and wireless internet communication of the computing device (100). Additionally, the communication interface (140) may support various communication methods other than internet communication. To this end, the communication interface (140) may be configured to include a communication module well known in the art of the present disclosure. In some embodiments, the communication interface (140) may be omitted.

[0084] Storage (150) can store computer programs (151) non-temporarily. When performing an AI-based insurance sales target company search result provision process through a computing device (100), storage (150) can store various information necessary to implement the AI-based insurance sales target company search result provision process.

[0085] The storage (150) may be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, a hard disk, a removable disk, or any form of computer-readable recording medium well known in the art to which the present disclosure belongs.

[0086] A computer program (151) may include one or more instructions that cause a processor (110) to perform a method / operation according to various embodiments of the present disclosure when loaded into memory (120). That is, the processor (110) may perform a method / operation according to various embodiments of the present disclosure by executing one or more instructions.

[0087] In one embodiment, a computer program (151) may include one or more instructions for performing an artificial intelligence-based method for providing search results for insurance sales target companies, comprising the steps of obtaining a natural language query including search conditions regarding insurance sales target companies, generating a query in the form of a structured query based on the obtained natural language query through a previously trained artificial intelligence model, and providing search results for insurance sales target companies derived by querying an enterprise information database using the generated query. Hereinafter, an artificial intelligence-based method for providing search results for insurance sales target companies performed by a computing device will be described with reference to FIGS. 3 to 9.

[0089] FIG. 3 is a flowchart of a method for providing search results for companies targeted for insurance sales based on artificial intelligence according to another embodiment of the present disclosure.

[0090] Referring to FIG. 3, in step S110, the computing device (100) can obtain a natural language query including search conditions regarding an insurance sales target company.

[0091] In various embodiments, the computing device (100) is connected to a user terminal (200) via a network (400) and can provide a user interface (UI) (e.g., FIG. 10 to FIG. 12) to the user terminal (200), and can obtain a natural language query including search conditions regarding an insurance sales target company through the user interface (UI).

[0092] Here, the term "insurance business target enterprise" may refer to a corporation or enterprise that is the target for conducting business activities for the purpose of selling insurance products.

[0093] In addition, here, a natural language query including search conditions may refer to various conditions for searching for insurance sales target companies described in everyday language expressions, and may include requirements regarding the characteristics, size, region, financial status, industry, etc. of the companies desired by the user.

[0094] For example, a natural language query containing search conditions may be a sentence that expresses specific search conditions in natural language form, such as "a manufacturing company located in Seoul with sales of 10 billion won or more," "an IT company located in Gyeonggi-do that has received venture certification," or "a company with 50 or more employees and a net profit of 500 million won or more."

[0095] Natural language queries can include not only simple forms of condition descriptions but also complex conditions. For example, complex conditions requiring logical operations (AND, OR) or arithmetic operations may be included, such as "companies located in Chungnam with advances or short-term loans of 200 million won or more" or "unlisted companies located in Seoul where the sum of capital reserves and retained earnings is 1.5 times or more of capital."

[0096] Furthermore, natural language queries may be expressions based on specific themes or sales concepts. For instance, they can be described in a form related to specific strategies or topics in insurance sales, such as "companies suitable for introducing a profit-sharing system" or "manufacturing companies requiring compliance with the Serious Accidents Compensation Act."

[0097] Here, natural language queries can be entered by the user typing directly into an input field, but depending on the case, they can be obtained in various ways, such as voice input, selection of a predefined theme, or re-search based on previous search history.

[0098] In step S120, the computing device (100) can generate a query in the form of a structured query through a pre-trained artificial intelligence model based on the natural language query obtained through step S110.

[0099] Typically, searching for companies targeted for insurance sales requires querying corporate information databases, and such database queries necessitate queries in the form of Structured Query Language (SQL). However, since it is technically difficult for general users to directly write SQL syntax, a process of converting natural language queries into SQL queries is required.

[0100] Here, the corporate information database refers to a database that systematically stores and manages various information necessary for corporate operations, such as basic corporate information, financial information, credit information, certification information, industry information, and management status. For example, it may include, but is not limited to, information such as company name, business registration number, address, industry classification, sales revenue, net profit, capital, number of employees, venture certification status, InnoBiz certification status, patent holding status, credit rating, and financial ratios.

[0101] Meanwhile, sales strategies and objectives vary among insurance consultants searching for target companies, and the content and form of natural language queries entered may differ depending on the sales methods or target customer bases pursued by each consultant. Accordingly, there is a need to accurately understand various forms of natural language queries and convert them into appropriate SQL queries.

[0102] For example, Consultant A has a sales style focused on new development sales, so he can input natural language queries that clearly specify the region, industry, and scale, such as "an IT service company located in Gangnam-gu, Seoul with sales of 5 billion won or more and less than 10 billion won."

[0103] In addition, Consultant B prefers a sales strategy targeting companies with excellent financial soundness, and can input natural language queries that include complex financial ratio calculations, such as "unlisted companies where the sum of capital reserves and retained earnings is 1.5 times or more of capital and retained earnings is 6.5 times or more of capital."

[0104] In addition, C consultants prefer to focus their sales efforts on innovative companies, so they can input natural language queries that include certification and intellectual property information, such as "a manufacturing company located in Gyeonggi-do that holds venture certification or InnoBiz certification and has a history of filing patents within the last 3 years."

[0105] That is, the method for providing search results for insurance sales target companies based on artificial intelligence according to various embodiments of the present disclosure takes into account that sales strategies and preferred target company characteristics differ among insurance consultants, and that the form and complexity of input natural language queries vary accordingly. Therefore, to accurately understand various types of natural language queries and convert them into appropriate SQL queries, schema information of an enterprise information database and various reference queries can be prepared and stored in advance, and appropriate database information and reference queries can be selected and utilized as needed. Below, a method for generating a query in the form of a structured query will be described in detail.

[0107] FIG. 4 is a flowchart of a method for generating a query in the form of a structured query in various embodiments, and FIG. 5 is a diagram illustrating the process of generating a query in the form of a structured query in various embodiments.

[0108] Referring to FIGS. 4 and FIGS. 5, in step S210, the computing device (100) can extract mapping column information corresponding to the acquired natural language query among a plurality of column information for an enterprise information database.

[0109] Here, multiple column information refers to information regarding each data field constituting the corporate information database, and may include, but is not limited to, column names and descriptive information for the corresponding columns. For example, multiple column information may include column names and detailed descriptions thereof, such as "Company Information_Company Name (Legal name of the company)", "Financial Information_23 Sales (2023 sales revenue, unit: million won)", "Company Information_Number of Employees_Latest Number of Workers (Number of workers most recently counted)", "Other_Venture Certification_Issuance Date (Date of receiving venture company certification)", etc.

[0110] In addition, mapping column information refers to column information that is semantically related to the acquired natural language query, and may include, but is not limited to, a set of database columns related to the conditions the user wishes to search for. For example, if a user enters a natural language query such as "manufacturing companies in Seoul with sales of 5 billion won or more," mapping column information may be extracted such as "company_information_region1(city / province information)", "financial_information_23-year sales", "company_information_industry sub-classification", etc.

[0111] Specifically, the computing device (100) can convert a natural language query obtained using an embedding model into a query vector.

[0112] Here, the embedding model is a pre-trained artificial intelligence model configured to output a result of converting input text data into a vector, which is pre-trained based on training data including insurance-related terminology, industry classification information, and corporate information database information. It may have a transformer-based encoder structure and may be implemented based on model architectures such as BERT, Sentence-BERT, and BGE (BAAI General Embedding), but is not limited thereto.

[0113] For example, when a computing device (100) inputs a natural language query, such as “a venture-certified company in Gyeonggi-do with sales of 3 billion won or more,” into an embedding model, the text undergoes a tokenization process and is converted into a sequence of tokens. It can then pass through an embedding layer and a transformer encoder to be converted into a dense vector of a fixed dimension (e.g., 768 dimensions or 1024 dimensions). The converted query vector is expressed in the form of a numeric array such as [0.23, -0.45, 0.67, ..., 0.12], which numerically encodes the semantic characteristics of the natural language query.

[0114] Next, the computing device (100) can obtain a plurality of column vectors that have been converted for each of the plurality of column information.

[0115] Here, multiple column vectors refer to the result of converting each column information of the enterprise information database into a vector space through an embedding model, and may include, but is not limited to, vectorized text combining column names and column description information.

[0116] For example, a computing device (100) can perform vector transformations on all column information of an enterprise information database in advance using an embedding model and store them in a storage such as a vector database (e.g., a vector store) or an index structure (e.g., FAISS, Chroma, etc.).

[0117] At this time, the computing device (100) can retrieve multiple column vectors from the storage. For example, column information such as “Financial Information_23 Year Sales: 2023 Year Sales, Unit is million won” is converted and stored as a vector such as [0.34, -0.21, 0.55, ..., 0.08], and the computing device (100) can retrieve this by loading it from memory or an external server.

[0118] Next, the computing device (100) can calculate the similarity between the converted query vector and each of the acquired multiple column vectors.

[0119] For example, a computing device (100) can calculate the similarity between a query vector and each column vector using a cosine similarity method. Here, cosine similarity is a method of measuring the angle between two vectors, having a value between -1 and 1, and the closer the value is to 1, the more similar the two vectors are.

[0120] As a specific example, let A be the query vector and B be the column vector. The cosine similarity can be calculated as (A·B) / (||A|| × ||B||). For instance, the similarity between the query vector "sales of 5 billion or more" and the column vector "financial_information_23year_sales" is calculated as 0.92, and the similarity with the column vector "company_information_number_of_employees" is calculated as 0.31, indicating that the query content has a high correlation with the sales column.

[0121] Finally, the computing device (100) can extract column information corresponding to a column vector whose calculated similarity among a plurality of column vectors is greater than or equal to a reference value as mapping column information.

[0122] For example, the computing device (100) can set a threshold value to 0.7 and extract only columns with a calculated similarity of 0.7 or higher as mapping column information. Specifically, for a natural language query "manufacturing company with sales of 5 billion won or more located in Seoul," the similarity of the "Company Information_Region 1" column is calculated to be 0.88, the similarity of the "Financial Information_23 Year Sales" column is 0.92, and the similarity of the "Company Information_Industry Detail Classification" column is 0.85. Since all of these exceed the threshold value of 0.7, the information of these three columns can be extracted as mapping column information.

[0123] In various embodiments, when multiple search conditions included in a natural language query belong to different semantic categories, the computing device (100) can extract mapping column information by dividing the natural language query into semantic units so that mapping column information corresponding to each condition is not omitted.

[0124] Specifically, the computing device (100) can divide the acquired natural language query into multiple search condition units using a large-scale language model. For example, the natural language query "manufacturer in Seoul with sales of 5 billion or more" can be divided into three semantic units: "in Seoul" (regional condition), "sales of 5 billion or more" (financial condition), and "manufacturer" (industry condition).

[0125] Next, the computing device (100) can convert each of the divided multiple search condition units into a query vector. For example, "in Seoul" can be converted into a first query vector, "sales of 5 billion or more" into a second query vector, and "manufacturer" into a third query vector.

[0126] Next, the computing device (100) can calculate the similarity between each converted query vector and a plurality of column vectors, and extract column information having the highest similarity for each converted query vector. For example, for the first query vector ("in Seoul"), the "Company Information_Region 1" column can be extracted with the highest similarity (0.88), for the second query vector ("sales of 5 billion or more"), the "Financial Information_23 Year Sales" column can be extracted with the highest similarity (0.92), and for the third query vector ("Manufacturer"), the "Company Information_Industry Detail Classification" column can be extracted with the highest similarity (0.85).

[0127] Finally, the computing device (100) can integrate each of the column information extracted for each converted query vector to form a final set of mapping column information. Through this, all search conditions of different categories included in the natural language query are reflected, and it is possible to prevent columns corresponding to specific conditions from being omitted.

[0128] In various embodiments, the computing device (100) can remove duplicates among the extracted mapping column information and filter so that columns representing semantically similar or identical information are not included as duplicates.

[0129] Specifically, the computing device (100) can calculate the mutual similarity between column vectors corresponding to each of the extracted multiple mapping column information. For example, the cosine similarity between the "financial_information_sales" column vector and the "financial_information_23year_sales" column vector can be calculated to quantitatively measure how semantically similar the two columns are.

[0130] Next, the computing device (100) can identify pairs of columns whose calculated mutual similarity is greater than or equal to a preset duplicate judgment threshold (e.g., 0.9) as duplicate candidates. For example, if the similarity between "Financial Information_Sales Amount" and "Financial Information_23 Year Sales" is calculated to be 0.95, these two columns can be identified as duplicate candidates.

[0131] Next, the computing device (100) may determine a priority for column pairs identified as duplicate candidates and select only one column as the final mapping column information. The priority determination may be performed based on at least one of the following criteria.

[0132] First, the computing device (100) can prioritize selecting a column with a higher similarity to a natural language query. For example, for a query "sales amount of 5 billion or more," if the query similarity of the "financial_information_sales amount" column is 0.87 and the query similarity of the "financial_information_23-year sales" column is 0.92, the "financial_information_23-year sales" column with the higher similarity can be selected.

[0133] Additionally, the computing device (100) may preferentially select a column containing more specific and clear information. For example, "23-year sales" with the year specified is more specific than "sales amount," so "financial_information_23-year sales" may be selected. This can be determined by analyzing the number of tokens included in the column name, whether numbers are included, whether time information is included, etc.

[0134] Alternatively, the computing device (100) may preferentially select a column representing more recent data. For example, if both "Financial Information_22 Year Sales" and "Financial Information_23 Year Sales" are extracted, the more recent year, "Financial Information_23 Year Sales," may be selected unless otherwise specified.

[0135] Furthermore, the computing device (100) can preferentially select columns that have a higher usage frequency within the database. For example, columns that are used more frequently in past search history or reference queries can be determined to have a higher priority.

[0136] Additionally, the computing device (100) can determine whether there is duplication and priority using a large-scale language model. Specifically, information on columns identified as duplicate candidates is input into the large-scale language model, and a prompt such as “Select the column that is more suitable for the condition of ‘sales of 5 billion or more’ among the following two columns: (1) Financial_Information_Sales_Amount, (2) Financial_Information_23Year_Sales” is provided, and a final selection can be made based on the judgment result of the model.

[0137] Through this, the computing device (100) can remove unselected duplicate columns from the mapping column information list and generate a final set of mapping column information consisting only of selected columns. For example, "financial_information_sales", "financial_information_23-year sales", and "financial_information_22-year sales" may all be extracted, but after a filtering process, only "financial_information_23-year sales" may remain as the final mapping column information.

[0138] Accordingly, this prevents unnecessary duplicate columns from being included in the query generation prompt and supports large-scale language models in generating queries in the correct structured query format using the most suitable columns without confusion.

[0139] In step S220, the computing device (100) can select at least one reference query from a plurality of reference queries prepared in advance based on the mapping column information extracted through step S210.

[0140] Here, multiple reference queries may refer to a database of example SQL queries constructed by collecting SQL queries that have been successfully executed in the past, and each reference query can serve as a template demonstrating the correct way to write SQL for specific search patterns or combinations of conditions. For example, reference queries may include examples of various search patterns, such as "SQL searching by combining specific regions and industries," "SQL including financial ratio calculations," and "SQL using certification information as a condition."

[0141] Specifically, the computing device (100) can obtain a previously converted mapping column vector for the extracted mapping column information.

[0142] For example, the computing device (100) can obtain previously transformed column vectors corresponding to the mapping column information (e.g., "Company Information_Region 1", "Financial Information_23 Year Sales", "Company Information_Industry Detail Classification") extracted through step S210. Specifically, the vectors of each mapping column used for similarity comparison in step S210 can be obtained by querying them again from a vector database (e.g., a vector store) or a repository.

[0143] Next, the computing device (100) can generate a single mapping column vector by integrating multiple acquired mapping column vectors. For example, a mapping column vector [0.41, -0.33, 0.72, ..., 0.19] can be generated by calculating the average for the "Company Information_Region 1" vector [0.45, -0.32, 0.68, ..., 0.21], the "Financial Information_23 Year Sales" vector [0.38, -0.29, 0.71, ..., 0.15], and the "Company Information_Industry Sub-classification" vector [0.40, -0.38, 0.77, ..., 0.21]. Alternatively, a mapping column vector may be generated by concatenating each column vector or by applying a weighted average. However, it is not limited to this, and the multiple acquired mapping column vectors may be generated individually without integrating them.

[0144] Next, the computing device (100) can obtain a plurality of reference query vectors that have been converted for each of the plurality of reference queries prepared in advance.

[0145] For example, a computing device (100) can retrieve a vector that has been converted in advance for each reference query from a pre-built vector database (e.g., a vector store), a storage, or a reference query database. In this case, the reference query vector may be a vector generated by reflecting the column information used in the reference query, the pattern of the WHERE condition, and the usage method of logical operators. For example, a reference query such as "SELECT company_information_company_name, financial_information_23year_sales FROM company_data WHERE company_information_region1 = 'Seoul' AND financial_information_23year_sales >= 5000" is converted into and stored as a vector such as [0.38, -0.29, 0.68, ..., 0.15], and the computing device (100) can retrieve it.

[0146] Next, the computing device (100) can calculate the similarity between the acquired mapping column vector and each of the acquired multiple reference query vectors.

[0147] For example, a computing device (100) can calculate the similarity between a mapping column vector and each reference query vector using a cosine similarity method. Specifically, if the mapping column vector represents a combination of "region, sales, industry," it will show high similarity with reference queries using similar column combinations. For example, the similarity with reference query A using the combination of "region + sales + industry" can be calculated as 0.89, and the similarity with reference query B using the combination of "region + number of employees" can be calculated as 0.56.

[0148] Finally, the computing device (100) can select a preset number of reference query vectors in order from the reference query vectors with high similarity among the acquired multiple reference query vectors, and select a reference query corresponding to the selected reference query vector.

[0149] For example, the computing device (100) can set the preset number to 3 and select the top 3 reference queries with the highest similarity. Specifically, the reference query with the 1st highest similarity ranking (e.g., similarity 0.92), the reference query with the 2nd highest similarity ranking (e.g., similarity 0.87), and the reference query with the 3rd highest similarity ranking (e.g., similarity 0.84) can be selected, and the selected reference queries can be used as reference examples when generating preliminary queries.

[0150] However, the method of selecting reference queries is not limited to this, and in various embodiments, the computing device (100) may select all reference queries whose calculated similarity is greater than or equal to a specific threshold (e.g., 0.8), or dynamically adjust the number of selections according to the number or complexity of the mapping columns.

[0151] For example, when the number of extracted mapping columns is small (e.g., 1-2), the computing device (100) can select only 2-3 reference queries by determining them as relatively simple search conditions. Specifically, in the case of a natural language query that includes only a region condition, such as "company located in Seoul," only one mapping column, "company_information_region1," is extracted, so the prompt length can be optimized by selecting only the top 2 reference queries with high similarity.

[0152] On the other hand, when the number of extracted mapping columns is large (e.g., 5 or more), the computing device (100) can select 5 to 7 reference queries by determining them as complex search conditions. Specifically, in the case of a natural language query containing complex conditions such as "an IT service company located in Gangnam-gu, Seoul, with sales of 5 billion or more and less than 10 billion, with 100 or more employees and holding a venture certification," 6 mapping columns are extracted, such as "Company Information_Region 1," "Company Information_Region 2," "Financial Information_23 Year Sales," "Company Information_Number of Employees," "Other_Venture Certification," and "Company Information_Industry Detail Classification," so the top 6 to 7 reference queries can be selected to refer to various condition combination patterns.

[0153] Additionally, the computing device (100) can analyze the complexity of conditions included in the natural language query and adjust the number of selected reference query statements. For example, if the natural language query includes logical operators (AND, OR), requires arithmetic operations (sum, ratio calculation, etc.), or includes a combination of range conditions (greater than or equal to, less than or equal to, greater than, less than), the complexity score can be calculated as high, and the number of selected reference query statements can be increased in proportion to this.

[0154] Specifically, since a query such as “a company where the sum of capital reserves and retained earnings is 1.5 times or more of capital and advances or short-term loans are 200 million or more” includes both arithmetic operations (sum, ratio calculation) and logical operations (OR) and has a high complexity score, the computing device (100) can select up to 7 to 10 reference queries that handle such complex patterns, and support a large-scale language model in generating accurate SQL queries by referring to sufficient examples.

[0155] In step S230, the computing device (100) can generate a preliminary query by inputting the natural language query obtained through step S110, the mapping column information extracted through step S210, the reference query selected through step S220, and the basic prompt prepared in advance into a large language model as input data.

[0156] Here, a large-scale language model is a pre-trained artificial intelligence model configured to generate queries in the form of structured query language and validate the generated queries. It may be implemented based on large-scale language model architectures such as the GPT series, Claude, and LLaMA, and may include, but is not limited to, versions specialized for code generation and SQL writing (e.g., Codex, Code-LLaMA).

[0157] In addition, pre-prepared default prompts refer to commands for a large-scale language model to generate a query in the form of a structured query language, and may include, but are not limited to, work instructions, descriptions of the enterprise information database, and query writing rules.

[0158] Here, work instructions refer to content defining specific tasks that a large-scale language model must perform, and may include role definitions and goal settings such as, "You are an expert in converting natural language queries into SQL queries. Convert the given natural language query into an accurate SQL SELECT statement for the enterprise information database." Description of the enterprise information database refers to content explaining the database structure, table names, the meaning of major columns, and data types, and may include schema information such as, "The database name is company_data, and major columns include Company_Info_CompanyName(TEXT), Financial_Info_23YearSales(INTEGER, Unit: Million KRW), Company_Info_Region1(TEXT), etc." Query writing rules refer to grammatical and formal rules that must be observed when writing SQL, and may include, but are not limited to, detailed writing guidelines such as, "Do not enclose column names in backticks (`) and use them as they are," "Use the CAST function to explicitly convert types when comparing numbers," and "Sort results in order of relevance."

[0159] Specifically, the computing device (100) can configure a query generation prompt by combining a natural language query obtained from a pre-prepared basic prompt, extracted mapping column information, and a selected reference query.

[0160] For example, the computing device (100) can configure a query generation prompt in the following format:

[0161] [Query Generation Prompt]

[0162] “1) Work Instructions

[0163] - You are an expert in converting natural language queries into SQL. Generate an accurate SQL query based on the given information.

[0164] 2) Database Schema

[0165] - Table name: company_data

[0166] - Related Columns: Company Information_Region 1 (Regional Information), Financial Information_23 Sales (Sales Amount, Unit: Million KRW), Company Information_Detailed Industry Classification (Industry Classification)

[0167] 3) Reference Example SQL

[0168] - SELECT company_info_company_name, financial_info_23year_sales FROM company_data WHERE company_info_region1 = 'Seoul' AND CAST(financial_info_23year_sales AS INTEGER) >= 5000;

[0169] - SELECT company_info_company_name FROM company_data WHERE company_info_industry_detailed_classification LIKE '%manufacturing%';

[0170] 4) User query

[0171] - Manufacturing companies based in Seoul with annual sales of over 5 billion KRW

[0172] Next, the computing device (100) can input a configured query generation prompt into a large language model to generate a preliminary query in the form of a structured query.

[0173] For example, when the computing device (100) inputs a query generation prompt into a large language model, the model can generate the following preliminary query:

[0174] [Generated Preliminary Question]

[0175] “SELECT Company Info_Company Name, Company Info_Industry Classification, Financial Info_23 Sales, Company Info_Business Registration Number, Company Info_Region 1

[0176] FROM company_data

[0177] WHERE business_information_region1 = 'Seoul'

[0178] AND Company_Info_Industry_Detailed_Classification LIKE '%Manufacturing%'

[0179] AND Financial_Info_23Year_Sales IS NOT NULL

[0180] AND CAST(FinancialInformation_23YearSales AS INTEGER) >= 5000

[0181] ORDER BY CAST(재무정보_23년매출 AS INTEGER) DESC;”

[0182] In step S240, the computing device (100) can perform validation on the preliminary query generated through step S230 and generate the query that passes the validation as the final query.

[0183] Here, validation refers to the process of comprehensively checking whether a generated preliminary query can be executed normally in an actual database, accurately reflects the user's search intent, and does not adversely affect database performance. This may include, but is not limited to, syntax validation, semantic validation, performance validation, and accuracy validation. Below, we will explain in detail how to generate a final query from a query that has passed validation.

[0185] FIG. 6 is a flowchart of a method for generating a final query from a query that has passed validation in various embodiments.

[0186] Referring to FIG. 6, in step S310, the computing device (100) can output a validation result for the generated preliminary query by inputting the preliminary query generated through step S230 and the pre-prepared validation prompt as input data into a large language model.

[0187] Here, a pre-prepared validation prompt is a command for a large-scale language model to perform validation of an input query, and may include, but is not limited to, work instructions and validation criteria.

[0188] In this context, work instructions refer to the definition of specific roles and objectives that a large-scale language model must perform during the process of validating a preliminary query. For example, they may include role definitions and explanations of validation objectives such as, "You are an SQL query validation expert. Systematically check whether the provided SQL query is written correctly according to the following validation criteria and determine whether it passes for each criterion." Validation criteria refer to specific requirements and evaluation items that the preliminary query must satisfy. They may include, but are not limited to, detailed evaluation criteria such as grammatical accuracy, semantic consistency, performance efficiency, and the absence of unnecessary elements. Below, the method for outputting validation results will be explained in detail.

[0190] FIG. 7 is a flowchart of a method for outputting validation results in various embodiments.

[0191] Referring to FIG. 7, in step S410, the computing device (100) can perform a first verification step to derive a first verification result by verifying whether the generated preliminary query is an executable query in the enterprise information database using a large-scale language model.

[0192] Here, an executable query means a query written in accordance with SQL syntax that can be executed without error in a database system, and may be, but is not limited to, a query that satisfies the correct SELECT statement structure, use of existing table and column names, appropriate WHERE clause syntax, correct data type conversion, and correct use of parentheses and operators.

[0193] For example, the computing device (100) can input a generated preliminary query into a large language model along with a verification prompt. Specifically, a prompt such as “Verify whether the following SQL query is executable against the company_data table. Check the table name, column name, syntax error, and syntax error, and determine whether it passes: [Preliminary query]” can be configured.

[0194] At this stage, the large-scale language model analyzes the preliminary query to check whether non-existent column names are used (e.g., "Company_Info_Company_Name" should be used instead of "Company_Info_Company_Name"), whether the SELECT and FROM clauses are correctly constructed, whether the conditional expressions in the WHERE clause are syntactically correct, and whether the CAST function or LIKE operator is used correctly. For instance, if the SELECT statement is written with a typo, such as "SELEC Company_Info_Company_Name FROM company_data," the result of the first validation will be 'Failure (Non-pass),' and "SELECT keyword typo" may be returned along with the error reason.

[0195] In step S420, the computing device (100) may perform a second verification step of deriving a second verification result by verifying whether the search field included in the generated preliminary query corresponds to a search condition regarding an insurance sales target company using a large-scale language model.

[0196] Here, the search field refers to a column that constitutes the search condition of an SQL query and may include a database column actually used as a filtering condition in a preliminary query; the search condition regarding insurance sales target companies refers to specific requirements for company search requested by the user through a natural language query and may include, but is not limited to, conditions regarding company characteristics desired by the user, such as region, industry, sales revenue, number of employees, and possession of certifications.

[0197] For example, the computing device (100) can input the acquired natural language query and the generated preliminary query together into a large language model to verify whether the user's search intent is correctly reflected in the SQL query. Specifically, it can configure a prompt such as, "User query: 'Manufacturer with sales of 5 billion or more located in Seoul', generated SQL: [preliminary query]. Check if this SQL accurately reflects all of the user's search conditions. Determine if there are any missing conditions or conditions that are misinterpreted."

[0198] At this time, the large-scale language model can check whether 'located in Seoul' in the natural language query is reflected in the WHERE clause as "business_information_region1 = 'Seoul'", whether 'sales revenue of 5 billion or more' is correctly converted to "CAST(financial_information_23year_sales AS INTEGER) >= 5000" (since the unit is million won, 5 billion is 5000), and whether 'manufacturer' is reflected as "business_industry_detailed_classification LIKE '%manufacturing%'". For example, if a user requests "manufacturer" but the industry condition is omitted in the preliminary query, the result of the second verification is 'failure (non-pass)', and the reason "industry condition (manufacturing) omitted" may be returned along with it.

[0199] In step S430, the computing device (100) may perform a third verification step to derive a third verification result by verifying whether a preliminary query generated using a large-scale language model includes performance degradation factors of the enterprise information database.

[0200] Here, performance degradation factors for enterprise information databases refer to inefficient query patterns that significantly slow down the database response speed or excessively consume system resources during the execution of SQL queries. These factors may include, but are not limited to, LIKE '%string%' searches on columns without indexes (using both wildcards), conditions that cause full table scans of large amounts of data, unnecessary nesting of subqueries, excessive JOIN operations, inefficient listing of OR conditions, and syntax that hinders index utilization by directly applying functions to columns in the WHERE clause.

[0201] For example, the computing device (100) inputs a preliminary query into a large-scale language model and can configure a prompt such as, “Check if there are elements in the following SQL query that may degrade database performance. In particular, check if there are patterns that do not utilize indexes, conditions that cause a full table scan, or excessive operations: [Preliminary Query]”. The large-scale language model can determine that performance degradation may occur if a function is applied directly to a column in the preliminary query, such as “WHERE SUBSTRING(Company_Information_CompanyName, 1, 2) = 'Samsung'”, because an index cannot be utilized, or if a leading wildcard is used, such as “WHERE Company_Information_CompanyName LIKE '%Company%'”, because a full table scan is required, and can identify these as performance degradation elements.

[0202] Here, a leading wildcard can mean that % or _ is placed at the beginning of the pattern string of the LIKE condition, and such patterns can restrict index-based search and cause a full table scan.

[0203] For example, if there are more than 300,000 corporate records stored in the corporate information database, a preliminary query containing both wildcard LIKE searches is likely to cause a full table scan, so the third validation result is 'failure (non-pass),' and the reason "possibility of full table scan due to preceding wildcard LIKE use" may be returned.

[0204] In step S440, the computing device (100) may perform a fourth verification step to derive a fourth verification result by verifying whether a preliminary query generated using a large-scale language model includes unnecessary conditions or unnecessary search fields.

[0205] Here, unnecessary conditions refer to filtering conditions that are not included in the user's natural language query but are additionally included in the preliminary query, and may include constraints not requested by the user or duplicate conditional expressions; unnecessary search fields refer to columns included in the SELECT clause that the user did not want to query, and may include cases where querying an excessive number of columns impairs result readability or causes unnecessary data transmission, but are not limited thereto.

[0206] For example, the computing device (100) can compare the acquired natural language query with the preliminary query and configure a prompt such as "User query: 'Manufacturer located in Seoul', generated SQL: [Preliminary query]. Check if this SQL contains unnecessary conditions or excessive search fields that the user did not request."

[0207] In this case, the large-scale language model may determine that a condition such as "AND Financial_23YearSales >= 1000" is unnecessary if the user has not specified a sales revenue condition, or identify it as an unnecessary search field if the SELECT clause includes an excessive number of columns that the user does not need (e.g., all columns of 20 or more). For example, if the user requests only "Venture Certified Company" but a condition such as "AND Company_Information_NumberOfEmployees >= 10" is arbitrarily added to the preliminary query, the result of the fourth validation will be 'Failure (Non-pass),' and the reason "Includes condition for the number of employees not requested by the user" may be returned. However, conditions for ensuring basic data quality (e.g., "WHERE column_name IS NOT NULL") may not be considered unnecessary conditions.

[0208] In step S450, the computing device (100) can output a validity verification result as passed if the first verification result obtained through step S410, the second verification result obtained through step S420, the third verification result obtained through step S430, and the fourth verification result obtained through step S440 are all passed.

[0209] For example, the computing device (100) can determine the final validation result by comprehensively evaluating the results of each validation step. Specifically, if the first validation result is 'passed' (grammatically executable), the second validation result is 'passed' (accurately reflects user conditions), the third validation result is 'passed' (no performance degradation factors), and the fourth validation result is 'passed' (no unnecessary conditions / fields), the computing device (100) can output the validation result as 'passed' and confirm the corresponding preliminary query as the final query.

[0210] On the other hand, if any of the four verification results is 'failure (non-pass),' the computing device (100) outputs the validity verification result as 'failure (non-pass)' and can record which verification stage failed and the specific reason for the failure.

[0211] For example, if the first and second verifications are passed but a performance degradation factor is found in the third verification, detailed information such as "Third verification failed (non-pass): Concerns about performance degradation due to prior use of wildcard LIKE" may be displayed, and the query can be improved based on the reason during the subsequent correction phase.

[0213] Referring again to FIG. 6, at step S320, the computing device (100) can confirm the query that passed the validation result output through step S310 as the final query.

[0214] Specifically, if the computing device (100) outputs the first verification result, the second verification result, the third verification result, and the fourth verification result as 'passed' in step S310, it determines that the preliminary query satisfies all validity criteria and can confirm it as the final query.

[0215] For example, it can be assumed that the preliminary query generated for the natural language query "manufacturing companies located in Seoul with sales of 5 billion or more" is as follows:

[0216] [Generated Preliminary Question]

[0217] “SELECT Company Info_Company Name, Company Info_Industry Classification, Financial Info_23 Sales, Company Info_Business Registration Number, Company Info_Region 1

[0218] FROM company_data

[0219] WHERE business_information_region1 = 'Seoul'

[0220] AND Company_Info_Industry_Detailed_Classification LIKE '%Manufacturing%'

[0221] AND Financial_Info_23Year_Sales IS NOT NULL

[0222] AND CAST(FinancialInformation_23YearSales AS INTEGER) >= 5000

[0223] ORDER BY CAST(재무정보_23년매출 AS INTEGER) DESC;"

[0224] If the above preliminary query is determined to be grammatically executable in the first verification, confirmed to accurately reflect all user search conditions in the second verification, evaluated to have no performance degradation factors in the third verification, and determined not to include unnecessary conditions or fields in the fourth verification, the computing device (100) can confirm this query as the final query.

[0225] Next, the computing device (100) can switch the query that has been finalized as the final query to a database execution waiting state. For example, the final query becomes ready to be actually transmitted to the corporate information database and executed, and the computing device (100) can use the final query that is ready to be executed to query the corporate information database.

[0226] In various embodiments, the computing device (100) may store the query confirmed as the final query in a log or history database. For example, the user's natural language query, the generated final query, the time of verification passing, user identification information, etc., may be stored together and used as a reference query for similar search requests in the future, or as data for system performance analysis and improvement.

[0227] In various embodiments, the computing device (100) may transmit the fact that the final query has been confirmed to the user terminal (200) as a notification. For example, a message such as "Search conditions have been successfully processed. Database is being queried" may be displayed on the user interface to enable the user to understand the progress of the search.

[0228] On the other hand, if the validation result output through step S310 is 'failure (non-pass),' that is, if at least one of the derived first validation result, derived second validation result, derived third validation result, and derived fourth validation result is not passed, the computing device (100) may not finalize the preliminary query as the final query, perform a subsequent correction process, or request the user to re-enter a natural language query. This will be explained in detail below.

[0230] FIG. 8 is a flowchart of a method for generating a final query when the validation result fails in various embodiments.

[0231] Referring to FIG. 8, in step S510, the computing device (100) can generate correction data for a preliminary query generated to correspond to a verification step in which a verification result other than a pass is derived using a large-scale language model.

[0232] Here, correction data may refer to specific information for correcting a preliminary query that failed validation, and may include, but is not limited to, instructions on which parts to correct and how, the location to be corrected, the content of the correction, and the method of correction.

[0233] Specifically, the computing device (100) can use a large-scale language model to identify at least one part requiring correction that indicates a target requiring correction in a preliminary query generated based on the type of verification step from which a non-pass verification result is derived.

[0234] Here, the parts requiring correction refer to specific syntax, conditional expressions, column names, or operators within the preliminary query that contain errors or require improvement. These may include, but are not limited to, specific WHERE conditions in the SQL query, specific columns in the SELECT clause, the wildcard position of the LIKE operator, or whether the CAST function is applied.

[0235] For example, if the computing device (100) fails (fails) in the first verification, it can provide a large language model with a prompt such as “Identify syntax errors or non-executable parts in the following SQL query: [Preliminary query]” to identify parts that need correction, such as a typo of “SELEC” or a non-existent column name “Company_Info_Company_Name”.

[0236] Alternatively, if the third verification fails (fails), the preceding wildcard part of “WHERE company_information_company_name LIKE '%Inc.co.uk%'” can be identified as a part requiring correction through the prompt “Identify the part causing performance degradation in the next SQL”. As a specific example, if it is determined in the second verification that the condition “sales revenue of 5 billion or more” is missing, the computing device (100) can identify the entire WHERE clause as a part requiring correction and record “financial_information_23-year sales condition missing” as the reason for correction.

[0237] Next, the computing device (100) can identify one of the correction types among deletion correction, addition correction, and change correction for the identified correction-needed parts.

[0238] For example, the computing device (100) can utilize a large-scale language model to determine an appropriate correction type for each part requiring correction. Specifically, if it is determined in the fourth verification that the condition “AND company_information_number_of_employees >= 10” which was not requested by the user is included, the computing device (100) can identify a ‘deletion correction’ type for the condition.

[0239] In addition, if it is determined in the second verification that the condition "sales of 5 billion or more" is missing, the computing device (100) can identify an 'additional correction' type for the WHERE clause. In addition, if a typo "SELEC" is found in the first verification, the computing device (100) can identify a 'change correction' type for the corresponding keyword. In addition, if "LIKE '%Company%'" is determined in the third verification to be a performance degradation factor, the computing device (100) can identify a 'change correction' type for the corresponding condition expression and decide to modify it to "LIKE 'Company%'" or replace it in another way.

[0240] Next, the computing device (100) can generate a correction element corresponding to any one of the identified correction types.

[0241] Here, the correction element refers to the specific modifications actually applied according to the identified correction type, and may include, but is not limited to, location information of the phrase to be deleted, SQL code text to be added, text pairs before and after the change, etc.

[0242] For example, the computing device (100) can generate information on the start and end positions of the phrase to be deleted as correction elements for the 'deletion correction' type. As a specific example, if "AND company_information_number_of_employees >= 10" needs to be deleted, the exact position of the corresponding condition expression (e.g., the third condition of the WHERE clause) and the range of text to be deleted can be generated as correction elements.

[0243] Additionally, the computing device (100) can generate SQL code text to be added and an insertion location as correction elements for the ‘additional correction’ type. For example, if the sales condition is missing, it can generate SQL code such as “AND Financial_23YearSales IS NOT NULL AND CAST(Financial_23YearSales AS INTEGER) >= 5000” and information that this should be inserted at a specific location in the WHERE clause as correction elements.

[0244] Additionally, the computing device (100) can generate a pair of text before change and text after change as a correction element for the 'change correction' type. For example, if "SELEC" needs to be changed to "SELECT", information such as {before change: "SELEC", after change: "SELECT"} can be generated as a correction element. Or, if "LIKE '%Company%'" needs to be changed to "LIKE 'Company%'", information such as {before change: "LIKE '%Company%'", after change: "LIKE 'Company%'"} can be generated as a correction element.

[0245] Finally, the computing device (100) can apply the generated correction element to the identified correction-needed parts to generate correction data.

[0246] For example, the computing device (100) can generate correction data by combining all generated correction elements in a structured form. Specifically, if one deletion correction, two addition corrections, and one modification correction are required for a preliminary query, the computing device (100) can generate correction data in the following form:

[0247] [Generated Correction Data]

[0248] "Correction1": {"Type": "Change", "Location": "SELECT keyword", "Before change": "SELEC", "After change": "SELECT"},

[0249] "Correction2": {"Type": "Add", "Location": "WHERE Clause", "Additional Content": "AND FinancialInformation_23YearSales IS NOT NULL AND CAST(FinancialInformation_23YearSales AS INTEGER) >= 5000"},

[0250] "Correction3": {"Type": "Change", "Location": "WHERE clause third condition", "Before change": "LIKE '%Inc.%'", "After change": "LIKE 'Inc.%'"},

[0251] "Correction4": {"Type": "Deletion", "Location": "WHERE clause fourth condition", "Delete content": "AND business_info_number_of_employees >= 10"}

[0252] In step S520, the computing device (100) can calculate the correction difficulty for the correction data generated through step S510.

[0253] Here, correction difficulty refers to an indicator that quantifies the complexity or difficulty of the task of modifying preliminary queries by applying generated correction data, and may be, but is not limited to, a score calculated by comprehensively considering the number of correction elements, the complexity of correction types, and the importance of failed verification steps.

[0254] Specifically, the computing device (100) can assign a score to each of the generated correction elements included in the generated correction data and determine the correction difficulty based on the assigned score.

[0255] More specifically, the computing device (100) can assign a preset base score to each of the generated multiple correction elements included in the generated correction data, corresponding to any one of the correction types of deletion, addition, and change.

[0256] For example, the computing device (100) may assign a base score of 1 point for 'deletion correction', a base score of 2 points for 'change correction', and a base score of 3 points for 'addition correction'. This is because deletion is generally the simplest, change is intermediate, and addition is the most complex task. As a specific example, a change correction such as correcting a simple typo may be assigned 2 points, deletion of unnecessary conditions may be assigned 1 point, and addition of missing complex conditions may be assigned 3 points as base scores.

[0257] Next, the computing device (100) can calculate a weighted score for each of the multiple correction elements generated by applying different weights to the base score assigned according to the type of verification step that was not passed.

[0258] For example, the computing device (100) can set the weight to 1.5 times if it fails the first verification (executability), the weight to 2.0 times if it fails the second verification (field suitability), the weight to 1.8 times if it fails the third verification (performance degradation), and the weight to 1.2 times if it fails the fourth verification (unnecessary elements).

[0259] This is based on the judgment that accurately reflecting the user's search intent (Verification 2) is the most important, performance optimization (Verification 3) and basic feasibility (Verification 1) are next most important, and the removal of unnecessary elements (Verification 4) is relatively less important. For example, in the case of additional correction due to a failure in Verification 2, a weighted score of 6 can be calculated by multiplying the base score of 3 by a weight of 2.0.

[0260] Finally, the computing device (100) can determine the final correction difficulty based on the final score obtained by summing each of the calculated weighted scores. For example, if the computing device (100) obtains weighted scores of 3 points, 6 points, 3.6 points, and 1.2 points for each of the four correction factors, it can calculate a final score of 13.8 points by summing them.

[0261] Then, the final score can be converted into a difficulty grade according to the pre-set difficulty grade criteria. For example, 0-5 points can be classified as 'Low', 5-10 points as 'Medium', 10-15 points as 'High', and 15 points or more as 'Very High', and the final score of 13.8 points can be determined as a 'High' grade adjusted difficulty.

[0262] In step S530, if the correction difficulty calculated through step S520 is less than the reference value, the computing device (100) can correct the preliminary query generated based on the generated correction data to generate a correction query.

[0263] For example, the computing device (100) may set the correction difficulty threshold value to 15 points (or a 'very high' grade) and determine that correction is possible if the calculated correction difficulty is 13.8 points ('high' grade). In this case, the computing device (100) may sequentially apply each correction element of the correction data generated in step S510 to the preliminary query. Specifically, correction may be performed in the following order:

[0264] 1) Apply change correction: Change "SELEC" to "SELECT"

[0265] 2) Apply additional adjustment: Add "AND 재무정보_23년매출 IS NOT NULL AND CAST(재무정보_23년매출 AS INTEGER) >= 5000" to the WHERE clause

[0266] 3) Apply change correction: Change "LIKE '%Company%'" to "LIKE 'Company%'"

[0267] 4) Apply deletion correction: Delete "AND business_info_number_of_employees >= 10"

[0268] Through this correction process, the following correction query can be generated:

[0269] [Generated Correction Query]

[0270] “SELECT Company Info_Company Name, Company Info_Industry Classification, Financial Info_23 Sales, Company Info_Business Registration Number

[0271] FROM company_data

[0272] WHERE business_information_region1 = 'Seoul'

[0273] AND Company_Info_Industry_Detailed_Classification LIKE '%Manufacturing%'

[0274] AND Financial_Info_23Year_Sales IS NOT NULL

[0275] AND CAST(FinancialInformation_23YearSales AS INTEGER) >= 5000

[0276] AND Company Info_Company Name LIKE 'Corporation%'

[0277] ORDER BY CAST(재무정보_23년매출 AS INTEGER) DESC;”

[0278] On the other hand, if the calculated correction difficulty is greater than or equal to a threshold value (e.g., 18 points or a ‘very high’ grade), the computing device (100) determines that the correction is difficult and can proceed to a process of requesting the user to re-enter a natural language query.

[0279] In step S540, the computing device (100) can re-perform the first verification step, second verification step, third verification step, and fourth verification step for the correction query generated through step S530, and confirm the correction query that has passed each of the respective verification steps as the final query.

[0280] For example, the computing device (100) can input the correction query generated in step S530 back into the validation process of step S310. Specifically, for the correction query, steps S410 (first verification: feasibility), S420 (second verification: field suitability), S430 (third verification: performance degradation factors), and S440 (fourth verification: unnecessary factors) can be sequentially re-executed.

[0281] As a result of re-verification, if all verification steps are passed, the computing device (100) confirms the correction query as the final query and can use the confirmed final query to query the corporate information database. For example, if the correction query generated above passes all verifications, it can be confirmed as the final query and a notification saying "Search conditions have been successfully processed" can be sent to the user.

[0282] However, if the re-verification result still fails to pass some verification, the computing device (100) may repeat the correction process. Specifically, steps S510 through S540 may be performed again to attempt additional correction, and if the number of repetitions (e.g., 3 times) is exceeded or the difficulty of correction exceeds a threshold value, the correction may be stopped and the user may be asked to re-enter a natural language query.

[0284] FIG. 9 is a flowchart of a method for requesting re-entry of a natural language query when the validation result fails in various embodiments.

[0285] Referring to FIG. 9, if the preliminary query generated through step S230 fails to pass the validation performed through step S240, the computing device (100) may request the user terminal to re-enter the natural language query.

[0286] Specifically, in step S610, the computing device (100) can use a large-scale language model to analyze the cause of failure to pass validation.

[0287] Here, the cause of failure to pass validation refers to the fundamental reason why the preliminary query failed to meet the validation criteria, and may include, but is not limited to, ambiguity of the natural language query, condition requests not supported by the database, unclear expressions of complex logical operations, and requirements that do not match the database structure.

[0288] For example, the computing device (100) can provide the original natural language query, the generated preliminary query, and the reason for the validation failure to the large language model, and provide a prompt such as, "User's natural language query: [natural language query], generated SQL: [preliminary query], reason for validation failure: [validation failure details]. Analyze the root cause of why this SQL failed validation and explain which part of the user's natural language query caused the problem."

[0289] Large-scale language models can analyze this and derive causes such as the following:

[0290] The user described it as a 'good company,' but this is a subjective expression with no clearly corresponding column in the database.

[0291] The user requested that 'the sum of capital reserves and retained earnings is at least 1.5 times the capital,' but the database does not have such a calculated field and requires complex arithmetic operations.

[0292] The phrase "recently growing companies" implies a time-series comparison, but it is difficult to generate SQL because the period to be compared is not specified.

[0293] There is an OR condition called 'Advances or Short-term Loans', but the exact column names of the two columns cannot be found in the database.

[0294] In step S620, the computing device (100) can transmit a natural language query re-entry request message to a user terminal, which includes a correction guide for a generated preliminary query that failed to pass validation, based on the cause analyzed through step S610.

[0295] Here, the modification guide refers to specific instructions on how a user should modify a natural language query to obtain correct search results, and may include, but is not limited to, methods for clarifying ambiguous expressions, examples of available search conditions, a list of fields supported by the database, and methods for writing the correct query.

[0296] For example, the computing device (100) can generate a user-friendly correction guide based on the cause analyzed in step S610 and transmit it to the user terminal (200). Specific examples of messages may be as follows:

[0297] [Generated Modification Guide]

[0298] We apologize. There was a problem processing the search criteria you entered.

[0299] 1) Problems

[0300] It is difficult to convert the expression 'good company' into specific search criteria.

[0301] 2) Revision Guide

[0302] Please use the following specific conditions instead of 'good company':

[0303] - Sales Revenue Criteria: 'Sales revenue of 10 billion KRW or more'

[0304] - Credit rating criteria: 'Credit rating A or higher'

[0305] - Certification Criteria: 'Possession of Venture Certification' or 'Possession of INNOBIZ Certification'

[0306] - Growth Criteria: 'Sales growth rate of 20% or more over the past 3 years'

[0307] 3) Modification Example

[0308] A manufacturing company located in Seoul with annual sales of over 10 billion KRW and holding venture certification

[0309] Please modify the conditions based on the information provided and re-enter them.

[0310] In various embodiments, the computing device (100) may provide an autocomplete or suggested search term function along with a modification guide. Specifically, when a user starts re-entering, the previous natural language query may be analyzed to present similar but clearer expressions as autocomplete options. For example, when a user starts typing "good," suggested search terms such as "sales of 5 billion or more," "credit rating A or higher," and "possess venture certification" may be displayed as a dropdown menu.

[0311] In various embodiments, the computing device (100) may provide an interactive search condition input interface. Specifically, upon a request to re-enter a natural language query, it may provide an interactive UI that allows the user to select search conditions step by step. For example, it may support the user in configuring clear search conditions through step-by-step questions such as "What region are you looking for?" → "What industry are you looking for?" → "Do you have a sales amount condition?"

[0312] In various embodiments, the computing device (100) may present similar successful cases. Specifically, it may provide examples of similar natural language queries that have been successfully processed in the past, along with a modification guide, to help the user intuitively understand how to formulate the query. For example, it may provide information such as, "Examples of search conditions successfully used by other users: 'IT service companies located in Gyeonggi-do with sales of 3 billion or more and less than 5 billion won', 'Companies in Gangnam-gu, Seoul with venture certification and 100 or more employees'".

[0313] In various embodiments, the computing device (100) may provide search condition templates. Specifically, search patterns frequently used in insurance sales may be pre-configured in the form of templates, and the user may be allowed to select a template and then input only the detailed conditions. For example, templates such as "Finding new target companies (region + industry + size)", "Finding innovative companies (certification + patent + growth potential)", and "Finding financially sound companies (sales + profit + capital)" may be provided.

[0315] Referring again to FIG. 3, in step S130, the computing device (100) can provide search results for insurance sales target companies derived by querying the corporate information database using the query generated through step S120.

[0316] In various embodiments, the search results provided through the user interface may include a first screen providing a list of companies targeted for insurance sales in a list format and a second screen providing detailed information about a selected specific company, and the computing device (100) provides a list of companies targeted for insurance sales in a list format, and when any one of the provided lists is selected, it may provide corporate management summary information including at least one of financial status, industry characteristics, and management status for the selected company.

[0317] Here, corporate management summary information refers to detailed information configured to enable a comprehensive understanding of the management status of a selected company, and may include, but is not limited to, corporate profiles, financial information, credit information, shareholder information, and certification information.

[0318] Specifically, the computing device (100) can provide a list of companies targeted for insurance sales in the form of a list through a first screen, and the list of companies targeted for insurance sales provided on the first screen can be displayed in the form of an AI recommended company list. At this time, summary information including at least one of a sequence number, company name, industry, number of employees, representative, business registration number, date of establishment, address, phone number, email address, and credit rating as basic information for each company can be provided in the form of a table. Through this, the user can compare multiple searched companies at a glance and quickly select companies of interest.

[0319] In various embodiments, the computing device (100) may provide corporate profile information for a company selected through a second screen. Here, the corporate profile may include basic identification information and operational status of the company, such as the company name, business registration number, date of establishment, number of employees, name of the representative, purchase amount for the first half of the year, address, and industry.

[0320] In various embodiments, the computing device (100) may provide financial statement information through a second screen. The financial statement may include total assets, total liabilities, and total equity based on a specific point in time (e.g., the most recent closing date), thereby allowing the financial health of the company to be determined.

[0321] In various embodiments, the computing device (100) may provide corporate credit rating information through a second screen. The corporate credit rating may include the trend of changes in the credit rating by year, thereby allowing the status of changes in the company's creditworthiness to be identified.

[0322] In various embodiments, the computing device (100) may provide income statement information through a second screen. The income statement may include sales revenue, net profit, and net profit margin for the previous three years, thereby allowing for the time-series analysis of the company's profitability and growth potential.

[0323] In various embodiments, the computing device (100) may provide shareholder status information through a second screen. The shareholder status may include shareholder names, shareholding percentages, the number of shares held, and the composition of the total shares, thereby allowing the ownership structure and governance structure of the company to be identified.

[0324] In various embodiments, the computing device (100) may provide information on the status of corporate certification through a second screen. The status of corporate certification may include whether a corporate research institute is recognized, whether it is Inno-Biz certified, whether it is Main-Biz certified, whether it is a venture company certified, and the status of patent ownership, and through this, the technological capability and innovativeness of the company can be evaluated.

[0325] In various embodiments, the computing device (100) may provide search results in various forms. For example, the computing device (100) may provide a function to download the searched company list in the form of an Excel file or a PDF file, thereby enabling the user to utilize the search results offline.

[0326] In various embodiments, the computing device (100) may provide a filtering or sorting function for search results. For example, the user may reorder the searched list of companies by sales revenue, number of employees, credit rating, etc., or further filter by specific region or industry.

[0327] In various embodiments, the computing device (100) may provide a function to store and manage search history. It may store search conditions and results previously performed by the user to enable the repetition of the same search or comparative analysis with previous search results when necessary.

[0328] In various embodiments, the computing device (100) may provide a function to register a specific company found as a favorite or company of interest. This allows the user to separately manage companies that they wish to continuously monitor with interest.

[0330] The aforementioned method for providing search results for target companies for insurance sales based on artificial intelligence has been explained with reference to the flowchart illustrated in the drawings. For the sake of simplicity, the method for providing search results for target companies for insurance sales based on artificial intelligence has been illustrated and described using a series of blocks; however, the present disclosure is not limited to the order of the blocks, and some blocks may be performed in a different order or simultaneously than those illustrated and described in this specification. Furthermore, new blocks not described in this specification and drawings may be added, or some blocks may be deleted or modified.

[0332] The method described above may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may continuously store a program executable by a computer, or temporarily store it for execution or download. Additionally, the medium may be various recording or storage means in the form of a single or multiple hardware components combined, and may not be limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Furthermore, other examples of media may include recording or storage media managed by app stores that distribute applications or sites and servers that supply or distribute various other software.

[0333] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will understand that the various exemplary logical blocks, modules, circuits, and algorithmic steps described in connection with the disclosure herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate such interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in terms of their functional aspects. Whether such functions are implemented in hardware or in software depends on the design requirements imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0334] In a hardware implementation, the processing units used to perform the techniques may be implemented in one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure, computers, or a combination thereof.

[0335] Accordingly, the various exemplary logic blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors coupled with a DSP core, or any other combination of configurations.

[0336] In firmware and / or software implementations, techniques may be implemented as instructions stored on a computer-readable medium such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. The instructions may be executable by one or more processors, and the processor(s) may be enabled to perform specific aspects of the functions described in this disclosure.

[0337] When implemented in software, the techniques described above may be stored on a computer-readable medium as one or more instructions or code, or transmitted through a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transmission of a computer program from one place to another. Storage media may be any available media accessible by a computer. As a non-limiting example, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium accessible by a computer that can be used to transfer or store desired program code in the form of instructions or data structures. Additionally, any connection is appropriately referred to as a computer-readable medium.

[0338] For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of a medium. As used herein, disks and discs include CDs, laser discs, optical discs, DVDs (digital versatile discs), floppy disks, and Blu-ray discs, wherein disks usually play data magnetically, whereas discs play data optically using a laser. The above combinations should also be included within the scope of computer-readable media.

[0339] The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other known form of storage medium. An exemplary storage medium may be connected to a processor so that the processor can read information from the storage medium or write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and the storage medium may exist within an ASIC. The ASIC may exist within a user terminal. Alternatively, the processor and the storage medium may exist as separate components within the user terminal.

[0340] Although the embodiments described above have been described as utilizing aspects of the subject matter disclosed herein in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or a distributed computing environment. Furthermore, aspects of the subject matter in the present disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across a plurality of devices. Such devices may include PCs, network servers, and portable devices.

[0341] Although the present disclosure has been described in relation to some embodiments, various modifications and changes may be made without departing from the scope of the present disclosure as understood by a person skilled in the art to which the invention of the present disclosure pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification. Explanation of the symbols

[0342] 100: Computing device 200 : User terminal 300 : External server 400 : Network

Claims

Claim 1 A method for providing search results for insurance sales target companies based on artificial intelligence, performed by a computing device, comprising: a step of obtaining a natural language query including search conditions regarding insurance sales target companies; and a step of generating a query statement in the form of a Structured Query Language (SQL) based on the obtained natural language query through a previously trained artificial intelligence model. The method includes the step of providing search results for the insurance sales target companies derived by querying the corporate information database using the generated query, wherein the previously trained artificial intelligence model includes a Large Language Model (LLM) configured to generate a query in the form of a structured query and to verify the generated query, and the step of generating the query includes: a step of extracting mapping column information corresponding to the acquired natural language query among a plurality of column information for the corporate information database; a step of selecting at least one reference query among a plurality of reference queries prepared in advance based on the extracted mapping column information; and a step of generating a preliminary query by inputting the acquired natural language query, the extracted mapping column information, the selected reference query, and a pre-prepared basic prompt into the Large Language Model as input data. The method includes the step of performing validation on the generated preliminary query and generating a query that passes the validation as a final query, wherein the step of generating the final query includes the step of outputting the validation result for the generated preliminary query as input to the large-scale language model using the generated preliminary query and a pre-prepared validation prompt as input data;The method for providing search results for insurance sales target companies based on artificial intelligence includes the step of confirming a query that passes the validation result output above as a final query, wherein the pre-prepared validation prompt includes work instructions and validation criteria for the large-scale language model to perform validation of the input query. Claim 2 delete Claim 3 In claim 1, the previously trained artificial intelligence model further includes an embedding model configured to output a result of converting input text data into a vector, which is pre-trained based on training data including insurance-related term information, industry classification information, and corporate information database information; the step of extracting mapping column information corresponding to the acquired natural language query includes: a step of converting the acquired natural language query into a query vector using the embedding model; a step of acquiring a plurality of previously converted column vectors for each of the plurality of column information; a step of calculating a similarity between the converted query vector and each of the acquired plurality of column vectors; and a step of extracting column information corresponding to a column vector among the plurality of column vectors for which the calculated similarity is greater than or equal to a reference value as mapping column information, wherein the mapping column information includes a column name and descriptive information regarding the column name. Claim 4 A method for providing search results for insurance sales target companies based on artificial intelligence, performed by a computing device, comprising: a step of obtaining a natural language query including search conditions regarding insurance sales target companies; and a step of generating a query statement in the form of a Structured Query Language (SQL) based on the obtained natural language query through a previously trained artificial intelligence model. The method includes the step of providing search results for the insurance sales target companies derived by querying the corporate information database using the generated query, wherein the previously trained artificial intelligence model includes a Large Language Model (LLM) configured to generate a query in the form of a structured query and to verify the generated query, and the step of generating the query includes: a step of extracting mapping column information corresponding to the acquired natural language query among a plurality of column information for the corporate information database; a step of selecting at least one reference query among a plurality of reference queries prepared in advance based on the extracted mapping column information; and a step of generating a preliminary query by inputting the acquired natural language query, the extracted mapping column information, the selected reference query, and a pre-prepared basic prompt into the Large Language Model as input data. The method includes the step of performing validation on the generated preliminary query and generating a query that passes the validation as a final query, wherein the step of selecting at least one reference query comprises: obtaining a mapping column vector that has been transformed for the extracted mapping column information; obtaining a plurality of reference query vectors that have been transformed for each of a plurality of reference queries prepared in advance; and calculating a similarity between the obtained mapping column vector and each of the obtained plurality of reference query vectors.A method for providing search results for insurance sales target companies based on artificial intelligence, comprising the step of selecting a predetermined number of reference query vectors in order from the reference query vectors with high similarity calculated among the plurality of reference query vectors obtained above, and selecting them as reference queries corresponding to the selected reference query vectors. Claim 5 A method for providing search results for insurance sales target companies based on artificial intelligence, performed by a computing device, comprising: a step of obtaining a natural language query including search conditions regarding insurance sales target companies; and a step of generating a query statement in the form of a Structured Query Language (SQL) based on the obtained natural language query through a previously trained artificial intelligence model. The method includes the step of providing search results for the insurance sales target companies derived by querying the corporate information database using the generated query, wherein the previously trained artificial intelligence model includes a Large Language Model (LLM) configured to generate a query in the form of a structured query and to verify the generated query, and the step of generating the query includes: a step of extracting mapping column information corresponding to the acquired natural language query among a plurality of column information for the corporate information database; a step of selecting at least one reference query among a plurality of reference queries prepared in advance based on the extracted mapping column information; and a step of generating a preliminary query by inputting the acquired natural language query, the extracted mapping column information, the selected reference query, and a pre-prepared basic prompt into the Large Language Model as input data. The method includes the step of performing validation on the preliminary query generated above and generating a query that passes the validation as a final query, wherein the pre-prepared basic prompt includes work instructions for the large-scale language model to generate a query in the form of a structured query, a description of the enterprise information database, and query writing rules, and the step of generating the preliminary query includes the step of configuring a query generation prompt by combining the acquired natural language query, the extracted mapping column information, and the selected reference query with the pre-prepared basic prompt;A method for providing search results for insurance sales target companies based on artificial intelligence, comprising the step of inputting the above-configured query generation prompt into the above-described large-scale language model to generate a preliminary query in the form of a structured query. Claim 6 delete Claim 7 In claim 1, the step of outputting the validation result comprises: a first validation step of deriving a first validation result by verifying whether the generated preliminary query is an executable query in the corporate information database using the large-scale language model; a second validation step of deriving a second validation result by verifying whether the search field included in the generated preliminary query corresponds to a search condition regarding the insurance sales target company using the large-scale language model; a third validation step of deriving a third validation result by verifying whether the generated preliminary query includes a performance degradation factor of the corporate information database using the large-scale language model; a fourth validation step of deriving a fourth validation result by verifying whether the generated preliminary query includes unnecessary conditions or unnecessary search fields using the large-scale language model; and a step of outputting the validation result as "passed" when the derived first validation result, the derived second validation result, the derived third validation result, and the derived fourth validation result are all "passed". Claim 8 In claim 7, the step of generating the final query further comprises: a step of generating correction data for the generated preliminary query using the large-scale language model to correspond to the verification step where the non-passing verification result was derived, if at least one of the derived first verification result, derived second verification result, derived third verification result, and derived fourth verification result is not passed; a step of calculating the correction difficulty for the generated correction data; a step of generating a correction query by correcting the generated preliminary query based on the generated correction data if the calculated correction difficulty is less than a reference value; and a step of re-performing the first verification step, the second verification step, the third verification step, and the fourth verification step for the generated correction query, and confirming the correction query that has passed each verification step as the final query. Claim 9 In claim 8, the step of generating the correction data comprises: using the large-scale language model to identify at least one correction-needing part indicating a target requiring correction in the generated preliminary query based on the type of verification step from which a non-pass verification result is derived; identifying one of a correction type among deletion correction, addition correction, and modification correction for the identified correction-needing part; generating a correction element corresponding to any one of the identified correction type; and applying the generated correction element to the identified correction-needing part to generate correction data, wherein the step of calculating the correction difficulty comprises assigning a score to each of the generated correction elements included in the generated correction data and determining the correction difficulty based on the assigned score. Claim 10 In claim 9, the step of determining the correction difficulty comprises: a step of assigning a preset base score corresponding to one of the correction types among deletion, addition, and change to each of the generated correction elements included in the generated correction data; a step of calculating a weighted score for each of the generated correction elements by applying different weights to the assigned base score according to the type of verification step in which a verification result other than passing is derived; and a step of determining the final correction difficulty based on the final score obtained by summing each of the calculated weighted scores. Claim 11 A method for providing search results for insurance sales target companies based on artificial intelligence, wherein the step of generating the query further includes the step of requesting a user terminal to re-enter a natural language query if the generated preliminary query fails to pass the validation performed above. Claim 12 In claim 11, the step of requesting a user terminal to re-enter the natural language query comprises: a step of analyzing the cause of failure to pass validation using the large-scale language model; and a step of transmitting a natural language query re-entry request message to the user terminal, including a correction guide for the generated preliminary query that failed validation, based on the analyzed cause. Claim 13 A method for providing search results for insurance sales target companies based on artificial intelligence, wherein the step of providing the search results comprises: a step of providing a list of insurance sales target companies in the form of a list; and, when any one of the provided list of companies is selected, a step of providing corporate management summary information including at least one of financial status, industry characteristics, and management status for the selected company. Claim 14 A computing device comprising: a memory; and at least one processor connected to the memory and configured to execute at least one computer-readable program contained in the memory, wherein the at least one program comprises instructions for performing a method according to any one of claims 1, 3 to 5, and 7 to 13 when executed by the at least one processor. Claim 15 A computer program stored on a computer-readable non-transient recording medium to execute a method according to any one of paragraphs 1, 3 through 5, and 7 through 13 on a computing device.

Images