Survey system and survey method

Abstract

This technology improves the accuracy of surveys using virtual panels. [Solution] A survey system is used that acquires value information from multiple monitors, divides the monitors into multiple segments based on the value information, creates a persona that reflects the value information for each of the multiple segments, obtains a questionnaire targeting one of the multiple segments, creates a prior distribution which is the trend of past responses of the segment targeted by the questionnaire, inputs prompts containing information about the personas, information about the questionnaire, and information about the prior distribution into a large language model, and obtains and outputs the responses from the large language model.

Description

Technical Field

[0001] The present invention relates to an investigation system and an investigation method.

Background Art

[0002] Market research plays an important role in product development and marketing strategies. Conventionally, panel surveys, in which a large number of monitors (respondents) are organized to form a panel and questionnaires are sent to the monitors, have been the mainstream.

[0003] However, the following problems have existed in conventional panel surveys. First, there is the problem of high cost. Enormous costs are incurred in each process such as monitor recruitment, survey design, data collection, and labor costs. For example, the cost per case may exceed several million yen, which has become an obstacle to implementation. In addition, there is also the problem that the lead time becomes long. It may take several weeks to several months from monitor recruitment, survey implementation, tabulation, and analysis. Such delays may lead to opportunity losses in the modern era where the market changes rapidly. There is also a risk of information leakage. That is, there is a possibility that confidential information such as pre-launch products and concepts may be leaked to the outside through monitors.

[0004] By the way, in recent years, the development of AI technology, particularly large language models (LLMs), has been remarkable, and its scope of application has been expanding. Since LLMs can simulate human language understanding, generation, and inference capabilities, their use in various fields is expected. As one such method of using LLMs, constructing a virtual panel by having the LLMs imitate consumers has been studied.

[0005] Patent Document 1 discloses a system including means for creating a plurality of virtual personalities and means for conducting hearings and questionnaire analysis on the virtual personalities for market research.

[0006] In addition, Patent Document 2 discloses a technique related to questionnaire surveys that uses digital clones that answer questions instead of humans. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2025-052304 [Patent Document 2] Patent No. 7101357 [Overview of the project] [Problems that the invention aims to solve]

[0008] However, the use of LLMs and AI in research is still in its infancy, and there is much room for improvement. For example, research using virtual personalities or digital clones, as disclosed in prior art, is limited to responses from uniform virtual personalities, making it difficult to reproduce the diverse values ​​and thought processes of consumers. As a result, there are challenges in the accuracy and depth of insight of the research. Therefore, further improvements in accuracy are needed.

[0009] This invention has been made in view of the above-mentioned problems, and its purpose is to provide a technology that improves the accuracy of surveys using virtual panels. [Means for solving the problem]

[0010] To achieve the above objective, the present invention employs the following configuration. That is, A means for acquiring value information that obtains value information from multiple monitors, A segmentation means that divides the plurality of monitors into a plurality of segments based on the aforementioned value information, A persona creation means for creating a persona that reflects the value information for each of the aforementioned multiple segments, A means for obtaining questionnaires targeting any of the aforementioned multiple segments, A means for creating a prior distribution which is the trend of past responses of the segment targeted by the aforementioned questionnaire, A prompt input means inputs a prompt to a large-scale language model that includes information about the persona created by the persona creation means, information about the questionnaire acquired by the questionnaire acquisition means, and information about the prior distribution created by the prior distribution creation means. A response output means that obtains and outputs a response from the aforementioned large-scale language model, This is a survey system characterized by having the following features:

[0011] The present invention also employs the following configuration: namely, The information processing device performs a value information acquisition step, which involves acquiring value information from multiple monitors, The information processing device performs a segmentation step, which divides the plurality of monitors into a plurality of segments based on the value information, The information processing device performs a persona creation step for each of the plurality of segments, which creates a persona that reflects the value information, The information processing device includes a questionnaire acquisition step of acquiring a questionnaire targeting any of the plurality of segments, The information processing device performs a prior distribution creation step, which creates a prior distribution that represents the trend of past responses of the segment targeted by the questionnaire, The information processing device inputs a prompt to a large-scale language model, which includes a prompt containing information about the persona created in the persona creation step, information about the questionnaire obtained in the questionnaire acquisition step, and information about the prior distribution created in the prior distribution creation step. The information processing device performs a response output step in which it obtains and outputs a response from the large-scale language model, This is a research method characterized by having the following features. [Effects of the Invention]

[0012] According to the present invention, it is possible to provide a technology that improves the accuracy of surveys using virtual panels. [Brief explanation of the drawing]

[0013] [Figure 1] Block diagram for explaining the physical configuration of the entire survey system [Figure 2] Block diagram for explaining the functional configuration of the survey system [Figure 3] Diagram for explaining the image of the questionnaire [Figure 4] Diagram for explaining an example of a prompt

Mode for Carrying Out the Invention

[0014] Hereinafter, preferred embodiments of the present invention will be described while referring to the drawings. However, the device configuration, functional block configuration, relative arrangement thereof, etc. described below should be appropriately changed according to various conditions of the system to which the invention is applied, and are not intended to limit the scope of this invention to the following description.

[0015] The present invention relates to a technique for conducting a questionnaire using a virtual panel to obtain answers and analyzing the obtained answers. The present invention can be regarded as a survey method, a survey system, or a survey device. The present invention can also be regarded as a control method for a survey system or a survey device. The present invention can also be regarded as a program that operates using the computing resources of a survey device or a survey system and executes each step of the survey method, and a storage medium in which the program is stored. The storage medium may be a non-temporary storage medium readable by a computer.

[0016] In the following examples, the "investigator" is the entity that conducts the questionnaire, which may be a group such as a company or an individual. It may also be a professional trader or organization that conducts a survey under commission. Also, the "monitor" refers to each individual entity (respondent) who creates an answer to the questionnaire, and a "panel" is constructed by a large number of monitors.

[0017] In this embodiment, the entity responding to the questionnaire differs depending on the stage of the processing flow, and both real natural persons and virtual personalities are possible. Hereafter, when it is necessary to distinguish between the personalities of the respondents, natural persons will be referred to as "real monitors" and "real panels," and virtual personalities will be referred to as "virtual monitors" and "virtual panels."

[0018] The following describes a technology for conducting consumer research by building a virtual panel using LLM. The following example illustrates a concept study conducted as part of marketing. This is a test where a new product or service is presented to a group of monitors before market launch to investigate their reactions.

[0019] However, the application of this invention is not limited to concept research. Typically, this invention can be used in research formats designed based on the "SOR model," which captures consumer behavior in three stages: "Stimulus," "Organism," and "Response." This is a research method that measures how consumers who are stimulated arrive at their final action by obtaining internal evaluations (O) and final actions (R) in response to a stimulus (S) through questionnaires. In addition to concept research, other possible research subjects include packaging, naming, logos, price acceptance, functional evaluation, brand image, advertising effectiveness, and loyalty.

[0020] (System configuration) Referring to Figure 1, the physical configuration of the survey system 1 will be described. The survey system 1 includes an information processing device 10 used by the surveyor and actual monitor terminals 20 (20a, 20b…) used by the actual monitors. Hereafter, subscripts (a, b) will be omitted unless necessary to distinguish between them. The information processing device 10, which functions as a server, and the actual monitor terminals 20 are connected to each other so that they can communicate with one another via a line such as the Web or a dedicated network. Note that the information processing device 10 may also serve as an actual monitor terminal 20.

[0021] The researcher uses the information processing device 10 to perform various information processing tasks such as surveys and analyses. The information processing device 10 comprises a control unit 1001 such as a CPU, a storage unit 1002 such as ROM, RAM, or HDD, a communication unit 1003 such as a communication adapter, an input unit 1004 such as a mouse, keyboard, or touch panel, and a notification unit 1005 such as a display or speaker. A PC or workstation is preferred as the information processing device 10. Alternatively, a cloud server that utilizes computing resources on the cloud may be used as the information processing device 10.

[0022] The actual monitor terminal 20 can be an electronic device such as a PC, smartphone, or tablet. The user may also use an electronic device they use daily as the actual monitor terminal 20 by installing the survey application on that device. Alternatively, the survey may be conducted via a browser installed on the actual monitor terminal 20. (Illustrated example of actual monitor terminal) Terminal 20a is a smartphone, and its touch panel serves as both a display unit 22 for displaying the survey screen and an operation unit 25 for receiving input from the actual monitor. The actual monitor terminal 20b is a PC, with the display being the display unit 22 and the keyboard and other components being the operation unit 25. The actual monitor terminal 20 is not limited to these; it only needs to have the functionality to display questions and accept input from the user.

[0023] (Process flow) Next, an example of the processing of the present invention will be described with reference to a specific embodiment. Figure 2 is a functional block diagram showing the functions realized by the information processing device 10. The information processing device 10 functions as a means corresponding to each block in the figure, as the control unit 1001 operates according to the program deployed in the storage unit 1002. In the following processing, each means of the investigation system 1 corresponds to each step when the present invention is considered as an investigation method.

[0024] Each block (means) of the information processing device 10 is broadly divided into segmentation unit A, pre-distribution creation unit B, and response generation unit C. However, the division of blocks and units is for convenience only and is not limited to this. Segmentation unit A performs pre-segmentation of value information prior to the concept survey and creates personas as virtual monitors. Pre-distribution creation unit B creates a pre-distribution showing past evaluation trends based on the content of the concept survey. Response generation unit C receives prompt input and generates responses.

[0025] In this embodiment, virtual monitors and a virtual panel are created prior to the concept research. The virtual panel is a market research tool in which AI plays the roles of numerous consumers, allowing for a simulation in advance of how they feel and behave towards new products, etc. Traditional market research required organizing a large number of natural people (real monitors) to form a real panel and conducting surveys and interviews. However, by using a virtual panel, it is possible to reduce time and costs and avoid the risk of information leakage.

[0026] Step 1: Provide information to the AI ​​to build a persona. For a virtual monitor to behave as a consumer, it's first necessary to provide the AI ​​with information about "what kind of person this is" to create a specific consumer's personality. Hereafter, this constructed virtual personality will be referred to as a "persona."

[0027] First, the values ​​information acquisition means 101 collects values ​​information by conducting questionnaires with a large number of actual monitors via the actual monitor terminals 20. Values ​​information is the judgment criteria that lie deep within the heart and serve as the reasons for behavior that appears on the surface, and it is thought that consumer behavior reflects the values ​​of each individual. Furthermore, since consumers who share common values ​​are thought to often exhibit similar behavioral patterns even if they differ in age or gender, it becomes possible to obtain a consumer understanding that cannot be achieved with statistical attribute information alone.

[0028] Here, we will consider "new gadget lovers" who prefer new products and "stability-oriented" people who prefer standard products as examples of values. The types and number of values ​​are arbitrary. For example, various values ​​can be set according to the marketing objective, such as "price-conscious," "quality-conscious," "convenience-first," "environmentally conscious," "brand / status-conscious," "experience / empathy-conscious," and "pursuit of self-expression."

[0029] The value information acquisition method 101 involves conducting a survey with actual monitors to determine whether they are "new-fashion enthusiasts" or "stability-oriented." For example, a direct question such as "Do you check out new products as soon as they come out?" could be asked. Alternatively, a question such as "What do you usually value when choosing products?" could be asked, with options that reflect various values. In addition, various existing survey methods can be used. Survey items include actual monitors. In addition to insightful data that helps understand the inner workings of the monitor, it is also possible to acquire behavioral data such as actual purchase history. Furthermore, information held by the actual panel (such as attribute information and past survey response history) can also be used.

[0030] Furthermore, the sample survey data used for information collection may be biased, and it is possible that the segmentation described later does not accurately reflect the proportions of actual consumers. Therefore, the composition ratio adjustment means 102 may correct the bias by performing weighting back as needed to match the composition ratios of a known population. In this embodiment, the composition ratio adjustment means 102 corrects the composition ratios based on actual panel information obtained from a conventional survey.

[0031] The segmentation means 103 performs segmentation (grouping) based on the value information acquired by the value information acquisition means 101 (or adjusted by the composition ratio adjustment means). Any clustering method can be used for segmentation. As an example, the Self-Organizing Map (SOM) technique can be used, which groups similar data together to create a two-dimensional data map from complex data. Alternatively, hierarchical clustering may be used in combination with SOM. This method allows for flexible adjustment of segment granularity while considering the nonlinear structure of the data.

[0032] This allows for segmentation that reflects hidden similarities and natural groupings, offering a different perspective than classifications based on predetermined criteria such as "women in their 20s" or "residents of Tokyo." Consequently, it enables pinpoint insights tailored to the values ​​of target consumers in fragmented markets.

[0033] Furthermore, by changing the structure of the questionnaire, it is possible to perform segmentation not only based on general consumer values ​​but also segmentation specialized for specific domains. In general segmentation, consumers can be classified by broad values ​​that influence their overall behavior and purchasing attitudes. On the other hand, in domain-specific segmentation, consumers can be classified based on the points and values ​​that consumers prioritize when choosing products in a specific product or service category (e.g., automobiles, cosmetics, home appliances, etc.).

[0034] For example, in the automotive market, if we use the common segmentation criteria of "environment-focused" and "price-focused," the former might tend to choose electric or hybrid vehicles, while the latter might tend to choose kei cars or compact cars. On the other hand, segmentation specific to a particular domain (automobiles) could include "performance-focused," emphasizing speed and handling; "design and brand-focused," emphasizing appearance and image; "economy-focused," emphasizing price and fuel efficiency; and "safety-focused," emphasizing crash safety. By classifying in this way, it becomes possible to understand the unique needs of each domain and conduct precise marketing. Furthermore, these various emphasis trends do not necessarily exist independently, but may have some degree of correlation with one another.

[0035] Persona creation method 104 creates profiles of representative personas for each group. In doing so, persona creation method 104 creates profiles using variables that have a high correlation with the target variables (such as concept research) based on correlation coefficients or extraction using Boruta. This creates a concrete hypothetical consumer profile. For example, it is set to include at least value information, such as "female in her 30s, office worker, lives in Tokyo, enjoys traveling, and values ​​the value of 'liking new things'." In addition, various settings necessary for the research, such as detailed attribute information and lifestyle, may be added. The created persona information should be written in a structured format such as Markdown to facilitate processing in LLM.

[0036] As will be discussed later, the personas created in this way are used to instruct the LLM on how to behave. This allows the LLM to answer questions not merely as a conditioned reflex, but as if the persona itself were thinking.

[0037] Step 2: Obtain information that shows real-world market conditions and statistical trends. By using personas, LLMs can improve the accuracy of their answers by taking on the role of specific consumers when answering questions. However, simply using personas may result in answers that do not reflect actual market trends. Therefore, in this step, pre-distribution creation unit B utilizes past market research data to generate information that shows real market and statistical trends for teaching to LLMs, thereby further improving the accuracy of the research.

[0038] The concept input means 111 (questionnaire acquisition means) receives input from the researcher and acquires information about the content of the current concept survey. Figure 3 is an example of the survey content, showing what would be displayed on the display unit 22 if an actual monitor survey were conducted. The image display field 221 shows an image of the new product (avocado wasabi chips) that is the subject of the survey. The text display field 222 shows the question number (Q1), the product concept text, and the question content and answer choices. The concept input means 111 refers to the raw data of the concept survey and understands the survey content.

[0039] The research history retention means 112 acquires and retains the history of various concept research. The research history retention means 112 retains past research history tagged from various perspectives. The research history itself may be stored in the memory unit 1002, and the research history retention means 112 may be equipped with a function to cooperate with the memory unit 1002.

[0040] The similar concept extraction means 113 (similar questionnaire extraction means) refers to the survey history retention means 112 based on the survey content received by the concept input means 111, and extracts evaluations of products similar to the current concept from the history of numerous market surveys conducted in the past. Specifically, the similar concept extraction means 113 analyzes the concept descriptions and other information to numerically calculate and determine the degree of similarity. A suitable method for calculating the degree of similarity is to use a text embedding model to vectorize the product to be evaluated and the group of past concepts, and then calculate the distance between the two vectors. The number of extractions is not limited to one; multiple concepts with high similarity (for example, 5) may be extracted for each segment. In this example, the extraction of concepts related to novel flavored foods (confectionery) is expected.

[0041] The segment determination means 114 analyzes the reaction trends of each pre-categorized consumer segment (e.g., novelty-seeking, stability-oriented, etc.) to determine the degree of purchasing intent that the presented concept will evoke in each pre-categorized consumer segment, based on the values ​​and other profile information that each persona already possesses. In doing so, it calculates the distribution of evaluations that each segment is likely to show towards the concept (e.g., the ratio of "want to buy" to "don't want to buy") based on evaluation data of similar concepts from the past.

[0042] The prior distribution creation means 115 examines how similar concepts extracted by the similar concept extraction means 113 have been evaluated in each segment in the past. For example, for the "new-fashion group," it may be obtained that "60% of people were positive about this type of novel food, 30% were neutral, and 10% were negative." Alternatively, it may be obtained as a frequency distribution or a more detailed graph. This past evaluation trend is then taught to the LLM as a prior distribution (a trend known in advance). This prior distribution is referenced by the LLM when considering its response, as "common sense and general reaction trends of a particular group that serve as the underlying assumptions."

[0043] The matching mechanism 121 matches the created prior distribution with persona information and incorporates it into prompts for the LLM, which mimics consumers. In essence, it matches individual personas with prior distributions using segments as a key. This allows the LLM to respond not only by being given a persona profile (individual values), but also by referring to general market response trends, such as how the segment to which the persona belongs has reacted to similar concepts in the past. As a result, the responses of the virtual monitors become more realistic and statistically plausible.

[0044] Step 3: Output the answer The response generation unit C inputs prompts to the LLM based on the persona obtained by the persona creation means 104 and the prior distribution obtained by the prior distribution creation means 115, and generates responses. The LLM may be a general-purpose LLM as is, or it may be fine-tuned based on the content of the survey, the target audience, the segments, etc. Alternatively, the researcher may prepare the training data themselves and generate a trained model. In this embodiment, the prompt input means is divided into a system prompt input means 131 and a user prompt input means 132, but the configuration is not limited to this.

[0045] The system prompt input means 131 generates and inputs a system prompt that includes instructions for the LLM. Figure 4(a) shows an example of a system prompt 251. Thus, the system prompt consists of a series of instructions that include at least information about the persona and information about the location of the prior distribution, in order to operate the LLM as a virtual monitor.

[0046] The user prompt input means 132 generates and inputs user prompts, including questionnaires for the virtual monitor. Figure 4(b) shows an example of a user prompt 252. Thus, a user prompt consists of a series of instructions that include the content that the user actually wants to investigate with the virtual monitor.

[0047] LLM answers questions presented in user prompts based on the persona (information indicating "what kind of person you are") and prior distribution (information indicating "what the actual market trends are") provided by the system prompt. In this embodiment, instead of a single definitive answer being given, the output is a distribution of probabilities for each option (posterior distribution), such as "the probability of having a positive opinion is 30%, the probability of having a negative opinion is 10%, and the probability of being neutral is 60%." The posterior distribution is not limited to three levels; it may be a frequency distribution divided into more levels, or in other forms.

[0048] By outputting the answers as a posterior distribution of probabilities, it becomes possible to express the breadth of human thought and hesitation, realistically reproducing human uncertainty. Furthermore, the LLM answers will not be biased towards any particular answer, resulting in diverse and natural results. The answer output method in this embodiment can be considered a Bayesian estimation-like method that estimates an unknown distribution (posterior distribution) from existing knowledge (prior distribution) and observed events (persona and evaluation target). In order to improve the reproducibility of the output results, a sampling method constrained by the distribution shape may be adopted instead of simple probabilistic sampling.

[0049] The response output means 133 outputs the responses obtained from the virtual monitors included in the virtual panel to the researcher. When using a large number of virtual monitors, the responses output from each may be weighted based on the actual market segment composition ratio. For example, if the simulation showed a large number of "people who like new things" but the actual market had fewer, it would be good to weight the responses to reflect the reality.

[0050] (effect) The research method described in this embodiment yields the following benefits. First, it allows for prediction of consumer behavior by delving into underlying factors such as "values." Second, it significantly reduces the effort, time, and cost of conducting actual surveys, enabling high-speed and low-cost simulations. When the applicant actually compared responses from a real panel and a virtual panel on the same concept, it was confirmed that the real panel could be reproduced with high accuracy. The correlation coefficient between the posterior distributions was a high value with an average of 0.86 (0.7-0.99). Furthermore, by combining the thinking of virtual monitors as "personas," collective intelligence obtained from past market data, and probabilistic response output, predictions closer to the real market become possible than with conventional simulations, improving accuracy. In addition, various hypotheses with slightly modified concepts can be easily simulated repeatedly.

[0051] As described above, the survey method according to the present invention enables the implementation of surveys with the same high level of reliability as conventional surveys using actual monitors or panels. Furthermore, it solves conventional problems such as high costs, long lead times, and information leakage risks.

[0052] Furthermore, the present invention allows for the free construction of panels with specific segments and response tendencies, enabling the implementation of surveys with excellent flexibility and scalability. Specifically, regarding segments, the segmentation means 103 may be provided with a user interface that allows the researcher to adjust how segments are divided and the composition ratio of monitors within each segment. Regarding response tendencies, the persona creation means 104 may be provided with a user interface that allows the researcher to adjust the response tendencies. Alternatively, the prior distribution creation means 115 may be made adjustable in terms of the ratio of prior distributions. [Explanation of symbols]

[0053] 1: Survey system, 10: Information processing device, 101: Value information acquisition means, 103: Segmentation means, 104: Persona creation means, 111: Concept input means, 112: Survey history retention means, 113: Similar concept extraction means, 115: Pre-distribution creation means, 131: System prompt input means, 132: User prompt input means

Claims

1. A means for acquiring value information that obtains value information from multiple monitors, A segmentation means that divides the plurality of monitors into a plurality of segments based on the aforementioned value information, A persona creation means for creating a persona that reflects the value information for each of the aforementioned multiple segments, A means for obtaining questionnaires targeting any of the aforementioned multiple segments, A means for creating a prior distribution which is the trend of past responses of the segment targeted by the aforementioned questionnaire, A prompt input means inputs a prompt to a large-scale language model that includes information about the persona created by the persona creation means, information about the questionnaire acquired by the questionnaire acquisition means, and information about the prior distribution created by the prior distribution creation means. A response output means that obtains and outputs a response from the aforementioned large-scale language model, A survey system characterized by comprising the following features.

2. A means for maintaining survey history, which is the history of past surveys, A similar questionnaire extraction means extracts from the survey history retention means survey history that has similar content to the questionnaire acquired by the questionnaire acquisition means, Furthermore, The prior distribution creation means creates the prior distribution based on the survey history extracted by the similar questionnaire extraction means. The investigation system according to feature 1.

3. The similar questionnaire extraction means evaluates the similarity between the questionnaire and the survey history using a text embedding model. The investigation system according to feature 2.

4. The response output means outputs the response from the large-scale language model as a posterior distribution in a format corresponding to the prior distribution. The investigation system according to any one of claims 1 to 3.

5. The segmentation means further comprises a ratio adjustment means for correcting the ratio of the monitors in the plurality of segments created by the segmentation means based on the ratio of known populations. The investigation system according to any one of claims 1 to 3.

6. The segmentation means can create the multiple segments that are specialized for a particular domain. The investigation system according to any one of claims 1 to 3.

7. The aforementioned specific domain corresponds to a specific category of goods or services. The investigation system according to feature 6.

8. The segmentation means allows the researcher to adjust the configuration of the segments. The investigation system according to any one of claims 1 to 3.

9. The persona creation method allows the researcher to adjust the response trends of the persona. The investigation system according to any one of claims 1 to 3.

10. The aforementioned pre-distribution creation means allows the researcher to adjust the pre-distribution. The investigation system according to any one of claims 1 to 3.

11. The aforementioned survey is a concept survey that presents the concept of a product or service to the respondents. The investigation system according to any one of claims 1 to 3.

12. The information processing device performs a value information acquisition step, which involves acquiring value information from multiple monitors, The information processing device performs a segmentation step, which divides the plurality of monitors into a plurality of segments based on the value information, The information processing device performs a persona creation step for each of the plurality of segments, which creates a persona that reflects the value information, The information processing device includes a questionnaire acquisition step of acquiring a questionnaire targeting any of the plurality of segments, The information processing device performs a prior distribution creation step, which creates a prior distribution that represents the trend of past responses of the segment targeted by the questionnaire, The information processing device inputs a prompt to a large-scale language model, which includes a prompt containing information about the persona created in the persona creation step, information about the questionnaire obtained in the questionnaire acquisition step, and information about the prior distribution created in the prior distribution creation step. The information processing device performs a response output step in which it obtains and outputs a response from the large-scale language model, A research method characterized by having the following features.

Images