Information processing device, information processing method, program, and information processing system

Abstract

To provide a technology that facilitates utilization of a model.SOLUTION: The present invention comprises: storage means for storing a model generated using text data as training data, the text data being based on at least one of voice information and character information received by input reception means; text information generation means for generating, based on a question and the model, text information indicating an answer to the question; and screen generation means for generating a screen including attribute information of data providers who have provided the at least one of voice information and character information.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, a program, and an information processing system.

Background Art

[0002] There is known a generative AI that can generate text information from various contents (such as text, images, and voices). Conventional AI only presented the most appropriate answer from the learned data, but generative AI can continuously learn by itself, learn information and data not given by humans, and output original content that has never been input.

[0003] Techniques for improving the quality of learning data in machine learning are known (see, for example, Patent Document 1). Patent Document 1 discloses a technique in which a learning model outputs a failure recovery procedure using the failure information received from a user as an input, and based on the skill information indicating the user's failure recovery skills, sets an evaluation weight regarding the usefulness of the failure recovery procedure used for re-learning by the learning model.

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the conventional technology has a problem that the promotion of model utilization has not been achieved.

[0005] In view of the above problems, the present invention provides a technique for promoting the utilization of a model.

Means for Solving the Problems

[0006] In view of the above problems, the present invention includes a storage means for storing a model generated using text data based on at least one of voice information and character information received by an input reception means as learning data, a text information generation means for generating text information indicating an answer to a question based on the question and the model, and a screen generation means for generating a screen including attribute information of a data provider who provided at least one of the voice information and the character information.

Effect of the Invention

[0007] It is possible to provide a technique for promoting the use of the model.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Embodiments for Carrying Out the Invention

[0009] Hereinafter, as an example of an embodiment for carrying out the present invention, an information processing system and an information processing method performed by the information processing system will be described with reference to the drawings.

[0010] <Supplement on Tacit Knowledge> In industries such as civil engineering and architecture, BIM / CIM implementation is underway for purposes such as addressing the declining birthrate and aging population, and improving labor productivity.

[0011] BIM is an abbreviation for Building Information Modeling, and is a solution for utilizing information in all processes from the design, construction, to maintenance management of a building by adding attribute data such as cost, finish, and management information to a three-dimensional digital model of a building (hereinafter referred to as a 3D model) created on a computer.

[0012] CIM is an abbreviation for Construction Information Modeling, and is a solution for the civil engineering field (covering all infrastructure such as roads, electricity, gas, and water supply) proposed following BIM which has been advanced in the construction field. Similar to BIM, it aims to improve and sophisticate a series of construction production systems by sharing information among relevant parties centered around a 3D model.

[0013] What is important in promoting BIM and CIM implementation is how to utilize the constructed BIM and CIM.

[0014] Specifically, the 3D model restored by BIM and CIM can be utilized not only for design and construction purposes, but also for other tasks such as maintenance management and site surveys. That is, not only the use as design drawings, but also other uses such as leaving records in the 3D model and sharing with others can be considered.

[0015] In addition, since the work performed on the 3D model can be recorded as a log, if tacit knowledge can be extracted based on these, it can be effectively utilized for technology transfer from experts to novices and the like. As a result, it is expected to lead to front-loading of operations and personnel training.

[0016] Here, focusing on the transfer of tacit knowledge, it can be said that the problem is how to transfer tacit knowledge not only between different operations but also between users with different levels of proficiency, not limited to 3D models but also for 2D data (such as omnidirectional images and planar images).

[0017] Specifically, since tacit knowledge is qualitative and difficult to quantify, even if a tacit knowledge model is generated from tacit knowledge, it is difficult to secure users' trust in the tacit knowledge model, and it is difficult to promote the use of the tacit knowledge model. For example, if the specialized field of the tacit knowledge model is different from that of the user, no matter how excellent the tacit knowledge model is, it has no use value for the user. Similarly, if the knowledge level of the tacit knowledge model is lower than that of the user, it has no use value for the user.

[0018] However, it is also a fact that the tacit knowledge model can give users new perspectives and insights. By using the tacit knowledge model, even users with little experience are full of the possibility of acquiring know-how and technology and applying them to work.

[0019] Therefore, in this embodiment, a technique for promoting the use of the tacit knowledge model is provided not only for users who recognize the usefulness of the tacit knowledge model but also for users who do not recognize the usefulness of the tacit knowledge model.

[0020] <Regarding terms> The attribute relevance is an index indicating the degree of relevance between the attributes of the user and the data provider or between the user and past users. The attribute relevance numerically represents, for example, the degree of relevance between the attribute information of the user and the attribute information of the data provider. Although the calculation method of the attribute relevance will be described later, existing methods such as focusing on the commonality of words are known.

[0021] A user is a person who uses the text information generated by the tacit knowledge model (which may output content other than text such as images). A data provider is a person who provides the data (voice information, character information, operation information, images, 3D data, etc.) used by the tacit knowledge model for learning.

[0022] Tacit knowledge is knowledge based on an individual's experience and intuition. A tacit knowledge model is a model that learns tacit knowledge and outputs an answer based on the learned tacit knowledge in response to a question. A model refers to a mechanism or artificial intelligence (AI) that learns the correspondence between input data and output data and outputs output data for the input data. Note that the output data does not depend on the presence or absence of teacher data.

[0023] [First Embodiment] [System Configuration Example] FIG. 1 is an overall configuration diagram of an information processing system 100 according to an embodiment of the present invention. The information processing system 100 of the present embodiment includes a terminal device 10, which is an example of an input / output device, and a server 40.

[0024] The server 40 is one or more information processing devices that can communicate with the terminal device 10 via a communication network N. The server 40 may be a Web server that returns a processing result to the terminal device 10 in response to a request from the terminal device 10. A server is a computer or software that performs a function of providing information and a processing result in response to a request from a client.

[0025] Server 40 may support cloud computing. Cloud computing refers to a usage model in which resources on a network are utilized without awareness of specific hardware resources. Cloud computing has forms such as SaaS (Software as a Service), PaaS (Platform as a Service), or IaaS (Infrastructure as a Service), and any form may be used. Therefore, server 40 does not necessarily need to be housed in a single enclosure or provided as an integrated device. The functions of server 40 may be distributed among multiple information processing devices, or multiple information processing devices may each have all the functions, and the information processing device for processing may be switched by load distribution or the like.

[0026] Also, instead of server 40 having the large language model 4005 described later, server 40 may call an API (Application Programming Interface) published by an external system and utilize the large language model 4005.

[0027] Terminal device 10 is a general-purpose information processing terminal used by the user of information processing system 100. In terminal device 10, a web browser or a dedicated native app for server 40 operates. When terminal device 10 executes a web browser, terminal device 10 and server 40 execute a web app. A web app is an application that operates by the cooperation of a program in a programming language (e.g., JavaScript (registered trademark)) operating on a web browser and a program on the web server (server 40) side. When executing the web app, the processing of this embodiment may be performed by server 40 or by terminal device 10 that has received the web app.

[0028] An application that cannot be executed unless it is installed on the terminal device 10 is called a native application. Regarding this embodiment as well, the application executed on the terminal device 10 may be a web application or a native application. Also in this case, the processing of this embodiment may be performed by the server 40, or may be performed by the terminal device 10 that executes the native application.

[0029] The terminal device 10 is, for example, a PC (Personal Computer), a smartphone, a PDA (Personal Digital Assistant), a tablet terminal, or the like. In addition, the terminal device 10 may be any device on which a web browser or a native application operates. The terminal device 10 may be an electronic blackboard, a television receiver, a glass device, or a wearable device. Also, a plurality of terminal devices 10 may exist.

[0030] The terminal device 10 and the server 40 can communicate via the communication network N. The communication network N is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), or the like. The communication network N may include not only wired communication but also networks by wireless communication such as 3G (3rd Generation), WiMAX (Worldwide Interoperability for Microwave Access), and LTE (Long Term Evolution). Also, the terminal device 10 can communicate by means of short-range communication technologies such as Bluetooth (registered trademark) and NFC (Near Field Communication) (registered trademark).

[0031] Note that in FIG. 1, the server 40 and the terminal device 10 are communicating via the communication network N, but the user may directly operate the server 40 from the console, or the terminal device 10 may have the functions of the server 40. That is, the terminal device 10 may provide the functions of the information processing system 100 in a stand-alone form.

[0032] <Hardware Configuration Example> FIG. 2 is a hardware configuration diagram of the diagnostic apparatus and the management apparatus according to the present embodiment. Each hardware configuration of the terminal device 10 is the same, and is indicated by a reference numeral in the 100s as the terminal device 10. Each hardware configuration of the management apparatus is indicated by a reference numeral in the 400s.

[0033] Hereinafter, each hardware configuration of the terminal device 10 will be described. However, since each hardware configuration of the server 40 is the same, the description thereof will be omitted.

[0034] The terminal device 10 is constructed by a computer and includes, as shown in FIG. 2, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an HD (Hard Disk) 104, an HDD (Hard Disk Drive) controller 105, a display I / F 106, and a communication I / F 107.

[0035] Among these, the CPU 101 controls the operation of the entire terminal device 10. The ROM 102 stores programs used for driving the CPU 101 such as an IPL (Initial Program Loader). The RAM 103 is used as a work area for the CPU 101.

[0036] The HD 104 stores various data such as programs. The HDD controller 105 controls reading and writing of various data to and from the HD 10 in accordance with the control of the CPU 101.

[0037] The display I / F 106 is a circuit that causes the display 106a to display an image. The display 106a is a type of display unit such as a liquid crystal or an organic EL (Electro Luminescence) that displays various information such as a cursor, a menu, a window, characters, or an image. The communication I / F 107 is an interface used for communication with other devices.

[0038] When the terminal device 10 is a glass device, the terminal device 10 may use a circuit that displays an image on a lens or the like as a transmissive-reflective member instead of the display I / F 106.

[0039] The communication I / F 107 is, for example, a NIC (Network Interface Card) or the like corresponding to TCP (Transmission Control Protocol) / IP (Internet Protocol).

[0040] The terminal device 10 also includes a sensor I / F 108, an audio input / output I / F 109, an input I / F 110, a media I / F 111, and a DVD-RW (Digital Versatile Disk Rewritable) drive 112.

[0041] The sensor I / F 108 is an interface that receives detection information from various sensors. The audio input / output I / F 109 is a circuit that processes the input / output of audio signals between the speaker 109a and the microphone 109b according to the control of the CPU 101. The input I / F 110 is an interface for connecting a predetermined input means to the terminal device 10.

[0042] The keyboard 110a is a type of input means having a plurality of keys for inputting characters, numerical values, various instructions, etc. The mouse 110b is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, operating on a display screen, etc.

[0043] The media I / F 111 controls the reading or writing (storage) of data to / from a recording medium 111a such as a flash memory. The DVD-RW drive 112 controls the reading or writing of various data to / from a DVD-RW 112a as an example of a removable recording medium. Note that it is not limited to DVD-RW, and it may be DVD-R or the like. Also, the DVD-RW drive 112 may be a Blu-ray drive that controls the reading or writing of various data to / from a Blu-ray Disc (registered trademark).

[0044] Further, the terminal device 10 includes a bus line 113. The bus line 113 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 101.

[0045] Note that recording media such as an HD or a CD-ROM storing each of the above programs can all be provided as a program product, either domestically or abroad. The terminal device 10 realizes, for example, the information processing method according to the present invention when the program according to the present invention is executed.

[0046] <Regarding functions> FIG. 3 is an example of a functional block diagram of the server 40 and the terminal device 10 in the information processing system 100 according to the present embodiment.

[0047] <<Terminal device>> As shown in FIG. 3, the terminal device 10 includes a transmission / reception unit 11, an input reception unit 12, a display control unit 13, an audio control unit 14, a conversion unit 15, and a storage / readout unit 19. Each of these units is a function or a means realized by any of the components shown in FIG. 2 operating according to an instruction from the CPU 101 based on a program expanded from the HD 104 onto the RAM 103. Further, the terminal device 10 has a storage unit 2000 constructed by at least one of the RAM 103 and the HD 104 shown in FIG. 2.

[0048] The transmission / reception unit 11 is an example of a transmission means or a second transmission means, and is realized by an instruction from the CPU 101 shown in FIG. 2 and the communication I / F 107, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network N.

[0049] The input reception unit 12 is an example of input reception means, mainly realized by instructions from the CPU 101 shown in FIG. 2 and the input I / F 110 and audio input / output I / F 109, and receives various inputs by the user using the microphone 109b, keyboard 110a, and mouse 110b.

[0050] The display control unit 13 is an example of display control means and output means, realized by instructions from the CPU 101 shown in FIG. 2 and the display I / F 106, and causes the display 106a, which is an example of a display unit, to display various images and screens. When the terminal device 10 is a glass device, the display control unit 13 causes a virtual image to be displayed on a transmissive and reflective member such as a lens instead of the display I / F 106.

[0051] The audio control unit 14 is an example of audio control means and output means, realized by instructions from the CPU 101 shown in FIG. 2 and the audio input / output I / F 109, and causes the speaker 109a, which is an example of an audio playback unit, to play sound.

[0052] The conversion unit 15 is an example of processing means, realized by instructions from the CPU 101 shown in FIG. 2, and performs processing for converting character information into audio information and processing for converting audio information into character information.

[0053] The storage / reading unit 19 is an example of storage control means, executed by instructions from the CPU 101 shown in FIG. 2 and the HD 104, media I / F 111, and DVD-RW drive 112, and performs processing for storing various data in the storage unit 2000, recording medium 111a, and DVD-RW 112a, and reading various data from the storage unit 2000, recording medium 111a, and DVD-RW 112a.

[0054] <Functional Configuration of Server> Server 40 has a transmission / reception unit 41, a screen generation unit 42, a determination unit 43, an identification unit 44, a text information generation unit 45, an update unit 46, an attribute relevance calculation unit 47, and a storage / reading unit 49. Each of these units is a function or means realized by any of the components shown in FIG. 2 operating according to instructions from CPU 401 developed on RAM 403 from HD 404. Also, server 40 has a storage unit 4000 constructed by HD 404 shown in FIG. 2. Storage unit 4000 is an example of storage means.

[0055] Server 40 may be configured to distribute and realize each function among a plurality of computers. Furthermore, although server 40 is described as a server computer existing in a cloud environment, it may also be a server existing in an on-premises environment.

[0056] Transmission / reception unit 41 is an example of transmission means or first transmission means, and is realized by instructions from CPU 401 shown in FIG. 2 and communication I / F 407, and transmits and receives various data (or information) with other terminals, devices, or systems via communication network N.

[0057] Screen generation unit 42 is an example of screen generation means, and is realized by instructions from CPU 401 shown in FIG. 2, and generates various screens. When terminal device 10 executes a web app, the screen information is created by HTML, XML, CSS (Cascade Style Sheet), JavaScript, etc. Therefore, the screen information may be referred to as a web app. When terminal device 10 executes a client app, the screen information is held by terminal device 10, and the information to be displayed is transmitted in XML or the like.

[0058] Determination unit 43 is an example of determination means, and is realized by instructions from CPU 401 shown in FIG. 2, and makes various determinations described later.

[0059] The specific unit 44 is an example of specific means, which is realized by an instruction from the CPU 401 shown in FIG. 2 and performs identification of the target image.

[0060] The text information generation unit 45 is an example of text information generation means, which is realized by an instruction from the CPU 401 shown in FIG. 2 and generates text information.

[0061] The update unit 46 is an example of update means, which is realized by an instruction from the CPU 401 shown in FIG. 2 and updates the model described later.

[0062] The attribute relevance calculation unit 47 is an example of attribute relevance calculation means, which is realized by an instruction from the CPU 401 shown in FIG. 2 and calculates the attribute relevance between the attribute information of the user and the attribute information of the data provider, or between the attribute information of the user and the attribute information of past users.

[0063] The storage / reading unit 49 is an example of storage control means, which is executed by an instruction from the CPU 401 shown in FIG. 2, as well as the HD 404, the media I / F 411, and the DVD-RW drive 412, and performs processes of storing various data in the storage unit 4000, the recording medium 411a, and the DVD-RW 412a, or reading out various data from the storage unit 4000, the recording medium 411a, and the DVD-RW 412a. The storage unit 4000, the recording medium 411a, and the DVD-RW 412a are examples of storage means.

[0064] In the storage unit 4000, a user information management DB 4001 constituted by a user information management table, an image management DB 4002 constituted by an image information management table, a caption model 4003, a tacit knowledge model 4004, and a large language model 4005 are constructed.

[0065] The user information management DB 4001 stores and manages various information related to the user such as the attribute information of the user, and the image management DB 4002 stores and manages various images such as three-dimensional images related to three-dimensional point clouds and three-dimensional models, omnidirectional shooting images, etc.

[0066] The caption model 4003 is generated by executing a learning process using a combination of an image and a caption comment (details will be described later) as learning data, and is a model that enables a computer to function so as to output a caption comment based on the image. Here, the caption comment is text data and is a comment that explains the image among comments indicated by voice or characters.

[0067] The tacit knowledge model 4004 is generated by executing a learning process using a combination of an image and a tacit knowledge comment for the image as learning data, and is a model that enables a computer to function so as to output a tacit knowledge comment based on the image. Here, the tacit knowledge comment is text data and is a comment that excludes the caption comment among comments indicated by voice or characters, that is, a comment related to content not expressed in the image.

[0068] The large language model 4005 is a computer language model composed of an artificial neural network having a large number of parameters, which is generated by executing a learning process using a huge amount of unlabeled text as learning data. The large language model 4005 can capture much of the syntax and meaning of human language by being sufficiently trained by a method for learning context, such as next sentence prediction for understanding context by determining whether sentence 1 and sentence 2 are consecutive, or masked language model for understanding context by masking a word in a sentence and predicting the masked word from the words before and after it.

[0069] <<User Information Management Table>> FIG. 4(a) is a conceptual diagram showing an example of a user information management table according to the present embodiment. In the storage unit 4000, a user information management DB 4001 configured by a user information management table as shown in FIG. 4(a) is constructed. In this user information management table, for each user ID, attribute information indicating the attributes of the user and the presence or absence of the usage right of the text information generation unit 45 are associated and managed.

[0070] The user ID is identification information for identifying the user. The user is a person who uses the text information generated by the tacit knowledge model 4004, and the identification information of the data provider is also managed as the user ID.

[0071] The attribute information is, as an example, company name, name, age, affiliation, business content, length of service, remarks, but is only an example. For example, the attribute information may include attributes such as process (the business content may be regarded as a process), qualification (manager, operator, business consignor, skilled person, unskilled person, etc.), industry, specialized field, business experience, position, etc.

[0072] <<Image Management Table>> FIG. 4(b) is a conceptual diagram showing an example of an image management table according to the present embodiment. In the storage unit 4000, an image management DB 4002 configured by an image management table as shown in FIG. 4(b) is constructed.

[0073] In this image management table, for each image ID, image information such as a three-dimensional image and an omnidirectional image, registration information regarding the registration scene in which the data provider registered learning data such as the image information, and usage information regarding the usage scene in which the user specified the image information and used the information processing system 100 are associated and managed.

[0074] The registration information includes the date of registration of the image information, the name of the property indicated by the image information, the process name indicating which process the image information relates to, etc. The process may be referred to as a work. The process may also be called a phase in construction work, and the processes include site investigation, design, construction, maintenance management, safety patrol, etc. These are processes premised on the construction industry, and the types of processes may vary by industry or may be further divided into more detailed processes. Also, the user ID of the person who provided the image information (the user ID of the data provider) is included in the registration information. The person who provided the image information is the "data provider" described later.

[0075] The usage information includes the date of using the information processing system 100 by specifying the image information, the name of the property using the image information, etc., the user ID of the user who used the information processing system 100, the process name using the image information, etc. For one image ID, multiple usage information is managed in the image management table. Also, the usage information includes the input type and examples of specific input sentences. The input type is question, summary, generation, translation, conversion, etc. The input sentence for question is "Please tell me ~", the input sentence for summary is "Please summarize ~", the input sentence for generation is "Please come up with ideas about ~" or "Please create source code", the input sentence for translation is "Please translate ~ into ~ language", and the input sentence for conversion is "Please change the image ~ to be like ~". When the input type and specific input sentences are presented to the user, a specific image of how to use the information processing system 100 comes to mind.

[0076] <Operation or process> <<Model update>> First, referring to FIG. 5, the model update process in which the tacit knowledge model 4004 learns data will be described. FIG. 5 is a sequence diagram showing an example of the model update process according to the present embodiment.

[0077] The input reception unit 12 of the terminal device 10 receives input operations related to the user ID, image ID, and registration information of the data provider for the input / output screen displayed on the display 106a (step S21). Further, the input reception unit 12 receives an input operation related to the attribute information of the data provider for an attribute information registration screen (described later) displayed on the display 106a. Note that this image ID identifies an image that is part of the data used for learning. Although it is described that the image itself is registered in the image management DB4002 in advance, the image may be transmitted to the server 40 in step S21.

[0078] The transmission / reception unit 11 transmits the user ID, image ID, registration information, and the attribute information of the data provider received in step S21 to the server 40, and the transmission / reception unit 41 of the server 40 receives the user ID, image ID, registration information, and the attribute information of the data provider transmitted from the terminal device 10 (step S22).

[0079] Next, the storage / reading unit 49 of the server 40 stores the registration information received in step S22 in the image management DB4002 in association with the image ID, and reads out the image information corresponding to the image ID by searching the image management DB4002 using the image ID received in step S22 as a search key (step S23). Further, the storage / reading unit 49 stores the attribute information of the data provider received in step S2 in the user information management DB4001.

[0080] The screen generation unit 42 generates a display screen including the image information read by the storage / reading unit 49 (step S24).

[0081] Here, in step S21, the data provider inputs a plurality of image IDs, and in step S23, by searching the image management DB4002 using the plurality of image IDs as search keys, the storage / reading unit 49 can read out a plurality of image information.

[0082] Also, in step S21, instead of the image ID, a document ID that identifies a document including a desired image may be input, and the image management DB4002 may manage one or more pieces of image information associated with each document ID.

[0083] Accordingly, in step S22, the transceiver unit 11 transmits the document ID instead of the image ID, and in step S23, by searching the image management DB4002 using the document ID as a search key, one or more pieces of image information corresponding to the document ID can be read out.

[0084] The transceiver unit 41 transmits display screen information related to the display screen generated in step S24 to the terminal device 10, and the transceiver unit 11 of the terminal device 10 receives the display screen information transmitted from the server 40 (step S25). As another form, instead of steps S21 to S25, the transceiver unit 11 of the terminal device 10 may receive information indicating a captured image transmitted from the imaging device as display screen information.

[0085] The display control unit 13 of the terminal device 10 causes the display screen to be displayed on the display 106a based on the display screen information received in step S25 (step S26). As another form, instead of the display screen received in step S25, the display control unit 13 may cause an image managed by the terminal device 10 to be displayed on the display 106a as the display screen.

[0086] The input reception unit 12 of the terminal device 10 receives input information input by the data provider with respect to the displayed display screen (step S27). A plurality of data providers may input input information.

[0087] This input information includes voice information, character information, and operation information input by the data provider, and the conversion unit 15 converts the voice information included in the input information into character information. The voice information, character information, and operation information include specific information that identifies the target image among the display screens. Also, the voice information and character information include a caption comment that explains the target image and a tacit knowledge comment related to content not represented in the target image.

[0088] The transmission / reception unit 11 transmits the input information related to the input operation received by the input reception unit 12 to the server 40, and the transmission / reception unit 41 of the server 40 receives the input information transmitted from the terminal device 10 (step S28). As another form, the transmission / reception unit 11 may transmit, to the server 40, the input information together with the captured image or the display screen information using the image managed by the terminal device 10 as the display screen, and the transmission / reception unit 41 of the server 40 may receive the input information transmitted from the terminal device 10 and the display screen information.

[0089] The specifying unit 44 specifies the target image in the display screen based on the specifying information included in the input information received in step S28 (step S29).

[0090] The determination unit 43 uses the target image specified in step S29 to obtain a caption comment based on the caption model 4003, and determines the relevance (a relevance different from the attribute relevance) between the caption comment and the comment included in the input information received in step S28 (step S30).

[0091] The determination unit 43 may determine the relevance between the obtained caption comment and the entire comment included in the input information received in step S28, or may divide the comment included in the input information received in step S28 into a plurality of parts and determine the relevance between the obtained caption comment and each divided comment.

[0092] The update unit 46 updates the caption model 4003 as learning data together with the target image specified in step S29 using the comment determined to have a high relevance in step S30 as the caption comment, and updates the tacit knowledge model 4004 as learning data together with the target image specified in step S29 using the comment determined to have a low relevance in step S30 as the tacit knowledge comment (step S31).

[0093] In step S31, the storage / reading unit 49 searches the user information management DB4001 using the user ID received in step S22 as a search key, reads out the attribute information of the data provider associated with the user ID, and the update unit 46 updates the tacit knowledge model 4004 as learning data by including the attribute information of the data provider and the registration information related to the target image in the tacit knowledge comment.

[0094] In the above, when the information processing system 100 includes a plurality of terminal devices 10, steps S21 and S22, steps S25 and S26, and steps S27 and S28 may be executed by different terminal devices respectively, may be executed by a plurality of terminal devices respectively, and the terminal device 10 shown in FIG. 5 may be different from the terminal device shown in FIG. 6.

[0095] Also, the functions of the server 40 in FIG. 3 may be integrated into the terminal device 10, and the terminal device 10 may execute the processing of the server 40 in FIG. 5.

[0096] <<Use of the information processing system>> Next, with reference to FIG. 6, the processing for a user to use the information processing system 100 will be described. FIG. 6 is a sequence diagram showing an example of text information generation processing according to the present embodiment.

[0097] First, the input reception unit 12 of the terminal device 10 receives an input operation related to the user ID, image ID, and usage information of the user with respect to the input / output screen displayed on the display 106a (step S1). Also, the input reception unit 12 may receive an input operation related to the attribute information of the user with respect to an attribute information registration screen (described later) displayed on the display 106a, or the attribute information of the user may be registered in the user information management DB4001 in advance.

[0098] The transmission / reception unit 11 transmits the user ID, image ID, usage information, and the user's attribute information received in step S1 to the server 40, and the transmission / reception unit 41 of the server 40 receives the user ID, image ID, usage information, and the user's attribute information transmitted from the terminal device 10 (step S2).

[0099] Next, the storage / reading unit 49 of the server 40 stores the usage information received in step S2 in the image management DB 4002 in association with the image ID, and searches the image management DB 4002 using the image ID received in step S2 as a search key to read out the image information, registration information, and usage information corresponding to the image ID (step S3). Also, the storage / reading unit 49 stores the user's attribute information received in step S2 in the user information management DB 4001.

[0100] The screen generation unit 42 generates a display screen including the image information read by the storage / reading unit 49 (step S4). Also, the attribute relevance calculation unit 47 calculates the attribute relevance between the current user's attribute information and the data provider's attribute information read by the storage / reading unit 49 from the user information management DB 4001. The screen generation unit 42 generates an image including the attribute information of the data provider with the highest attribute relevance. Also, the attribute relevance calculation unit 47 may calculate the attribute relevance between the current user's attribute information and the past user's attribute information read by the storage / reading unit 49 from the user information management DB 4001. The screen generation unit 42 generates an image including the attribute information of the past user with the highest attribute relevance.

[0101] Here, in step S1, a plurality of image IDs are input, and in step S3, by searching the image management DB 4002 using the plurality of image IDs as search keys, a plurality of image information can be read out.

[0102] Also, in step S1, instead of the image ID, a document ID for identifying a document including a desired image may be input, and the image management DB 4002 may manage one or more pieces of image information associated with each document ID.

[0103] As a result, in step S2, the transmission / reception unit 11 transmits the document ID instead of the image ID, and in step S3, by searching the image management DB 4002 using the document ID as a search key, one or more pieces of image information corresponding to the document ID can be read out.

[0104] The transmission / reception unit 41 transmits the display screen information related to the display screen generated in step S4 to the terminal device 10, and the transmission / reception unit 11 of the terminal device 10 receives the display screen information transmitted from the server 40 (step S5). As another form, instead of steps S1 to S5, the transmission / reception unit 11 of the terminal device 10 may receive information indicating the captured image transmitted from the imaging device as the display screen information.

[0105] The display control unit 13 of the terminal device 10 causes the display screen to be displayed on the display 106a based on the display screen information received in step S5 (step S6). As another form, instead of the display screen received in step S5, the display control unit 13 may cause an image managed by the terminal device 10 to be displayed on the display 106a as the display screen. In step S6, as one of the display screens, a question-and-answer screen described later is displayed, and the attribute information of the data provider or past user with the highest attribute relevance to the user's attribute information can be displayed.

[0106] The input reception unit 12 of the terminal device 10 receives the input information input by the user with respect to the displayed display screen (step S7).

[0107] This input information includes voice information, character information, and operation information input by the user, and the conversion unit 15 may convert the voice information included in the input information into character information. The voice information, character information, and operation information include specific information for specifying the target image among the display screens and questions regarding the target image.

[0108] In addition, the input information may include one or more pieces of attribute information of the data provider or past users. For example, the user can view the attribute information of the data provider or past users with the highest degree of attribute relevance to the user's own attribute information, and input questions such as "want an answer centered on the tacit knowledge comments of data providers in their 30s" that include one or more of those attributes.

[0109] The transmission / reception unit 11 transmits the input information received by the input reception unit 12 to the server 40, and the transmission / reception unit 41 of the server 40 receives the input information transmitted from the terminal device 10 (step S8). As another form, the transmission / reception unit 11 may transmit, to the server 40, the input information together with display screen information having a captured image or an image managed by the terminal device 10 as a display screen, and the transmission / reception unit 41 of the server 40 may receive the input information transmitted from the terminal device 10 and the display screen information.

[0110] The storage / reading unit 49 searches the user information management DB 4001 using the user ID received in step S2 as a search key, reads out the attribute information and usage rights of the user associated with the user ID, and the determination unit 43 determines whether the text information generation unit 45 has the usage rights based on the usage rights read from the user information management DB 4001 (step S9).

[0111] If it is determined in step S9 that there are usage rights, the specifying unit 44 specifies the target image in the display screen based on the specific information included in the input information received in step S8 (step S10).

[0112] The text information generation unit 45 uses the target image specified in step S10 to obtain a tacit knowledge comment based on the tacit knowledge model 4004, and generates text information based on the large language model 4005 using the tacit knowledge comment and the question extracted from the input information received in step S8 (step S11). The large language model 4005 can generate a natural conversation from the tacit knowledge comment and the question.

[0113] The text information generation unit 45 may convert the voice information included in the input information into character information, and the text information generated by the text information generation unit 45 may be either voice information or character information.

[0114] The text information generation unit 45 may generate text information without using a question, or may generate a fixed question inside the system and use the fixed question. In this case, the question sentence is not visible to the user. Alternatively, the text information generation unit 45 may generate a fixed question inside the system, display the fixed question on the display unit for the user to select, and use the selected question.

[0115] The transmission / reception unit 41 transmits the text information generated in step S11 to the terminal device 10, and the transmission / reception unit 11 of the terminal device 10 receives the text information transmitted from the server 40 (step S12).

[0116] When the received text information is character information, the display control unit 13 of the terminal device 10 causes the text information to be displayed on the display 106a, or the conversion unit 15 converts the received text information into voice information, and the voice control unit 14 causes the converted text information to be reproduced by the speaker 109a (step S13). Also, when the received text information is voice information, the text information is reproduced by the speaker 109a, or the conversion unit 15 converts the received text information into character information, and the converted text information is displayed on the display 106a.

[0117] In the above, when the information processing system 100 includes a plurality of terminal devices 10, steps S1 and S2, steps S5 and S6, steps S7 and S8, steps S12 and S13 may be executed by different terminal devices, respectively, or may be executed by a plurality of terminal devices, respectively.

[0118] Also, the functions of the server 40 in FIG. 3 may be integrated into the terminal device 10, and the terminal device 10 may execute the processing of the server 40 in FIG. 6.

[0119] <<Detailed processing>> FIG. 7 is a flowchart showing an example of various processes according to the present embodiment. FIG. 7(a) shows the processes corresponding to steps S30 and S31 in FIG. 5.

[0120] The determination unit 43 determines the degree of relevance between the caption comment obtained from the caption model 4003 using the target image and the comment included in the input information received from the terminal device 10 (step S51). For example, the determination unit 43 determines that the degree of relevance is high with the caption comment when the ratio of the comment included in the input information that matches the caption comment is equal to or higher than a predetermined value. Here, a high degree of relevance with the caption comment means that the probability that the comment describes the content of the target image is high.

[0121] The determination unit 43 may determine the degree of relevance with the obtained caption comment for the entire comment included in the input information received from the terminal device 10, or may divide the comment included in the input information received from the terminal device 10 into a plurality of parts and determine the degree of relevance with the obtained caption comment for each divided comment.

[0122] For example, after the determination unit 43 determines the object in the target image by image recognition or object detection, the determination unit 43 may determine as the degree of relevance whether a word related to the caption comment representing the object is included in the comment included in the input information, or the ratio of the number of included words.

[0123] The update unit 46 updates the tacit knowledge model 4004 as learning data together with the target image, the registration information and usage information related to the target image, and the attribute information of the user, using the comment determined to have a low degree of relevance in step S51 as the tacit knowledge comment (step S52).

[0124] The update unit 46 updates the caption model 4003 as learning data together with the target image, using the comment determined to have a high degree of relevance in step S51 as the caption comment (step S53).

[0125] Here, the update of the tacit knowledge model 4004 and the caption model 4003 may be optional. That is, when a predetermined condition is satisfied, the update unit 46 may execute step S52 to update the tacit knowledge model 4004, or execute step S53 to update the caption model 4003.

[0126] FIG. 7(b) shows the processing corresponding to step S10 in FIG. 6 and step S29 in FIG. 5.

[0127] The determination unit 43 determines whether the operation information included in the input information received from the terminal device 10 includes an operation for specifying the target image (step S41). If the operation for specifying is included, the specifying unit 44 specifies the target image according to the operation for specifying (step S42).

[0128] The determination unit 43 determines whether the voice information or character information included in the input information received from the terminal device 10 includes a comment for specifying the target image (step S43). If the comment for specifying is included, the specifying unit 44 specifies the target image according to the comment for specifying (step S44). The comment for specifying the target image is, for example, a comment for specifying a position in the display screen such as right or left, or a comment for specifying the registration information of the image.

[0129] If the operation information does not include an operation for specifying the target image and the voice information and character information do not include a comment for specifying the target image, the specifying unit 44 specifies the entire display screen as the target image (step S45).

[0130] In step S42, when the operation information includes an operation for specifying each of a plurality of target images, the specifying unit 44 may specify each of the plurality of target images according to each operation.

[0131] In step S44, when the specific part 44 includes a comment that specifies each of a plurality of target images in the voice information or the character information, each of the plurality of target images may be specified according to each comment.

[0132] Further, when the specific part 44 includes an operation for specifying a target image in the operation information and includes a comment for specifying another target image in the voice information or the character information, the target images may be respectively specified according to each of the operation and the comment.

[0133] Furthermore, in step S45, when the display screen includes text information, the specific part 44 may specify, as the target image, a part of the display screen excluding the text information.

[0134] FIG. 7(c) shows the process corresponding to step S11 in FIG. 6.

[0135] The text information generation unit 45 uses the target image to obtain an implicit knowledge comment based on the implicit knowledge model 4004 (step S61), and uses the implicit knowledge comment, the question extracted from the input information received in step S8, the registration information and usage information read in step S3 related to the target image, and the user attribute information read in step S9 to generate text information based on the large language model 4005 (step S62).

[0136] FIG. 8 shows the process of displaying the attribute information of the data provider with the highest degree of attribute relevance to the user's attribute information. The process of FIG. 8 can be implemented in step S4 of FIG. 6, and may also be appropriately implemented according to an operation from the user.

[0137] The attribute relevance calculation unit 47 acquires the attribute information of the data provider from the user information management DB 4001 (step S71). The user ID of the data provider is registered in the registration information of the image management DB 4002. This data provider may be all the data providers included in the registration information of the image management DB 4002, or may be the data providers specified by the plurality of image IDs when the user specifies a plurality of image IDs in step S1 of FIG. 6.

[0138] Next, the attribute relevance calculation unit 47 calculates the attribute relevance between the attribute information of the currently used user and the attribute information of the data provider for each data provider (step S72). There are the following methods for calculating the attribute relevance. Note that it is not necessary to limit to the following as long as it is a method that can measure the similarity between texts.

[0139] 1. Judgment is made based on the number of matching words included in the two pieces of attribute information to be compared (comparing one item of one piece of attribute information with all items of the other piece of attribute information is performed for each item of one piece of attribute information) 2. For each item of the attribute information, the words included in the user's attribute information are compared with the words included in the data provider's attribute information, and judgment is made based on the number of matching words

[0140] In the method of 1 or 2, it is determined that the larger the number of matches, the higher the relevance. Also, in the method of 1 or 2, the more information is described in the "Remarks" of the attribute information, the higher the measurement accuracy of the relevance. Note that the attribute information is split into words by morphological analysis as necessary.

[0141] 3. In natural language processing, words and sentences are often embedded in a multi-dimensional space for use such as expanding the amount of information (vector conversion). There are methods such as BERT (Bidirectional Encoder Representations from Transformers) and word2vec for vector conversion in natural language processing. Therefore, the attribute relevance calculation unit 47 may generate two vectors converted from the attribute information in the vector space instead of the number of matching words, and judge the relevance based on the vector similarity (distance) or cosine similarity.

[0142] The attribute relevance calculation unit 47 determines the data provider with the highest attribute relevance (step S73). The attribute information of the data provider with the highest attribute relevance is transmitted to the terminal device 10.

[0143] <Example of the screen displayed by the terminal device> FIG. 9 is an explanatory diagram displayed by the terminal device 10 in the text information generation process and the model update process according to the present embodiment. FIG. 9(a) is an explanatory diagram of the model update process corresponding to steps S26 and S27 in FIG. 5 and FIGS. 7(a) and 7(b).

[0144] The display control unit 13 of the terminal device 10 causes the display screen 1000 received from the server 40 to be displayed on the display 106a, and the display screen 1000 includes the target image 1100 and the text 1200.

[0145] The input reception unit 12 of the terminal device 10 receives, from the microphone 109b, voice information indicating the dialogues Q1, A1, Q2, and A2 between the data provider M1 and the data provider M2 as input information input by the data provider with respect to the displayed display screen 1000. It is preferable that the data providers M1 and M2 have rich knowledge including tacit knowledge about the business. By updating the tacit knowledge model 4004 through the dialogues of such data providers M1 and M2, the user can obtain useful tacit knowledge comments.

[0146] The specifying unit 44 specifies the target image 1100, which is the part of the display screen 1000 excluding the text 1200.

[0147] Then, the determination unit 43 determines the relevance between the caption comment obtained from the caption model 4003 using the target image 1100 and the dialogues Q1, A1, Q2, and A2.

[0148] The update unit 46 updates the tacit knowledge model 4004 as learning data together with the target image 1100, etc., using the comments determined to have low relevance among the dialogues Q1, A1, Q2, and A2 as tacit knowledge comments, and updates the caption model 4003 as learning data together with the target image 1100, using the comments determined to have high relevance as caption comments.

[0149] FIG. 9(b) is an explanatory diagram of the text information generation process corresponding to steps S6, S7, S13 in FIG. 6 and FIGS. 7(b) and 7(c).

[0150] The display control unit 13 of the terminal device 10 causes the display screen 1000 received from the server 40 to be displayed on the display 106a, and the display screen 1000 includes an image 1110 and text 1210.

[0151] The input reception unit 12 of the terminal device 10 receives, as input information input by the user with respect to the displayed display screen 1000, voice information indicating questions Q11 and Q12 by the user M3 via the microphone 109b.

[0152] The specifying unit 44 specifies an image 1110 that does not include the text 1210 as the target image.

[0153] The text information generation unit 45 uses the image 1110 to obtain tacit knowledge comments based on the tacit knowledge model 4004, and uses the tacit knowledge comments and questions Q11, Q12, etc. to generate text information related to answers A11 and A12 for each of the questions Q11 and Q12 based on the large language model 4005.

[0154] The display control unit 13 of the terminal device 10 causes the text information related to the answers A11 and A12 received from the server 40 to be displayed on the display 106a.

[0155] FIG. 10 is another explanatory diagram displayed by the terminal device 10 in the model update process and the text information generation process according to the present embodiment.

[0156] FIG. 10(a) is an explanatory diagram of the model update process corresponding to steps S26 and S27 in FIG. 5 and FIGS. 7(a) and (b). FIG. 10(a) shows an example in which the tacit knowledge model 4004 is updated with voice information and partial images of one data provider, rather than conversations between data providers.

[0157] The display control unit 13 of the terminal device 10 causes the display screen 1000 received from the server 40 to be displayed on the display 106a, and the display screen 1000 includes a first image 1100A and a second image 1100B.

[0158] The input receiving unit 12 of the terminal device 10 receives, from the keyboard 110a, character information indicating comments C1 to C4 by the data provider M4 as input information input by the data provider with respect to the displayed display screen 1000.

[0159] Also, the input receiving unit 12 receives, from the mouse 110b, operation information indicating an operation for specifying a partial image 1100B1 in the second image 1100B by the data provider M4 as input information input by the data provider M4 with respect to the displayed display screen 1000.

[0160] The specifying unit 44 may specify the partial image 1100B1 as the target image, or may specify the first image 1100A or the second image 1100B as the target image.

[0161] Then, the determination unit 43 determines the relevance between the caption comment obtained from the caption model 4003 using the target image and the comments C1 to C4.

[0162] The update unit 46 updates the tacit knowledge model 4004 as learning data together with the partial image 1100B1, etc., with the comments determined to have low relevance among the comments C1 to C4 as tacit knowledge comments, and updates the caption model 4003 as learning data together with the partial image 1100B1 with the comments determined to have high relevance as caption comments.

[0163] FIG. 10(b) is an explanatory diagram of a text information generation process corresponding to steps S6, S7, S13 in FIG. 6 and FIGS. 7(b) and (c).

[0164] The display control unit 13 of the terminal device 10 causes the display screen 1000 received from the server 40 to be displayed on the display 106a, and the display screen 1000 includes an image 1110.

[0165] User M5 does not perform any input on the displayed display screen 1000, and the input reception unit 12 does not receive the input information that the user inputs with respect to the displayed display screen 1000. The specifying unit 44 specifies the entire displayed display screen 1000, the image 1110, as the target image.

[0166] When user M5 performs an operation to specify the partial image 1100B1 in FIG. 10(a) on the display screen 1000, the input reception unit 12 receives, as input information, operation information indicating the operation to specify the partial image from the mouse 110b. In this case, the specifying unit 44 specifies the partial image of the display screen 1000 as the target image according to the operation information.

[0167] The text information generation unit 45 uses the image 1110 to obtain tacit knowledge comments based on the tacit knowledge model 4004, and uses the tacit knowledge comments and the like to generate text information related to comments C11 to C14 based on the large language model 4005. The text information generation unit 45 may generate text information using preset stereotyped questions.

[0168] The display control unit 13 of the terminal device 10 causes the display 106a to display the text information related to the comments C11 to C14 received from the server 40.

[0169] <Present the information of the provider with the highest attribute relevance among the data providers> Regarding the process in which the server 40 transmits the attribute information of the data provider with the highest attribute relevance to the terminal device 10, several screens will be described. By displaying the attribute information of the data provider with the highest attribute relevance, the current user can refer to the registration information provided by the data provider with attributes similar to his own when using the server 40, which leads to the promotion of the use of text information.

[0170] First, referring to FIG. 11, the registration of user attribute information will be described. FIG. 11(a) shows a question-and-answer screen 210 displayed on the terminal device 10. In order for the server 40 to calculate the attribute relevance degree, the user needs to register the attribute information with the server 40 in advance. The question-and-answer screen 210 has a user information registration icon 211, a question-and-answer display tab 212, and an input information input field 213. Note that the question-and-answer screen 210 in FIG. 11 is displayed, for example, in step S1 of FIG. 6. The attribute information of the data provider or the user may be registered with the server 40 in advance.

[0171] When the user presses (clicks or taps) the user information registration icon 211, an attribute information registration screen 220 will be pop-up displayed. FIG. 11(b) shows the pop-up displayed attribute information registration screen 220. The attribute information registration screen 220 is a screen for registering the user's attribute information. As an example, the attribute information registration screen 220 has a company name column 221, a name column 222, an age column 223, an affiliation column 224, a business content column 225, a continuous service years column 226, and a remarks column 227. The user inputs the attribute information into these input fields and presses the determination button 228. The user's attribute information is sent to the server 40, and the storage / reading unit 49 stores it in the user information management DB 4001. Similarly, the data provider also registers the attribute information of the data provider in step S21 of FIG. 5 or in advance.

[0172] Although the figures are created on the premise that there is a relevance degree, the input of attribute information is optional, and it is not the case that the information processing system 100 cannot be used if the user does not input the attribute information. If the user does not input the attribute information, since the attribute relevance degree between the user and the data provider cannot be calculated, for example, the information of a randomly selected data provider will be displayed. The same applies to the following embodiments.

[0173] Next, referring to FIG. 12, the attribute information of the data provider with a high degree of attribute relevance, which is displayed in step S6 of FIG. 6, will be described. FIG. 12 shows a question-and-answer screen 230 displayed when the user uses the information processing system 100. FIG. 12(a) is a question-and-answer screen 230 where input information can be input. The question-and-answer screen 230 has an input information input field 213. In the input information input field 213, the user can display any input information. The question-and-answer screen 230 in FIG. 12(a) has a data provider confirmation tab 231. When the user presses the data provider confirmation tab 231, the attribute information of the data provider with the highest degree of attribute relevance to the user is displayed.

[0174] FIG. 12(b) shows the question-and-answer screen 230 in which a data provider confirmation column 232 is displayed by pressing the data provider confirmation tab 231. In FIG. 12(b), the company name 233, name 234, age 235, affiliation 236, business content 237, years of continuous service 238, and remarks 239 of the data provider with the highest degree of attribute relevance to the user are displayed.

[0175] In addition, an icon 240 representing the provided data is displayed in the data provider confirmation column 232. When the user presses the icon 240, the provided data provided by the data provider with the highest degree of attribute relevance to the user is displayed. The provided data may include, for example, registration information, image information, and input information (voice information, character information, and operation information) at the time of registration of the registration information. In this way, since the user can confirm the attribute information and provided data of the data provider with the highest degree of attribute relevance, it can be used as a reference for how to utilize the text information generated by the tacit knowledge model 4004, to what extent it can be trusted, etc., which leads to the promotion of the use of the tacit knowledge model 4004.

[0176] In addition, the remarks column 239 may include various pieces of information input by the data provider. The data provider can input, for example, information regarding what provided data is in the remarks column 227. For example, since the remarks 239 are like "Thanks to the information processing system, I was able to master the know-how of the ○○ process", users who are not familiar with the tacit knowledge model can confirm how to utilize the information processing system 100.

[0177] Note that each piece of attribute information and provided data in FIG. 12(b) may be listed and displayed in the data provider confirmation column 232, or only the pressed ones among each piece of information may be displayed on a separate screen so that the user can check the details. The provided data may be the original data provided by the data provider itself or data that has been processed in some way. Known processing includes deleting confidential information, personal names, phone numbers, and the like. Generally, providing the original data itself is not recommended from the viewpoints of security and privacy.

[0178] <Main effects> According to this embodiment, the user can refer to, for example, the reliability of the tacit knowledge model that the user is about to use by seeing what attributes a data provider having attributes close to the user's own has, or what provided data the data provider is providing. Therefore, it can lead to the promotion of utilization.

[0179] [Second Embodiment] In the first embodiment, the attribute information of the data provider with the highest degree of attribute relevance to the user's attribute information was presented. In this embodiment, among the data providers, statistical information created based on the attribute information of the top N data providers with a high degree of relevance is presented. Since the user can refer to the statistical information of a plurality of data providers, the user can grasp the tendency of the learning data used by the tacit knowledge model.

[0180] Figure 13(a) shows a question-and-answer screen 250 in which a statistical information column 252 is displayed by pressing a statistical information tab 251. The question-and-answer screen 250 in Figure 13(a) has transitioned from the question-and-answer screen 230 in Figure 12(a) and is assumed to be displayed by pressing the statistical information tab 251. The statistical information column 252 has a number input column 253, an attribute selection column 254, and a process selection column 255. · The number input column 253 is a column where the user sets how many of the top data providers with a high degree of attribute relevance are to be extracted. · In the attribute selection column 254, the items of the attribute information of the data provider are displayed together with check boxes. The user selects the attributes of the data provider having the attributes for which statistical information is to be displayed using the check boxes. · In the process selection column 255, the items of the process are displayed together with check boxes. The user selects the process to be displayed as statistical information using the check boxes.

[0181] When the user presses the execute button 257, a pie chart 256 (an example of statistical information) is displayed. The pie chart 256 shows the age distribution of 100 people whose process is related to "construction". In this way, the user can set the AND conditions for the items of attributes and processes. By changing the number of people referred to by the user in the number input column 253, the pie chart 256 also changes.

[0182] Also, when multiple attributes are selected in the attribute selection column 254, the pie chart 256 is also displayed as many times as the number of selections. Also, the graph is not limited to a pie chart, and a bar graph, a radar chart, etc. may be used.

[0183] Also, the statistical information may be displayed as a word cloud. A word cloud is a graph that visualizes by changing the size and color of characters according to the frequency of appearance of words extracted from a text or the like. For example, in a word cloud, words with a higher frequency of appearance are displayed in the center and in larger characters. In the example of Figure 13(a), the characters from "teens" to "seventies" are displayed as a word cloud.

[0184] In addition, rankings may be displayed in the statistical information. For example, in the example of FIG. 13(a), the ranking is displayed as "1st place: 50s, 2nd place: 40s, 3rd place: 60s...". Also, in the ranking of the number of users by job position, the site supervisors, site workers, call center employees, etc. are displayed in the order of the number of users.

[0185] When the user presses a category of statistical information (e.g., 40s) in the pie chart 256, it is sent to the server 40 that one category in the graph has been selected, and the provided data provided by the data providers included in that category is displayed.

[0186] FIG. 13(b) is a question-and-answer screen 260 displayed by pressing a category. The question-and-answer screen 260 in FIG. 13(b) has a data list column 261. In the data list column 261, a list 262 of the provided data provided by the data providers of the selected category (e.g., 40s) is displayed. This provided data is the provided data provided by the data providers in their 40s about the construction process. When the user presses any provided data, it is sent to the server 40 that the provided data has been selected, and the details of the provided data are displayed. The details of the provided data may be the same as those in FIG. 12(b).

[0187] By displaying statistical information as shown in FIG. 13, the user can confirm what data the employees of, for example, the company to which the user belongs have provided for learning. Taking the trouble of building construction as an example, if it is known that it is a tacit knowledge model that has learned, for example, 1,000 "constructions", it can be expected that it will also answer questions regarding construction troubles, so the user will be willing to try it.

[0188] FIG. 14 shows a process of displaying statistical information generated from the attribute information of the top N data providers with a high degree of attribute relevance to the user's attribute information. The process in FIG. 14 can be implemented in step S4 of FIG. 6, and may also be appropriately implemented according to the operation from the user. In the description of FIG. 14, the differences from FIG. 8 will be mainly described. Steps S81 and S82 may be the same as steps S71 and S72 in FIG. 8.

[0189] In step S83, the attribute relevance calculation unit 47 determines the top N data providers with high attribute relevance (step S83). These N people are the number of people specified by the user in the number input field 253.

[0190] Next, based on the attributes and processes selected by the user on the terminal device 10, the attribute relevance calculation unit 47 creates, for example, a pie chart 256 of the age distribution of data providers whose process is construction (S84). The screen information for displaying the pie chart 256 is transmitted to the terminal device 10.

[0191] <Main effects> According to this embodiment, since the user can refer to the statistical information of a plurality of data providers with high attribute relevance to the user's attribute information, the tendency of the learning data of the tacit knowledge model to be used can be grasped.

[0192] [Third Embodiment] In this embodiment, the server 40 that presents chat information including the user's questions and the answers given by the server 40 during past use will be described.

[0193] First, in this embodiment, the usage information of the image management DB4002 also stores the answers (text information) from the information processing system 100. FIG. 15 shows the image management table in this embodiment. The difference between the image management table in FIG. 15 and FIG. 4(b) is that the usage information includes answers. The answer is the text information (for example, tacit knowledge comment) transmitted by the server 40 to the terminal device 10 in response to the user's question. More specifically, the following specific chat information may be included in the usage information. Question: What items should be carried out during safety patrol? Answer: Check the items placed on the floor, check the wearing of the operator's helmet

[0194] FIG. 16 shows a question-and-answer screen 270 in which a user confirmation field 272 is displayed by pressing a user confirmation tab 271. The question-and-answer screen 270 in FIG. 16 has a user confirmation tab 271, and it is assumed that it has transitioned from FIG. 12(a) by pressing the user confirmation tab 271. Also, in the description of FIG. 16, the differences from FIG. 12(b) will mainly be explained.

[0195] On the question-and-answer screen 270 in FIG. 16, the company name 273, name 274, age 275, affiliation 276, business content 277, length of service 278, and remarks 279 of the past user with the highest attribute relevance to the user are displayed. Furthermore, the question-and-answer screen 270 in FIG. 16 displays the items of chat information 219. When the user presses the chat information 219, the content of the chat information (questions and answers) is displayed.

[0196] By doing so, the user can refer to what chat information was generated between past users with attributes similar to their own and the information processing system 100, so it can be used as a reference during use.

[0197] Also, it is advisable for the server 40 to store in the image management DB 4002, in association with each response, information such as the satisfaction of past users with the answer (was the response useful?) and various comments (why were they satisfied? how was this response utilized?). The question-and-answer screen 270 can also display this information.

[0198] FIG. 17 shows a process of displaying the attribute information of the past user with the highest attribute relevance to the user's attribute information. The process in FIG. 17 can be implemented in step S4 of FIG. 6, and it may also be implemented as appropriate in response to an operation from the user.

[0199] The specific part 44 acquires the attribute information of past users from the user information management DB 4001 (step S91). These past users may be all past users, or they may be the users of the usage information specified by the image ID specified by the user in step S1 of FIG. 6.

[0200] Next, the attribute relevance calculation unit 47 calculates the attribute relevance between the attribute information of the currently used user and the attribute information of past users (step S92).

[0201] The attribute relevance calculation unit 47 determines the past user with the highest attribute relevance (step S93).

[0202] Next, the attribute relevance calculation unit 47 acquires the questions and answers of the input information input by the identified past user from the input information (step S94). This question and answer are the chat information 219. The chat information is transmitted to the terminal device 10 together with the attribute information of the past user with the highest attribute relevance.

[0203] <Main effect> According to the present embodiment, the user can refer to what chat information was generated between the past user most similar to himself and the information processing system 100, so it can be used as a reference during use.

[0204] [Fourth Embodiment] In the third embodiment, the attribute information and chat information of the past user with the highest attribute relevance were presented. In this embodiment, among the past users with high attribute relevance, statistical information created based on the attribute information of the top N past users with high relevance is presented. Since the user can refer to the statistical information regarding the attributes of past users similar to himself, it can be used as a reference for the reliability of the tacit knowledge model and the like.

[0205] FIG. 18(a) shows a question-and-answer screen 280 in which a statistical information column 282 is displayed by pressing the statistical information tab 281. In the description of the question-and-answer screen 280 in FIG. 18(a), the differences from FIG. 13(a) will be mainly described. The statistical information column 282 has a number input column 283, an attribute selection column 284, a process selection column 285, and a pie chart 286. These may be the same as in FIG. 13(a). However, the statistical information in FIG. 18(a) is the statistical information of past users, not the statistical information of data providers.

[0206] Figure 18(b) is the question-and-answer screen 290 displayed by pressing the section. The question-and-answer screen 290 in Figure 18(b) has a user confirmation field 291. The user confirmation field 291 contains generated data (mainly chat information) generated by the server 40 for past users. The user confirmation field 291 displays, for example, a list 292 of chat information of past users in their forties among past users who used the information processing system 100 for construction processes. That is, each chat information includes questions and answers from the server 40 when the user used the server 40 in the past. When the user presses any chat information, the questions and answers included in this chat information are displayed.

[0207] Figure 19 shows a process of displaying statistical information generated from the attribute information of the top N past users with a high degree of attribute relevance to the user's attribute information. In the description of Figure 19, the differences from Figure 17 will be mainly explained. Steps S101 and S102 may be the same as steps S91 and S92 in Figure 19.

[0208] In step S103, the attribute relevance calculation unit 47 determines the top N past users with a high degree of attribute relevance (step S103). These N people are the values specified by the user in the number input field 283.

[0209] Next, the attribute relevance calculation unit 47 creates a pie chart 286 of the age distribution of past users who used the information processing system 100 for construction processes (step S104). The screen information for displaying the pie chart 286 is transmitted to the terminal device 10.

[0210] <Main effects> According to this embodiment, the user can refer to statistical information regarding the attributes of past users similar to themselves, and thus can use it as a reference for the reliability of the tacit knowledge model and the like.

[0211] [Fifth Embodiment] In this embodiment, in the first to fourth embodiments, an information processing system 100 corresponding to images, videos, and sounds in addition to text will be described.

[0212] Figure 20(a) shows the question-and-answer screen 300 of this embodiment. In the question-and-answer screen 300 of Figure 20(a), the input information input field 213 is divided into a text part 302 and other content parts 301. In the text part 302, character information is input in the same manner as in the first embodiment. In the other content parts 301, content other than text, such as images, videos, sounds, point clouds, 3D models, etc., is input. That is, the user can view text information (tacit knowledge comments) for images, videos, and sounds.

[0213] Note that Figure 20(b) is the same question-and-answer screen 230 as Figure 12(b). The display method of the question-and-answer screen 230 may be the same as in the first embodiment. Also, the question-and-answer screen 230 transitioning from Figure 20(a) may be any of the question-and-answer screens 230, 250, 260, 280 in Figures 12, 13, 16, and 18. That is, even if the user inputs an image, video, or sound, the attribute information of the data provider, the statistical information of the attributes of the data provider, the attribute information of the user, or the statistical information of the attributes of the user can be displayed in the same manner.

[0214] Some examples of combinations of input information and tacit knowledge comments will be described. Although the above-described model was based on a large language model, in this embodiment, a multimodal model that inputs a plurality of data formats (images, text, gestures, etc.) and outputs in a predetermined data format is used. · When the input information is a character string and content other than text information is generated as tacit knowledge comments Input a character string to generate an image Input a character string to generate a video Input a character string to generate a sound Input a character string to generate a 3D model · When the input information includes a character string and something other than a character string and text information is generated as tacit knowledge comments Input an image and a character string to generate text information Input a 3D model and a character string to generate text information Input a sound and a character string to generate text information ·When the input information includes a string and something other than a string, and content other than text information is generated as tacit knowledge comments Input an image and a string to generate an image Input a video and a string to generate a video Input a 3D model and a string to generate a 3D model Input voice and a string to generate voice <Main effect> According to this embodiment, since the user can obtain content other than text information as provided data or chat information, the use of the information processing system 100 can be promoted.

[0215] [Sixth Embodiment] In this embodiment, an information processing system 100 that displays content other than text on the question-and-answer screen in the first to fifth embodiments will be described.

[0216] FIG. 21(a) is a question-and-answer screen 300 similar to FIG. 20(a). FIG. 21(b) is a question-and-answer screen 250 transitioned from FIG. 21(a) by pressing the statistical information tab 251. The question-and-answer screen 250 in FIG. 21(b) may be the same as FIG. 13(a).

[0217] Then, when the user presses a statistical information category (for example, the 40s) in the pie chart 256, the provided data (or the usage information registered by the user) provided by the data providers of the age group included in that category is displayed.

[0218] FIG. 22 is a question-and-answer screen 310 displayed by pressing a category. In the user confirmation column 312 of the question-and-answer screen 310 in FIG. 22, a list 313 of the provided data provided by the data providers of the selected category (for example, the 40s) is displayed. This provided data is the provided data provided by the data providers who are in their 40s and provided learning data about the construction process.

[0219] In this provided data, not only character information but also an icon 314 representing content is displayed. When the icon 314 is pressed, the content (for example, an image of a process) is displayed in a pop-up or on a separate screen. In FIG. 22, the content is a thumbnail of an image, but other types of content may also be included in the list 313 in the same manner. Also, different types of content can be displayed in each provided data. Different types of content include, for example, character information + image + audio, etc.

[0220] Also, in the case of the information processing system 100 that generates content other than character information, only the content corresponding to the information processing system 100 may be displayed. For example, the information processing system 100 may display only the images provided by the data provider or the images generated for the user.

[0221] Also, although the attribute information of the data provider and the provided data have been described with reference to FIGS. 21 and 22, past user attribute information and chat information generated by the information processing system 100 can also be displayed.

[0222] <Main effects> According to this embodiment, since the user can confirm the provided data other than the text provided by the data provider or past users, and the chat information other than the text generated by past users when using the information processing system, it leads to further promotion of utilization.

[0223] [Seventh Embodiment] The caption model 4003, the tacit knowledge model 4004, and the large language model 4005 may be generated in groups. The groups may be by company, process, or attribute (or a combination thereof). As a user, there may be a case where you want to know which data providers provided the learning data only for the company you belong to or the processes you are related to, or which users there are only for the company you belong to or the processes you are related to. Therefore, in the present embodiment, an information processing system 100 will be described in which a user can select a caption model 4003, a tacit knowledge model 4004, and a large language model 4005 (hereinafter sometimes simply referred to as a model) that generate text information. That is, in the present embodiment, the caption model 4003, the tacit knowledge model 4004, and the large language model 4005 are generated for each group. Also, data providers and users are classified into the groups they belong to in advance.

[0224] FIG. 23 shows a question-and-answer screen 320 in which a model can be selected. The question-and-answer screen 320 has a message 321 saying "Please select a model", a company selection field 322, and a process selection field 323. The company selection field 322 displays a list of companies, and the process selection field 323 displays a list of processes.

[0225] The user can select a model learned from the data within a certain company from the company selection field 322, or select a model learned from the data of a certain process in the process selection field 323. Either the company or the process may be selectable only, or may be selectable under an AND or OR condition.

[0226] The attribute relevance calculation unit 47 calculates the attribute relevance between the attribute information of the data provider that provided the data used when generating the selected model and the attribute information of the user. Alternatively, the attribute relevance calculation unit 47 calculates the attribute relevance between the attribute information of past users who used the selected model and the attribute information of the user. Therefore, the user can know which data providers in the group to which the user belongs provided the learning data, or which past users in the group to which the user belongs used the information processing system.

[0227] <Main effect> According to the present embodiment, since information with a high usage promotion effect can be presented for each group such as a company, the usage promotion effect is improved.

[0228] [Other application examples] The present invention is not limited to the above-described embodiments specifically disclosed, and various modifications and changes can be made without departing from the scope of the claims. Note that the server 40 described in the present embodiment is an example, and it goes without saying that there are various system configuration examples according to the use and purpose.

[0229] For example, in the present embodiment, an example in which the tacit knowledge model in industries such as civil engineering and architecture answers questions has been described, but the tacit knowledge model may be used in industries where tacit knowledge is effective, such as medical care, dental care, and investment judgment.

[0230] In addition, the display of the attribute information in FIG. 12 etc. is not limited to the case where the user uses the server 40, and the attribute information may be displayed at an arbitrary timing.

[0231] In addition, not limited to the data provider or past user with the highest attribute relevance to the user's attribute information, a list of several top ones may be displayed, and the user may select the person to display the attribute information from among them.

[0232] Also, in this embodiment, the client-server type information processing system 100 has been described, but the functions of the server 40 may be installed in the terminal device 10 as an application. That is, the user may be able to use the functions of this embodiment in a stand-alone manner.

[0233] In addition, the configuration examples such as in FIG. 3 are divided according to the main functions in order to facilitate the understanding of the processing by the server 40. The present invention is not limited by the way of dividing the processing units or the names. The processing of the server 40 can be further divided into more processing units according to the processing content. Also, one processing unit can be divided so as to include more processes.

[0234] Each function of the embodiment described above can be realized by one or a plurality of processing circuits. Here, the "processing circuit" in this specification includes a processor programmed to execute each function by software like a processor implemented by an electronic circuit, an ASIC (Application Specific Integrated Circuit) designed to execute each function described above, a DSP (digital signal processor), an FPGA (field programmable gate array), and devices such as conventional circuit modules.

[0235] The device group described in the examples only shows one of a plurality of computing environments for implementing the embodiments disclosed in this specification. In one embodiment, the server 40 includes a plurality of computing devices such as a server cluster. The plurality of computing devices are configured to communicate with each other via an arbitrary type of communication link including a network, a shared memory, etc., and implement the processing disclosed in this specification.

[0236] Furthermore, the server 40 can also combine the disclosed processing steps in various ways. Each element of the server 40 may be integrated into one device or divided into multiple devices. Also, each process performed by the server 40 may be performed by the terminal device 10.

Explanation of Signs

[0237] 10 Terminal device 40 Server 100 Information processing system

Prior Art Documents

Patent Documents

[0238]

Patent Document 1

Claims

1. a storage means for storing a model generated using text data based on at least one of the voice information and the character information received by the input receiving means as learning data; a text information generating means for generating text information indicating an answer to the question based on the question and the model; a screen generating means for generating a screen including attribute information of a data provider who has provided at least one of the voice information and the character information; An information processing device comprising:

2. A terminal device is capable of communicating with the terminal device via a network, 2. The information processing apparatus according to claim 1, further comprising a transmission means for transmitting screen information relating to said screen to said terminal device.

3. an attribute relevance calculation means for calculating an attribute relevance between attribute information of a user of the terminal device and attribute information of the data provider; the screen generation means generates a screen including the attribute information of the data provider extracted based on the attribute relevance degree; 3. The information processing apparatus according to claim 2, wherein said transmission means transmits screen information of said screen including said attribute information to said terminal device.

4. the screen generation means generates a screen including attribute information of the data provider having the highest attribute relevance with the attribute information of the user; 4. The information processing apparatus according to claim 3, wherein said transmission means transmits to said terminal device screen information of a screen including the attribute information of said data provider having the highest attribute relevance with said attribute information of said user.

5. the screen generating means generates a screen including at least one of the voice information and the character information provided by the data provider having the highest attribute relevance with the attribute information of the user; 4. The information processing apparatus according to claim 3, wherein said transmission means transmits screen information of a screen including at least one of said voice information and said character information to said terminal device.

6. the attribute relevance calculation means creates statistical information using attribute information of the data providers who are in the top N positions having the highest attribute relevance with the attribute information of the user; The screen generating means generates a screen including the statistical information, 4. The information processing apparatus according to claim 3, wherein said transmission means transmits screen information of a screen including said statistical information to said terminal device.

7. The statistical information is displayed in a graph showing attribute information of the data provider as a segment; When a notification that one of the segments in the graph has been selected is sent from the terminal device, the screen generating means generates a screen including a list of the voice information and the text information provided by the data provider corresponding to the selected category, and the transmitting means transmits screen information relating to the screen to the terminal device; When the terminal device transmits a message indicating that the voice information and the character information in the list have been selected, 7. The information processing apparatus according to claim 6, wherein said screen generating means generates a screen including the selected voice information and character information, and said transmitting means transmits screen information relating to the screen to said terminal device.

8. an attribute relevance calculation means for calculating an attribute relevance between attribute information of a user of the terminal device and attribute information of a past user who has used the model in the past; the screen generation means generates a screen including past user attribute information extracted based on the attribute relevance; 3. The information processing apparatus according to claim 2, wherein said transmission means transmits screen information of said screen including said attribute information to said terminal device.

9. the screen generation means generates a screen including the attribute information of the past user having the highest attribute relevance with the attribute information of the user; 9. The information processing apparatus according to claim 8, wherein the transmission means transmits to the terminal device screen information of a screen including the attribute information of the past user having the highest attribute relevance with the attribute information of the user.

10. the screen generation means generates a screen including the text information provided to the past user having the highest attribute relevance with the attribute information of the user; 9. The information processing apparatus according to claim 8, wherein said transmission means transmits screen information of a screen including said text information to said terminal device.

11. the attribute relevance calculation means creates statistical information using attribute information of the top N past users having high attribute relevance with the attribute information of the user; The screen generating means generates a screen including the statistical information, 9. The information processing apparatus according to claim 8, wherein said transmission means transmits screen information of a screen including said statistical information to said terminal device.

12. the statistical information is displayed in a graph that represents the attribute information of the past users as segments; When a notification that one of the segments in the graph has been selected is sent from the terminal device, the screen generating means generates a screen including a list of the text information provided to the past users corresponding to the selected category, and the transmitting means transmits screen information regarding the screen to the terminal device; When a message is sent from the terminal device indicating that the text information in the list has been selected, 12. The information processing apparatus according to claim 11, wherein the screen generating means generates a screen including the selected text information, and the transmitting means transmits screen information relating to the screen to the terminal device.

13. The data providers are classified into groups according to their affiliations, and the storage means stores a plurality of the models generated for each group; the screen generating means generates a screen for displaying a list of the groups; The transmission means transmits screen information relating to the screen to the terminal device; When a message indicating that the group in the list has been selected is transmitted from the terminal device, the screen generating means generates a screen including attribute information of the data providers classified into the groups; 8. The information processing apparatus according to claim 3, wherein the transmission means transmits screen information relating to the screen to the terminal device.

14. The past users are classified into groups according to their affiliations, and the storage means stores a plurality of the models generated for each of the groups; the screen generating means generates a screen for displaying a list of the groups; The transmission means transmits screen information relating to the screen to the terminal device; When a message indicating that the group in the list has been selected is transmitted from the terminal device, The screen generation means generates a screen including attribute information of the past users classified into the group; 13. The information processing apparatus according to claim 8, wherein the transmission means transmits screen information relating to the screen to the terminal device.

15. The model is generated by performing a learning process using images and the text data as learning data; The text information generating means generates the text information based on a target image specified by a user on the terminal device and the model; 8. The information processing apparatus according to claim 3, wherein the transmitting means transmits the text information to the terminal device.

16. the screen generation means generates a screen including the image provided by the data provider that has the highest attribute relevance with the attribute information of the user; 16. The information processing apparatus according to claim 15, wherein the transmission means transmits screen information of a screen including the image to the terminal device.

17. An information processing method performed by an information processing device, based on a storage means for storing a model generated using text data based on at least one of the voice information and the character information received by the input receiving means as learning data, generating text information indicative of an answer to the question based on the question and the model; A process of generating a screen including attribute information of a data provider who has provided at least one of the voice information and the character information; The information processing method used.

18. An information processing device, based on a storage means for storing a model generated using text data based on at least one of the voice information and the character information received by the input receiving means as learning data, a text information generating means for generating text information indicating an answer to the question based on the question and the model; a screen generating means for generating a screen including attribute information of a data provider who has provided at least one of the voice information and the character information; A program to make it function as such.

19. An information processing system in which a terminal device and an information processing device can communicate with each other via a network, The information processing device includes: a storage means for storing a model generated using text data based on at least one of the voice information and the character information received by the input receiving means as learning data; a text information generating means for generating text information indicating an answer to the question based on the question and the model; a screen generating means for generating a screen including attribute information of a data provider who has provided at least one of the voice information and the character information; a first transmission means for transmitting screen information relating to the screen to the terminal device; The terminal device the input receiving means for receiving an input of the question; a second transmission means for transmitting the question accepted by the input acceptance means to the information processing device; a display control means for displaying a screen including attribute information of the data provider based on the screen information received from the information processing device; An information processing system having the above configuration.

Images