Information provision system, information provision method, and information provision program

Abstract

This system provides an information provision system, information provision method, and information provision program that comprehensively determines the optimal information for a user based on the user's role, purpose, and status when using output devices such as multifunction printers. [Solution] The information provision system comprises: a first feature extraction means for extracting a first feature quantity that includes the type of document, the content of the document, and the type of user handling the document, which are features of the document output from the output device; a second feature extraction means for extracting a second feature quantity that includes personal information, the situation of the person, and the emotions of the person, which are features of the user using the output device; a database storing content linked to the first and second feature quantities; a content extraction means for extracting the content useful to the user from the database using a machine learning model based on the first and second feature quantities; and an output means for outputting the extracted content to the user.

Description

Technical Field

[0001] The present invention relates to an information providing system, an information providing method, and an information providing program.

Background Art

[0002] In Patent Document 1, news information from a news agency and advertisement information from an advertiser are sequentially transmitted to an advertising company to update the database of the advertising company. When a university student makes a copy using a copying machine of a university cooperative with a mobile phone, the data of the mobile phone is read and the personal information of the user is transmitted from a personal information management company to the advertising company, and news and advertisement information matching the personal information is transmitted to the copying machine and printed on the back side of the copy paper. The technology is disclosed.

Summary of the Invention

Problems to be Solved by the Invention

[0003] However, in the technology described in Patent Document 1, since the history of past use is used, it is difficult to protect the personal information of the user, and the real-time information of the user who uses an output device such as a copying machine cannot be grasped.

[0004] The present invention has been made in view of the above, and an object of the present invention is to provide an information providing system, an information providing method, and an information providing program that can comprehensively judge and provide optimal information for a user from the role, purpose, and state of the user who uses an output device such as a multifunction machine.

Means for Solving the Problems

[0005] To solve the above-mentioned problems and achieve the objective, the present invention provides an information provision system that provides useful information to a user of an output device, comprising: a first feature extraction means for extracting a first feature quantity that is a feature quantity of a document output from the output device, including the type of document, the content of the document, and the type of user handling the document; a second feature extraction means for extracting a second feature quantity that is a feature quantity of a user of the output device, including person information, person situation, and person emotions; a database storing content associated with the first feature quantity and the second feature quantity; a content extraction means for extracting the user-beneficial content from the database using a machine learning model based on the first feature quantity and the second feature quantity; and an output means for outputting the extracted content to the user. [Effects of the Invention]

[0006] According to the present invention, it is possible to comprehensively determine and provide the most suitable information for the user based on the user's role, purpose, and status when using an output device such as a multifunction printer. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a diagram illustrating an example of how the information provision system according to this embodiment can solve the problem. [Figure 2] Figure 2 is a diagram illustrating an example of how the information provision system according to this embodiment can solve the problem. [Figure 3] Figure 3 is a diagram illustrating an example of how the information provision system according to this embodiment can solve the problem. [Figure 4] Figure 4 is a diagram illustrating an example of the configuration of the information provision system according to this embodiment. [Figure 5] Figure 5 shows an example of the functional configuration of the information provision system according to this embodiment. [Modes for carrying out the invention]

[0008] The embodiments of the information provision system, information provision method, and information provision program will be described in detail below with reference to the attached drawings.

[0009] The information provision system according to this embodiment has the following features when providing information that matches the user using an output device such as a multifunction printer. The information provision system according to this embodiment includes an output device such as a multifunction printer that acquires the user's document information, a machine learning model that applies natural language processing (NPL) to the document information and extracts the features of the document information, a camera that acquires video information of the user using the multifunction printer, a machine learning model that applies image recognition technology to the video information and extracts the user's features, a machine learning model that links the document information, the user's features, and the information in the database that is optimal for the user, a device that provides the optimal information on paper or on a display, means for determining the usefulness of the provided information, and means for reflecting the determination result as training data for the machine learning model, and the accuracy of the machine learning model can be improved by periodically retraining it with updated training data.

[0010] In short, the information provision system according to this embodiment determines the user's personal attributes from document information output from an output device such as a multifunction printer and video information when using the multifunction printer, determines and provides information best suited to the personal attributes using a machine learning model, and provides usefulness feedback after provision as training data to the machine learning model for retraining. By presenting the most suitable advertisements based on information collected from user behavior in real time, without using the user's behavior history, the accuracy of the matching system can be continuously improved.

[0011] The features of the information provision system according to this embodiment will be described in detail below with reference to the drawings. First, an example of the problems (1) to (3) of the information provision system will be explained.

[0012] (1) When providing information to users, using their past behavioral history makes it difficult to protect their privacy.

[0013] (2) Even if a user's past behavioral history is used when providing information to a user, it is not possible to understand the user's role, purpose, and circumstances at the time of using the multifunction printer. Specifically, each user of a multifunction printer has a role and purpose. This includes work roles, family roles, hobbies, etc. On the other hand, each user has a state of being when using the multifunction printer. This state includes the actions the user is taking to fulfill their role and purpose, the state or circumstances the user is in at that time (for example, being outside the house, carrying luggage), the user's emotions, etc. The optimal information to provide to a user should be determined comprehensively based on the user's role, purpose, and state.

[0014] Furthermore, the mobile phone data used in Patent Document 1 as a means of understanding the user's role, purpose, and state is intended to predict the role, purpose, and state from the user's past behavioral history, and it is not possible to recognize these in real time. In contrast, this embodiment is characterized by making decisions based on real-time information obtained when using the multifunction printer.

[0015] (3) The matching system cannot be continuously improved in accuracy. Patent Document 1 only provides information based on the matching system, but does not verify the usefulness of the content provided. Therefore, it misses opportunities to acquire data to improve the accuracy of the information provided by the matching system, and the accuracy of the system cannot be improved efficiently.

[0016] Therefore, as described above, the information provision system according to this embodiment comprehensively determines and provides the most suitable information for the user based on the user's role, purpose, and status when using the multifunction printer.

[0017] First, using Figures 1 and 2, we will explain an example of how to solve problems (1) and (2) in the information provision system according to this embodiment. Figures 1 and 2 are diagrams illustrating an example of how to solve the problems using the information provision system according to this embodiment.

[0018] In this embodiment, instead of predicting the user's role, purpose, and state from past behavioral history, the system makes real-time determinations based on information gathered when the user uses the multifunction printer. This eliminates the need to collect past behavioral history data, thus limiting the areas of privacy protection that need to be considered, and also eliminates the need for a big data collection system, thus reducing costs.

[0019] The configuration required to realize the information provision system according to this embodiment includes a multifunction printer and a camera capable of capturing video of the user using the multifunction printer. The camera may be a separately installed camera capable of capturing the environment in which the multifunction printer is installed, or a camera built into the controller panel of the multifunction printer.

[0020] In multifunction printers, the user's role, purpose, and state are understood by reading the text information of documents when copying or printing them. To achieve this, natural language processing (NPL) is applied to the text information, and a machine learning model is prepared to extract features of the user's document information. This machine learning model is trained on data that links various types of documents (e.g., business documents, musical scores, recipes, drawings, ledgers, patents) and the types of users handling the documents (e.g., business people, musicians, chefs, designers, accountants, intellectual property personnel). By inputting text information, the model can infer the user type itself, or the user type as a feature.

[0021] Some multifunctional devices installed in office environments have an employee authentication function for on-demand printing. In such cases, by also using information related to the organization, position, and job content of the user linked to the employee ID, the user type can be determined with higher accuracy. Also, by training the in-house document database, the interpretation accuracy for in-house terms can be improved, making it possible to determine the user type with higher accuracy.

[0022] On the other hand, by applying video recognition technology to the video of the user using the multifunctional device, the state of the user is grasped. For this purpose, a machine learning model for determining the state of the user is prepared. This machine learning model learns, as teacher data, data that links information such as person information (e.g., user appearance, age, gender, dominant hand, items worn (glasses, presence of a wristwatch), brand of the clothes worn), person situation (e.g., who the person is with, whether carrying luggage, type and brand name of the luggage, whether operating while doing something like using a phone or smartphone, number of companions), and person emotion (e.g., expression, operation time of the multifunctional device, satisfaction with the printed material or redoing the settings) to the person video (video information). Therefore, the machine learning model is a model that can output person information, person situation, person emotion itself, or those as feature amounts by inputting video information.

[0023] The above machine learning models may be installed in a system on the cloud or on-premises, or may be held in a multifunctional device or camera as edge AI. By using these machine learning models, the role, purpose, and state of the user are grasped, and the attributes of the user are determined.

[0024] As a means of linking the document information of the user, the feature amounts of the user himself / herself, or the person attributes determined therefrom, with the information optimal for the user in the database, methods such as content-based filtering and hybrid filtering are used.

[0025] In content-based filtering, the optimal user attribute type for each type of information that can be provided is predetermined. Based on the acquired information, relevant advertisements and other information are provided based on the user's attributes (personal attributes).

[0026] Hybrid filtering determines the user's personal attributes based on the acquired information, creates a profile, searches the database for other users with similar attributes based on that profile, and provides information preferred by similar users.

[0027] These methods utilize machine learning models. These machine learning models learn from user attributes and previously provided information (e.g., the type and content of the information provided, and usefulness feedback data (user response data to the information)) as training data. When user attributes and information data are input, the model outputs appropriate information options and confidence scores.

[0028] The information provided could include, for example, advertisements and news. If the multifunction printer is installed in an office, it could also include company information; if it's in a store such as a convenience store, it could include coupons for services and goods offered there. For example, if providing information to a user with many packages, it could include information on delivery services; if providing information to a user who seems stressed at work, it could include coupons for products that reduce stress.

[0029] Methods of providing information include printing the back side of single-sided documents using a multifunction printer, or, if there is digital signage installed near the multifunction printer, transmitting and displaying the information there. In stores that have introduced a POS system, such as convenience stores, it is possible to link the user with the information provided when using the multifunction printer, and then install a camera near the POS system to identify the user making the payment and print the information on the receipt.

[0030] Next, using Figure 3, an example of a method for solving problem (3) in the information provision system according to this embodiment will be described. Figure 3 is a diagram illustrating an example of a method for solving the problem using the information provision system according to this embodiment.

[0031] In this embodiment of the information provision system, the process doesn't end with providing information; it tracks whether the information was useful to the user and uses the results as a learning model to improve the accuracy of the machine learning model when it is retrained.

[0032] For example, to determine whether information displayed on signage installed near a multifunction printer was useful, a quantitative determination can be made by measuring the viewing time of the information displayed on the signage using eye-tracking data from an installed camera. Similarly, if coupons printed on receipts are provided, the usefulness of the information can be determined by tracking whether the coupons were actually used. This determination information is used as feedback on the usefulness of the provided information and as training data for retraining machine learning models that match user attributes with the provided information.

[0033] Next, an example of the configuration of the information provision system according to this embodiment will be described using Figure 4. Figure 4 is a diagram illustrating an example of the configuration of the information provision system according to this embodiment.

[0034] As shown in Figure 4, the information provision system according to this embodiment includes an information database 401, a user attribute determination system 402, an information provision history database 403, and an information presentation system 404.

[0035] The information database 401 is located on a local or cloud server. When information is input from an information source such as an advertising company, the information database 401 classifies the information (e.g., by tagging or vectorizing it) and stores the input information and the classification results.

[0036] The information provision history DB 403 is located on a local or cloud server. The information provision history DB 403 receives the information provision history from the information presentation system 404. Furthermore, the information provision history DB 403 receives feedback from the information provision means regarding whether or not the user's information has been utilized. The information provision history DB 403 then stores the information provision history and the received feedback.

[0037] The user attribute determination system 402 receives document information output by the multifunction printer when a person uses it, and classifies the attributes of the document information (document attributes) and the attributes of the user who used the multifunction printer (person attributes) based on the document information. Alternatively, when a person uses the multifunction printer, the user attribute determination system 402 receives video information of the user using the multifunction printer captured by a camera, and classifies the attributes of the user who used the multifunction printer (person attributes) based on the received video information. Alternatively, when a person uses the multifunction printer, the user ID is transmitted from the multifunction printer to the user database, and when the multifunction printer retrieves user information from the user database, it classifies the attributes of the user who used the multifunction printer (person attributes) based on that user information. The user attribute determination system 402 also outputs the document attributes and person attributes classified by the above processes to the information presentation system 404.

[0038] The information presentation system 404 receives document attributes and person attributes from the user attribute determination system 402. The information presentation system 404 also refers to or learns from the information DB 401 and the information provision history DB 403 and selects the most appropriate information for the document attributes and person attributes. Subsequently, the information presentation system 404 transmits the selected information to the information provision means (e.g., multifunction printer, signage, cash register system). Furthermore, the information presentation system 404 transmits the information provision history to the information provision history DB 403.

[0039] Next, an example of the functional configuration of the information provision system according to this embodiment will be described using Figure 5. Figure 5 is a diagram showing an example of the functional configuration of the information provision system according to this embodiment.

[0040] The information provision system according to this embodiment is an example of an information provision system that provides useful information to users who use output devices such as multifunction printers. As shown in Figure 5, the information provision system according to this embodiment includes a first feature extraction means 501, a second feature extraction means 502, a database 503, a content extraction means 504, an output means 505, a camera 506, and the like.

[0041] Camera 506 is an example of a camera installed near an output device such as a multifunction printer and used to photograph users of the output device.

[0042] The first feature extraction means 501 is implemented by a user attribute determination system 402, etc., and extracts document attributes (an example of the first feature) that include the type of document information, the content of the document information, and the type of user handling the document information, which are examples of features of document information (an example of a document) output from an output device such as a multifunction printer. For example, the first feature extraction means 501 may learn data linking the type of document information, the content of the document information, and the type of user handling the document information as training data, and extract document attributes by natural language processing based on a part of the document information.

[0043] The second feature extraction means 502 is implemented by the user attribute judgment system 402, etc., and extracts human attributes (an example of the second feature), including person information, person situation, and person emotions, which are features of a user using an output device such as a multifunction printer. For example, the second feature extraction means 502 may extract human attributes from video information (an example of an image) of a user using the output device captured by the camera 506 using image recognition technology. Alternatively, for example, the second feature extraction means 502 may learn data linking person information, person situation, and person emotions as training data, and extract human attributes using image recognition technology based on information about the person (e.g., video information).

[0044] Database 503 is implemented using information DB 401, etc., and stores information (an example of content) linked to document attributes and person attributes. For example, database 503 may store training data that links document attributes, person attributes, and information useful to the user.

[0045] Here, content-based filtering may be used as a means of linking document attributes, person attributes, and user-relevant information. Alternatively, collaborative filtering may be used as a means of linking document attributes, person attributes, and user-relevant information.

[0046] The content extraction means 504 is implemented by an information presentation system 404, etc., and extracts information useful to the user from the database 503 using a machine learning model based on document attributes and person attributes.

[0047] The output means 505 is implemented by the information presentation system 404, etc., and outputs the information extracted by the content extraction means 504 to the user.

[0048] Thus, according to this embodiment of the information provision system, it is possible to comprehensively determine and provide the most suitable information for the user based on the user's role, purpose, and status when using the multifunction printer.

[0049] The program executed by the information provision system of this embodiment is provided pre-installed in ROM (Read Only Memory) or the like. The program executed by the information provision system of this embodiment may also be configured to be provided as an installable or executable file recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD (Digital Versatile Disk).

[0050] Furthermore, the program executed by the information provision system of this embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading it via the network. Alternatively, the program executed by the information provision system of this embodiment may be provided or distributed via a network such as the Internet.

[0051] The program executed in the information provision system of this embodiment has a modular configuration that includes the above-described parts (first feature extraction means 501, second feature extraction means 502, content extraction means 504, output means 505). In actual hardware, an example of a processor such as a CPU (Central Processing Unit) reads the program from the ROM and executes it, loading the above-described parts onto the main memory, and generating the first feature extraction means 501, second feature extraction means 502, content extraction means 504, and output means 505 on the main memory.

[0052] Examples of the present invention are as follows: <1> An information provision system that provides useful information to users of the output device, A first feature extraction means for extracting a first feature quantity which is a feature quantity of a document output from the output device, including the type of document, the content of the document, and the type of user handling the document. A second feature extraction means for extracting a second set of features, which include personal information, personal situation, and personal emotions, that are characteristic features of the user using the output device. A database storing content associated with the first and second features, A content extraction means that extracts user-beneficial content from the database using a machine learning model based on the first and second features, Output means for outputting the extracted content to the user, An information provision system equipped with this system. <2> The database stores the first feature, the second feature, and the user-beneficial content as training data, linking them together. <1> The information provision system described above. <3> The system includes a camera located near the output device that photographs the user using the output device. The second feature extraction means extracts the second feature from an image of a user using the output device using image recognition technology. <1> or <2> The information provision system described above. <4> The first feature extraction means learns data linking the type of document, the content of the document, and the type of user handling the document as training data, and extracts the first features by natural language processing based on a portion of the document. <1> from <3> The information provision system described in any one of the following. <5> The second feature extraction means learns data linking the person information, the person's situation, and the person's emotions as training data, and extracts the second feature based on the information about the person using image recognition technology. <1> or <2> The information provision system described above. <6> As a means of linking the first feature, the second feature, and the user-beneficial content, content-based filtering is used. <1> from <5> The information provision system described in any one of the following. <7> Collaborative filtering is used as a means of linking the first feature, the second feature, and the user-beneficial content. <1> from <5> The information provision system described in any one of the following. <8> An information provision method performed in an information provision system that provides useful information to users of an output device, A step of extracting a first feature quantity which is a feature quantity of the document output from the output device, including the type of document, the content of the document, and the type of user handling the document. A step of extracting a second set of features, which include personal information, personal situation, and personal emotions, that are characteristic features of the user using the output device, Based on the first and second features, the process involves extracting user-friendly content from a database using a machine learning model, and The steps include outputting the extracted content to the user, Information provision methods including those mentioned above. <9> A computer that provides useful information to users who use the output device, A first feature extraction means for extracting a first feature quantity which is a feature quantity of a document output from the output device, including the type of document, the content of the document, and the type of user handling the document. A second feature extraction means for extracting a second set of features, which include personal information, personal situation, and personal emotions, that are characteristic features of the user using the output device. A content extraction means that extracts user-beneficial content from a database using a machine learning model based on the first and second features, Output means for outputting the extracted content to the user, An information provision program to enable it to function. [Explanation of symbols]

[0053] 401 Information DB 402 User Attribute Determination System 403 Information Provision History Database 404 Information Display System 501 First feature extraction means 502 Second feature extraction means 503 Databases 504 Content Extraction Method 505 Output means 506 Camera [Prior art documents] [Patent Documents]

[0054] [Patent Document 1] Japanese Patent Publication No. 2010-134620

Claims

1. An information provision system that provides useful information to users of the output device, A first feature extraction means for extracting a first feature quantity which is a feature quantity of the document output from the output device, including the type of the document, the content of the document, and the type of user handling the document. A second feature extraction means for extracting a second set of features, which are characteristic features of the user utilizing the output device, including person information, person situation, and person emotions. A database storing content associated with the first feature and the second feature, A content extraction means that extracts user-beneficial content from the database using a machine learning model based on the first and second features, Output means for outputting the extracted content to the user, An information provision system equipped with this system.

2. The information provision system according to claim 1, wherein the database stores the first feature, the second feature, and the user-beneficial content as training data.

3. The system includes a camera located near the output device that photographs the user using the output device. The information provision system according to claim 1 or 2, wherein the second feature extraction means extracts the second feature from an image of a user taking a photograph of the user using the output device using image recognition technology.

4. The information provision system according to claim 1 or 2, wherein the first feature extraction means learns data linking the type of document, the content of the document, and the type of user handling the document as training data, and extracts the first features by natural language processing based on a part of the document.

5. The information provision system according to claim 1 or 2, wherein the second feature extraction means learns data linking the person information, the person situation, and the person's emotions as training data, and extracts the second feature based on the information about the person using image recognition technology.

6. The information provision system according to claim 1 or 2, wherein content-based filtering is used as a means for linking the first feature quantity, the second feature quantity, and the content useful to the user.

7. The information provision system according to claim 1 or 2, wherein collaborative filtering is used as a means for linking the first feature quantity, the second feature quantity, and the content useful to the user.

8. An information provision method performed in an information provision system that provides useful information to users of an output device, A step of extracting a first feature quantity which is a feature quantity of the document output from the output device, including the type of the document, the content of the document, and the type of user handling the document. A step of extracting a second set of features, which include personal information, personal situation, and personal emotions, that are characteristic features of the user using the output device, Based on the first and second features, the process involves extracting user-friendly content from a database using a machine learning model, and The steps include outputting the extracted content to the user, Information provision methods including those mentioned above.

9. A computer that provides useful information to users who use the output device, A first feature extraction means for extracting a first feature quantity which is a feature quantity of the document output from the output device, including the type of the document, the content of the document, and the type of user handling the document. A second feature extraction means for extracting a second set of features, which are characteristic features of the user utilizing the output device, including person information, person situation, and person emotions. A content extraction means that extracts user-beneficial content from a database using a machine learning model based on the first and second features, Output means for outputting the extracted content to the user, An information provision program to enable it to function.

Images