Information processing system, image processing device, information processing device, information processing method, and program

Abstract

To support work for understanding sentences included in image data read from a manuscript.SOLUTION: An information processing system has: a read control part which controls reading of image data from a manuscript in response to an instruction from a user; a user feature acquisition part which acquires feature information on the user; a reception part which receives information indicating conversion processing to be executed on sentences included in the image data from the user; a character recognition part which extracts the sentences from the image data; a conversion part which inputs information including that the conversion processing is executed on the sentences and that a result of the conversion processing is suitable to a person associated with the feature information, to a large-scale language model, and acquires a conversion result output by the large-scale language model; and an output part which outputs the conversion result.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an information processing system, an image processing apparatus, an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, it has been considered to apply a machine learning model to text extracted from image data by optical character recognition (OCR).

[0003] For example, Patent Document 1 discloses a system that corrects text obtained by OCR using a neural network learned based on misrecognized portions of OCR and the surrounding text.

Summary of the Invention

Problems to be Solved by the Invention

[0004] On the other hand, easy-to-understand text (sentences) can vary depending on the user. For example, even for the same Japanese, for those who do not have Japanese as their native language, sentences with fewer Chinese characters are more likely to be easy to understand. Also, in a company, familiar expressions vary depending on the department to which one belongs. For example, for a user belonging to the legal department, they are familiar with sentences such as "A does... to B", but for users belonging to other departments, such sentences are difficult to understand.

[0005] The present invention has been made in view of the above points, and an object thereof is to assist the work for understanding the sentences included in the image data read from a manuscript.

Means for Solving the Problems

[0006] Therefore, to solve the above problems, an information processing system includes a reading control unit that controls reading of image data from a manuscript according to a user's instruction, a user feature acquisition unit that acquires the user's feature information, a reception unit that receives information indicating conversion processing to be performed on the text included in the image data from the user, a character recognition unit that extracts the text from the image data, and inputs information including an instruction to perform the conversion processing on the text and an instruction that the result of the conversion processing is suitable for the person related to the feature information to a large language model, and a conversion unit that acquires a conversion result output by the large language model, and an output unit that outputs the conversion result.

Effect of the Invention

[0007] It is possible to assist in the work for understanding the text included in the image data read from the manuscript.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a configuration example of the information processing system in the first embodiment. In FIG. 1, the information processing system includes one or more image processing apparatuses 10 and an information processing apparatus 20. The image processing apparatus 10 is connected to the information processing apparatus 20 via a network such as a LAN (Local Area Network) or the Internet.

[0010] The image processing apparatus 10 is a device having functions such as a function of reading (scanning) image data from a document (paper document), a communication function, and an information processing function. For example, the image processing apparatus 10 may be a device including a scanner and a printer, or a multifunction device or the like. The image processing apparatus 10 requests the information processing apparatus 20 to execute a conversion process (for example, translation, summarization, etc.) instructed by the user for the text included in the image data read from the document (hereinafter referred to as "scanned image"). The image processing apparatus 10 receives the conversion result from the information processing apparatus 20 and outputs the conversion result.

[0011] The information processing device 20 is one or more computers that execute the conversion process requested by the image processing device 10. The information processing device 20 extracts a sentence (character string) from the scanned image by applying optical character recognition (OCR) to the scanned image, and executes the conversion process on the sentence. The information processing device 20 uses a large language model (LLM (Large Language Model)) specialized for natural language processing among the neural networks as the generation AI in the conversion process. The large language model is a learned model used for natural language processing (NLP). The large language model learns a large amount of text data and can realize natural language generation and understanding like humans. The large language model has a higher natural language understanding ability than conventional natural language processing models and shows excellent results in natural language processing tasks such as natural sentence generation and question answering. Examples of such LLM include GPT-3 and GPT-4 developed by OpenAI, and BERT developed by Google (registered trademark).

[0012] When using such a large language model, the information processing device 20 causes the large language model to execute the conversion process so as to obtain a conversion result suitable for the user of the image processing device 10. Note that conversion means changing (arranging) part or all of a sentence into a different set of sentences. Examples of conversion include summarization and translation.

[0013] Note that the environment in which the information processing system is used is not limited to a specific one, but in this embodiment, for the sake of convenience, the case where the information processing system is used within a company will be described.

[0014] FIG. 2 is a diagram showing a hardware configuration example of the image processing device 10 in the first embodiment. In FIG. 2, the image processing device 10 has hardware such as a controller 11, a scanner 12, a printer 13, a modem 14, an operation panel 15, a network interface 16, and an SD card slot 17.

[0015] The controller 11 includes a CPU 111, a RAM 112, a ROM 113, an HDD 114, an NVRAM 115, etc. The ROM 113 stores various programs and data used by the programs. The RAM 112 is used as a storage area for loading programs and a work area for the loaded programs. The CPU 111 realizes various functions by processing the programs loaded into the RAM 112. The HDD 114 stores programs and various data used by the programs. The NVRAM 115 stores various setting information, etc.

[0016] The scanner 12 is hardware (image reading means) for reading image data from a document. The printer 13 is hardware (printing means) for printing print data on printing paper. The modem 14 is hardware for connecting to a telephone line and is used to execute transmission and reception of image data by FAX communication. The operation panel 15 is hardware provided with input means such as buttons for receiving input from the user and display means such as a liquid crystal panel. The liquid crystal panel may have a touch panel function. In this case, the liquid crystal panel also serves as the function of the input means. The network interface 16 is hardware for connecting to a network such as a LAN (regardless of whether it is wired or wireless). The SD card slot 17 is used to read the program stored in the SD card 80. That is, in the image processing apparatus 10, not only the programs stored in the ROM 113 but also the programs stored in the SD card 80 can be loaded into the RAM 112 and executed. Note that the SD card 80 may be replaced by another recording medium (for example, a CD-ROM or a USB (Universal Serial Bus) memory, etc.). That is, the type of the recording medium corresponding to the position of the SD card 80 is not limited to a predetermined one. In this case, the SD card slot 17 may be replaced by hardware corresponding to the type of the recording medium.

[0017] FIG. 3 is a diagram showing a hardware configuration example of the information processing apparatus 20 in the first embodiment. The information processing apparatus 20 in FIG. 3 includes a drive device 200, an auxiliary storage device 202, a memory device 203, a processor 204, and an interface device 205, etc., which are mutually connected by a bus B.

[0018] A program for realizing the processing in the information processing apparatus 20 is provided by a recording medium 201 such as a CD-ROM. When the recording medium 201 storing the program is set in the drive device 200, the program is installed from the recording medium 201 via the drive device 200 into the auxiliary storage device 202. However, the installation of the program does not necessarily have to be performed from the recording medium 201, and it may be downloaded from another computer via a network. The auxiliary storage device 202 stores the installed program and also stores necessary files, data, etc.

[0019] When an instruction to start the program is given, the memory device 203 reads out and stores the program from the auxiliary storage device 202. The processor 204 is a CPU, a GPU (Graphics Processing Unit), or both a CPU and a GPU, and executes the functions related to the information processing apparatus 20 according to the program stored in the memory device 203. The interface device 205 is used as an interface for connecting to a network.

[0020] FIG. 4 is a diagram showing a functional configuration example of the information processing system in the first embodiment. In FIG. 4, the image processing apparatus 10 includes an authentication unit 121, a reception unit 122, a reading control unit 123, a user feature acquisition unit 124, a conversion request transmission unit 125, a conversion result reception unit 126, and an output unit 127. Each of these units is realized by one or more programs installed in the image processing apparatus 10 being executed by the CPU 111. The image processing apparatus 10 also uses a user information storage unit 151 and an image storage unit 152. Each of these storage units can be realized, for example, using an HDD 114 or a storage device that can be connected to the image processing apparatus 10 via a network.

[0021] The authentication unit 121 authenticates the user who uses the image processing apparatus 10. The user information storage unit 151 is referred to during authentication. The user information storage unit 151 stores information necessary for authenticating a user who can use the image processing apparatus 10, the attribute information of the user, and the like.

[0022] The reception unit 122 receives instructions from the user. For example, the reception unit 122 receives from the user instructions regarding reading (scanning) of image data from a document, information indicating conversion processing to be performed on the text (character string) included in the image (scanned image), and the like.

[0023] The reading control unit 123 controls the reading of image data from a document according to the user's instructions. The reading control unit 123 stores the image data in the image storage unit 152.

[0024] The user feature acquisition unit 124 acquires feature information of the user of the image processing apparatus 10 (hereinafter referred to as "user feature information"). The user feature information refers to information indicating the features of the user. The features of the user refer to the characteristics of the user that can affect the understanding of the text. For example, the department to which the user belongs, the language used, the age (age group), etc. are examples of information constituting the features of the user. However, what is regarded as the user's features may be appropriately selected according to how the information processing system is to be operated.

[0025] The conversion request transmission unit 125 transmits a conversion request including the scanned image, the user feature information, and information indicating the conversion processing to the information processing apparatus 20 via the network.

[0026] The conversion result reception unit 126 receives from the information processing apparatus 20 the result of the conversion processing (conversion result) for the text included in the scanned image.

[0027] The output unit 127 outputs the conversion result received by the conversion result reception unit 126.

[0028] On the one hand, the information processing device 20 includes a conversion request receiving unit 21, a character recognition unit 22, a conversion unit 23, a large language model 24, and a conversion result transmitting unit 25. Each of these units is realized by the processing executed by one or more programs installed in the information processing device 20 on the processor 204. The information processing device 20 also uses a text storage unit 26. The text storage unit 26 can be realized, for example, using an auxiliary storage device 202 or a storage device that can be connected to the information processing device 20 via a network, etc.

[0029] The conversion request receiving unit 21 receives the conversion request transmitted from the conversion request transmitting unit 125. That is, the conversion request receiving unit 21 receives from the image processing device 10 the image data (scanned image) read from the document according to the user's instruction, the user characteristic information, and the information indicating the conversion process to be executed on the text included in the image data.

[0030] The character recognition unit 22 extracts text from the image data (scanned image) related to the conversion request. The character recognition unit 22 extracts, for example, a character string (text) indicating the text by applying character recognition (OCR) to the scanned image. The character recognition unit 22 saves the extracted text in the text storage unit 26. Note that the character recognition unit 22 may be realized by an external information processing device separate from the information processing device 20 (for example, an OCR server having dictionary data).

[0031] The conversion unit 23 executes the conversion process related to the conversion request on the text extracted by the character recognition unit 22, and inputs information (prompt) including as an instruction that the result of the conversion process is suitable for the person related to the user characteristic information related to the conversion request to the large language model 24, and obtains the conversion result (text as) output by the large language model 24. Here, "being suitable for the person related to the user characteristic information" means making it easier to understand (easier to read) according to the person related to the user characteristic information. For example, it is to summarize, translate, or convert the text according to the user's age group, used language, and affiliated department.

[0032] The large language model 24 is the large language model described above. For example, ChatGPT may be used as the large language model 24. The large language model 24 may not be possessed by the information processing apparatus 20 and may be called from the information processing apparatus 20 via a network.

[0033] The conversion result transmission unit 25 transmits the conversion result acquired by the conversion unit 23 to the image processing apparatus 10.

[0034] Hereinafter, the processing procedure executed by the information processing system will be described. FIG. 5 is a sequence diagram for explaining an example of the processing procedure executed by the information processing system.

[0035] In step S101, the authentication unit 121 of the image processing apparatus 10 receives a login operation from the user via the login screen displayed on the operation panel 15. By the login operation, information necessary for user authentication, such as a user ID and a password, is input as login information. Note that information used by another known authentication method, such as biometric information, may be input as login information.

[0036] Subsequently, the authentication unit 121 authenticates the user by comparing the input login information with the information stored in the user information storage unit 151 (S102).

[0037] FIG. 6 is a diagram showing a first configuration example of the user information storage unit 151. The user information storage unit 151 shown in FIG. 6 stores a user ID, a password, a name, an affiliation, a date of birth, etc. for each user permitted to use the image processing apparatus 10.

[0038] The user ID is identification information of the user. The password is the correct password registered for the user related to the user ID. The name is the name of the user related to the user ID. The affiliation is information indicating the organization to which the user related to the user ID belongs (for example, a department name, etc.). The date of birth is the date of birth of the user. The age of the user can be specified based on the date of birth and the current date and time.

[0039] If a record containing a pair of a user ID and a password that matches the input login information is stored in the user information storage unit 151, the authentication unit 121 determines that the authentication is successful, and identifies the user related to the user ID of the login information as the logged-in user.

[0040] Also, FIG. 7 is a diagram showing a second configuration example of the user information storage unit 151. The user information storage unit 151 shown in FIG. 7 is different from FIG. 6 in that it stores biometric information instead of passwords for each user permitted to use the image processing apparatus 10. The biometric information is, for example, data indicating features of a user's face (feature points such as the positions of eyes, mouth, nose, etc.) extracted from image data of the user's face. Other biometric information (fingerprint, vein, etc.) may be used. In this case, in step S101, the authentication unit 121 may acquire the corresponding biometric information as the login information. If a record containing biometric information that matches the acquired login information is stored in the user information storage unit 151, the authentication unit 121 determines that the authentication is successful, and identifies the user related to the user ID of the login information as the logged-in user.

[0041] When the authentication is successful, the authentication unit 121 records the content of the record including the user ID in the user information storage unit 151 as the logged-in user information in the RAM 112 and erases the login screen. As a result, the home screen is displayed on the operation panel 15. The home screen is a screen for receiving execution instructions for various processes from the user. If a record containing the user ID and password or biometric information that matches the input login information is not stored in the user information storage unit 151, the authentication unit 121 determines that the authentication has failed. In this case, steps S103 and subsequent steps are not executed.

[0042] Note that in the present embodiment, the authentication is executed to acquire user characteristic information. Therefore, if user characteristic information can be acquired by other methods, the authentication does not necessarily have to be executed.

[0043] Subsequently, the reception unit 122 receives a conversion instruction from the user via the home screen (S103). In the conversion instruction, settings related to reading of image data from the document (hereinafter referred to as "scan settings"), information indicating a desired conversion process for the text included in the image data (hereinafter referred to as "conversion information"), etc. are input. In consideration of reducing the operation burden on the user, the conversion process may be selectable from a plurality of predetermined candidates (for example, "summarization", "translation", etc.). In this case, a character string indicating the selected conversion process (for example, "summarization", "translation", etc.) becomes the conversion information. Alternatively, the conversion information may be input in natural language. For example, "Translate into English", "Summarize within 400 characters", etc. may be input as the conversion information.

[0044] Subsequently, the reading control unit 123 controls the scanner 12 according to the scan settings input in the conversion instruction, thereby controlling the reading of the scanned image from the document (S104). The reading control unit 123 stores the scanned image in the image storage unit 152 and generates identification information such as a URL indicating the storage destination of the scanned image (hereinafter referred to as "image ID").

[0045] Subsequently, the user feature acquisition unit 124 acquires user feature information of the logged-in user (S105). For example, among the logged-in user information stored in the RAM 112, the affiliation and date of birth may be acquired as information constituting the user feature information. At this time, the user feature acquisition unit 124 may convert the date of birth into age or age group based on the current date and time. If the large language model 24 has already learned the preference for articles for each user (for each name), the name may also be included in the user feature information. The user feature acquisition unit 124 may also acquire the language setting for the operation panel 15 as information constituting the user feature information. The language setting for the operation panel 15 refers to the setting regarding which language is used as the display language of the operation panel 15. The user is likely to select a language that is easy for themselves to understand. Therefore, it can be said that the language setting is information indicating a language that the user is likely to understand. Also, in the present embodiment, the operation panel 15 functions as a user interface of the information processing system. Therefore, the language setting for the operation panel 15 is an example of the setting information regarding the display language of the information processing system. If the image processing apparatus 10 is equipped with a camera, the user feature acquisition unit 124 may further acquire information constituting the user feature information from the user's image data (image data including the user as a subject) captured (input) by the camera. For example, the user feature acquisition unit 124 may use a model that has learned the relationship between the image data and the user's features (age, race, etc.) to acquire such features from the image data obtained from the camera.

[0046] Subsequently, the conversion request transmission unit 125 transmits a conversion request including the scanned image, the user feature information acquired by the user feature acquisition unit 124, and the conversion information to the information processing apparatus 20 (S106).

[0047] When the conversion request reception unit 21 of the information processing apparatus 20 receives the conversion request, it requests the character recognition unit 22 to perform character recognition on the scanned image included in the conversion request (S107).

[0048] The character recognition unit 22 applies character recognition to the scanned image to extract a character string as a sentence from the scanned image, and saves the sentence (hereinafter referred to as the "target sentence") in the sentence storage unit 26 (S108). The character recognition unit 22 generates identification information such as a URL indicating the storage destination of the target sentence (hereinafter referred to as the "target sentence ID"). Subsequently, the character recognition unit 22 transmits the target sentence ID and the target sentence to the conversion request receiving unit 21 (S109).

[0049] Subsequently, the conversion request receiving unit 21 transmits a conversion request including the target sentence ID, the target sentence, user characteristic information, and conversion information to the conversion unit 23 (S110).

[0050] The conversion unit 23 generates a prompt for the large language model 24 in response to the conversion request (S111). A prompt refers to text information indicating an instruction for the large language model 24. The conversion unit 23 generates a prompt including, as an instruction, performing a conversion process related to the conversion information on the target sentence and that the result of the conversion process is suitable for the person related to the user characteristic information related to the conversion request. For example, a template of the prompt may be prepared in advance. In this case, the conversion unit 23 may generate a prompt by fitting the target sentence, the user characteristic information, and the conversion information to the template.

[0051] FIG. 8 is a diagram showing an example of a template of a prompt. In FIG. 8, (1) shows the template of the entire prompt. Parts other than those surrounded by < > are fixed sentences. <Sentence> is a part to be replaced by the target sentence to be converted. <Conversion information> is a part to be replaced by the conversion information. <User characteristic information> is a part to be replaced by the user characteristic information.

[0052] (2) in Fig. 8 is an example of a template for user characteristic information. That is, the conversion unit 23 may apply the result of replacing <department>, <age>, and <language setting> in (2) with the department, age, and language setting (language setting for the operation panel 15) based on the user characteristic information to <user characteristic information> in (1). Note that including the department in the user characteristic information can increase the possibility of obtaining a conversion result in which the use of jargon in a department different from the department to which the user belongs is restricted. Also, by including the language setting in the user characteristic information, when translation is performed, translation into a language that the user can understand can be executed even if the target language is not specified, and when summarization is performed, it can be expected that the summary text will be automatically translated into the language.

[0053] Subsequently, the conversion unit 23 causes the large language model 24 to execute a conversion process (the conversion process indicated by the conversion information) such that a conversion result suitable for the person related to the user characteristic information can be obtained for the target text by inputting the prompt to the large language model 24 (S113). Subsequently, the conversion unit 23 acquires the output information from the large language model 24 as the conversion result for the target text (S114). Subsequently, the conversion unit 23 transmits the target text ID and the conversion result to the conversion result transmission unit 25 (S115). The conversion result transmission unit 25 transmits the target text ID and the conversion result to the image processing apparatus 10 (S116).

[0054] When the conversion result receiving unit 126 of the image processing apparatus 10 receives the target text ID and the conversion result, the output unit 127 executes output processing for the conversion result (S117). The output form is not limited to a predetermined one. The output unit 127 may cause the image processing apparatus 10 to print the conversion result, may display the conversion result on the operation panel 15, may store the conversion result in the storage device, or may convert the conversion result into a PDF and transmit it via the network.

[0055] Note that the output unit 127 may convert information indicating the storage destination of the scanned image or the target text, such as the image ID of the scanned image or the target text ID, into a two-dimensional code, and output the two-dimensional code together with the conversion result. For example, when the conversion result is printed, the output unit 127 may control the image processing apparatus 10 so that the two-dimensional code is printed on the printing paper together with the conversion result. Also, when the conversion result is converted into a PDF, the output unit 127 may synthesize the two-dimensional code with the conversion result. By using the two-dimensional code to access the scanned image and the target text, the user can easily confirm the content before conversion for the conversion result.

[0056] Further, the output unit 127 may identify words with relatively high appearance frequencies in the text that is the conversion result (for example, a summary text, a translation result, etc.), and add a predetermined character decoration (color or font) to the corresponding words. By doing so, the user can easily identify the key parts in the conversion result.

[0057] Also, the image processing apparatus 10 may be configured to receive an input of an evaluation of the conversion result from a user who has referred to the conversion result. For example, the user may input the expected conversion result. In this case, the image processing apparatus 10 transmits the conversion result expected by the user to the information processing apparatus 20. The information processing apparatus 20 may perform additional learning of the large language model 24 using the conversion result.

[0058] Note that in the above, an example where there is one large language model 24 has been described. However, for example, large language models 24 that have learned specialized terms specific to each department (affiliation) may be prepared for each department. In this case, in step S112, the conversion unit 23 may input a prompt to the large language model 24 corresponding to the affiliation included in the user characteristic information.

[0059] As described above, according to the first embodiment, conversion processing (arrangement) instructed by the user can be performed on the text included in the image data read from the document. At this time, the conversion processing is executed in consideration of the characteristics of the user. As a result, the possibility of obtaining a conversion result suitable for each user can be increased. Therefore, it is possible to assist the work for understanding the text included in the image data read from the document.

[0060] For example, by performing summarization using words that are easy for the user to understand, the user can understand the content of the text included in the document (for example, minutes) without reading the entire text.

[0061] In addition, by performing translation into a language suitable for the user, the user can understand the content of the document more accurately and quickly.

[0062] Next, a second embodiment will be described. In the second embodiment, differences from the first embodiment will be described. Therefore, points not particularly mentioned may be the same as those in the first embodiment.

[0063] FIG. 9 is a diagram showing a functional configuration example of the information processing system in the second embodiment. In FIG. 9, the same parts as those in FIG. 4 are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0064] In FIG. 9, the image processing apparatus 10 has a character recognition unit 22 and a text storage unit 26. In this case, for example, between the execution of step S104 and the execution of step S106, processing similar to step S108 is executed in the image processing apparatus 10, and text is extracted from the scanned image. The conversion request in step S106 includes the text instead of the scanned image. The information processing apparatus 20 that has received the conversion request executes steps S110 and subsequent steps without executing steps S107 to S109.

[0065] In this way, the character recognition processing may be performed outside or inside the image processing apparatus 10.

[0066] Next, a third embodiment will be described. In the third embodiment, differences from the first embodiment will be described. Therefore, points not particularly mentioned may be the same as those in the first embodiment.

[0067] FIG. 10 is a diagram showing a functional configuration example of the information processing system according to the third embodiment. In FIG. 10, the same parts as those in FIG. 4 are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0068] In FIG. 10, the information processing system further includes an authentication device 30. The authentication device 30 is connected to the image processing device 10 via a network such as a LAN or the Internet. The authentication device 30 has an authentication unit 121 and a user information storage unit 151 that the image processing device 10 had in FIG. 4.

[0069] On the other hand, the image processing device 10 has an authentication request unit 128. The authentication request unit 128 receives the input of login information in step S101 of FIG. 5 and transmits the login information to the authentication device 30. The authentication unit 121 of the authentication device 30 executes an authentication process in the same manner as in step S102 of FIG. 5. When the authentication is successful, the authentication unit 121 transmits, for example, the information stored in the user information storage unit 151 to the authentication request unit 128 for the logged-in user.

[0070] Other processing procedures may be the same as those in the first embodiment. Also, the third embodiment may be combined with the second embodiment.

[0071] Next, a fourth embodiment will be described. In the fourth embodiment, differences from the first embodiment will be described. Therefore, points not particularly mentioned may be the same as those in the first embodiment.

[0072] FIG. 11 is a diagram showing a functional configuration example of the information processing system according to the fourth embodiment. In FIG. 11, the same parts as those in FIG. 4 are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0073] In FIG. 11, the image processing apparatus 10 has all the functions that the information processing apparatus 20 had in FIG. 4. That is, the image processing apparatus 10 alone constitutes an information processing system. Therefore, in the fourth embodiment, all the steps of FIG. 5 are executed in the image processing apparatus 10.

[0074] Note that each function of the above-described embodiments can be realized by one or a plurality of processing circuits. Here, the "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, an ASIC (Application Specific Integrated Circuit) designed to execute each function described above, a DSP (digital signal processor), an FPGA (field programmable gate array), and devices such as conventional circuit modules.

[0075] Also, the device groups described in the above embodiments merely show one of a plurality of computing environments for implementing the embodiments disclosed in this specification.

[0076] In one embodiment, the information processing apparatus 20 includes a plurality of computing devices such as a server cluster. The plurality of computing devices are configured to communicate with each other via an arbitrary type of communication link including a network or a shared memory, and perform the processing disclosed in this specification. Similarly, the image processing apparatus 10 can include a plurality of computing devices configured to communicate with each other.

[0077] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0078] Aspects of the present invention are, for example, as follows. <1> A reading control unit that controls the reading of image data from a manuscript according to a user's instruction, A user feature acquisition unit that acquires the user's feature information, A reception unit that receives from the user information indicating a conversion process to be executed on the text included in the image data, A character recognition unit that extracts the text from the image data, Input information including an instruction to execute the conversion process on the text and that the result of the conversion process is suitable for the person related to the feature information into a large language model, and a conversion unit that acquires a conversion result output by the large language model, An output unit that outputs the conversion result, An information processing system characterized by having the above. <2> The conversion process is a summary of the text or a translation of the text, The information processing system according to <1>, characterized by the above. <3> The user feature acquisition unit acquires setting information regarding the language for display of the information processing system as information constituting the feature information, The information processing system according to <1> or <2>, characterized by the above. <4> The output unit outputs information indicating a storage destination of the image data or the text extracted from the image data together with the conversion result, The information processing system according to any one of <1> to <3>, characterized by the above. <5> The output unit adds a predetermined character decoration to words having a relatively high appearance frequency in the text indicated by the conversion result, The information processing system according to any one of <1> to <4>, characterized by the above. <6> The feature information includes information indicating an organization to which the user belongs, The information processing system according to any one of <1> to <5>, characterized by the above. <7> The user feature acquisition unit acquires information constituting the feature information from image data including the user as a subject. The information processing system according to any one of <1> to <6>, characterized by this. <8> The reception unit further receives an evaluation of the user with respect to the conversion result. The information processing system according to claim 1, characterized by this. <9> A reading control unit that controls reading of image data from a document according to a user's instruction; A user feature acquisition unit that acquires the feature information of the user; A reception unit that receives from the user information indicating a conversion process to be executed on the text included in the image data; A conversion request transmission unit that transmits the image data, the feature information, and the information indicating the conversion process to an information processing apparatus via a network; A conversion result reception unit that executes the conversion process on the text extracted from the image data and receives from the information processing apparatus a conversion result from a large language model including as an instruction that the conversion result is suitable for a person related to the feature information; An output unit that outputs the conversion result; An image processing apparatus, characterized by having these. <10> An information processing apparatus connected to an image processing apparatus via a network, A conversion request reception unit that receives from the image processing apparatus image data read from a document according to a user's instruction, the feature information of the user, and information indicating a conversion process to be executed on the text included in the image data; A character recognition unit that extracts the text from the image data; A conversion unit that inputs information including as an instruction to execute the conversion process on the text and that the conversion result is suitable for a person related to the feature information to a large language model, and acquires a conversion result output by the large language model; A conversion result transmission unit that transmits the conversion result to the image processing apparatus An information processing apparatus characterized by including <11> A reading control unit that controls reading of image data from a document according to a user's instruction A user feature acquisition procedure for acquiring the user's feature information A reception procedure for receiving from the user information indicating conversion processing to be performed on the text included in the image data A conversion request transmission procedure for transmitting the image data, the feature information, and the information indicating the conversion processing to an information processing apparatus via a network Performing the conversion processing on the text extracted from the image data, and receiving from the large language model a conversion result including as an instruction that the result of the conversion processing is suitable for the person related to the feature information, and receiving the conversion result from the information processing apparatus An output procedure for outputting the conversion result An information processing method characterized by a computer executing <12> An information processing apparatus connected to an image processing apparatus via a network A conversion request reception procedure for receiving from the image processing apparatus the image data read from a document according to a user's instruction by the image processing apparatus, the user's feature information, and information indicating conversion processing to be performed on the text included in the image data A character recognition procedure for extracting the text from the image data Performing the conversion processing on the text, and inputting to the large language model information including as an instruction that the result of the conversion processing is suitable for the person related to the feature information, and acquiring the conversion result output by the large language model A conversion result transmission procedure for transmitting the conversion result to the image processing apparatus An information processing method characterized by executing <13> A reading control procedure for controlling reading of image data from a document according to a user's instruction A user feature acquisition procedure for acquiring the feature information of the user, A reception procedure for receiving from the user information indicating a conversion process to be performed on the text included in the image data, A conversion request transmission procedure for transmitting the image data, the feature information, and the information indicating the conversion process to an information processing apparatus via a network, A conversion result reception procedure for receiving from the information processing apparatus a conversion result from a large language model that has executed the conversion process on the text extracted from the image data and includes as an instruction that the result of the conversion process is suitable for the person related to the feature information, An output procedure for outputting the conversion result, A program characterized by causing a computer to execute the above. <14> To an information processing apparatus connected to an image processing apparatus via a network, A conversion request reception procedure for receiving from the image processing apparatus the image data read from a document in response to a user instruction, the feature information of the user, and information indicating a conversion process to be performed on the text included in the image data, A character recognition procedure for extracting the text from the image data, A conversion procedure for inputting information including an instruction to perform the conversion process on the text and that the result of the conversion process is suitable for the person related to the feature information to a large language model and obtaining a conversion result output by the large language model, A conversion result transmission procedure for transmitting the conversion result to the image processing apparatus, A program characterized by causing the above to be executed.

Explanation of Signs

[0079] 10 Image processing apparatus 11 Controller 12 Scanner 13 Printer 14 Modem 15 Operation panel 16 Network interface 17 SD Card Slot 20 Information Processing Device 21 Conversion Request Receiving Unit 22 Character Recognition Unit 23 Conversion Unit 24 Large Language Model 25 Conversion Result Sending Unit 26 Text Memory Unit 30 Authentication Device 80 SD Card 111 CPU 112 RAM 113 ROM 114 HDD 115 NVRAM 121 Authentication Unit 122 Reception Unit 123 Reading Control Unit 124 User Feature Acquisition Unit 125 Conversion Request Sending Unit 126 Conversion Result Receiving Unit 127 Output Unit 128 Authentication Request Unit 151 User Information Memory Unit 152 Image Memory Unit 200 Drive Device 201 Recording Medium 202 Auxiliary Storage Device 203 Memory Device 204 Processor 205 Interface Device B Bus

Prior Art Documents

Patent Documents

[0080]

Patent Document 1

Claims

1. A reading control unit that controls reading of image data from a manuscript according to a user's instruction; A user feature acquisition unit that acquires the user's feature information; A reception unit that receives from the user information indicating a conversion process to be performed on the text included in the image data; A character recognition unit that extracts the text from the image data; A conversion unit that inputs information including an instruction to perform the conversion process on the text and that the result of the conversion process is suitable for the person related to the feature information into a large language model, and acquires a conversion result output by the large language model; An output unit that outputs the conversion result; An information processing system characterized by including the above.

2. The conversion process is a summary of the text or a translation of the text. The information processing system according to claim 1, characterized by the above.

3. The user feature acquisition unit acquires setting information regarding the language for display of the information processing system as information constituting the feature information. The information processing system according to claim 1, characterized by the above.

4. The output unit outputs information indicating a storage destination of the image data or the text extracted from the image data together with the conversion result. The information processing system according to claim 1, characterized by the above.

5. The output unit adds a predetermined character decoration to words having a relatively high appearance frequency in the text indicated by the conversion result. The information processing system according to claim 1, characterized by the above.

6. The feature information includes information indicating an organization to which the user belongs. The information processing system according to claim 1, characterized by the above.

7. The user feature acquisition unit acquires information constituting the feature information from image data including the user as a subject. The information processing system according to claim 1, characterized by the above.

8. The reception unit further receives an evaluation of the user with respect to the conversion result. The information processing system according to claim 1, characterized by the above.

9. A reading control unit that controls reading of image data from a manuscript according to a user's instruction; A user feature acquisition unit that acquires the user's feature information; A reception unit that receives from the user information indicating a conversion process to be performed on the text included in the image data; A conversion request transmission unit that transmits the image data, the feature information, and the information indicating the conversion process to an information processing device via a network; Performing the conversion process on the text extracted from the image data, and receiving, from a large language model that includes, as an instruction, information indicating that the result of the conversion process is suitable for the person related to the feature information, a conversion result from the information processing apparatus; An output unit that outputs the conversion result; An image processing apparatus characterized by comprising the above.

10. An information processing apparatus connected to an image processing apparatus via a network, A conversion request receiving unit that receives, from the image processing apparatus, image data read from a document according to a user's instruction, the user's feature information, and information indicating a conversion process to be performed on the text included in the image data; A character recognition unit that extracts the text from the image data; Performing the conversion process on the text, and inputting, into a large language model, information including, as an instruction, information indicating that the result of the conversion process is suitable for the person related to the feature information, and obtaining a conversion result output by the large language model; A conversion result transmitting unit that transmits the conversion result to the image processing apparatus; An information processing apparatus characterized by comprising the above.

11. A reading control procedure for controlling the reading of image data from a document according to a user's instruction; A user feature acquisition procedure for acquiring the user's feature information; A reception procedure for receiving, from the user, information indicating a conversion process to be performed on the text included in the image data; A conversion request transmission procedure for transmitting the image data, the feature information, and the information indicating the conversion process via a network to an information processing apparatus; Performing the conversion process on the text extracted from the image data, and receiving, from an information processing apparatus, a conversion result from a large language model that includes, as an instruction, information indicating that the result of the conversion process is suitable for the person related to the feature information; An output procedure for outputting the conversion result; An information processing method characterized in that a computer executes the above.

12. An information processing apparatus connected to an image processing apparatus via a network, A conversion request receiving procedure for receiving, from the image processing apparatus, image data read from a document according to a user's instruction, the user's feature information, and information indicating a conversion process to be performed on the text included in the image data; A character recognition procedure for extracting the text from the image data; Input information including an instruction to execute the conversion process on the text and that the result of the conversion process is suitable for the person related to the feature information into a large language model, and obtain a conversion result output by the large language model; a conversion procedure A conversion result transmission procedure for transmitting the conversion result to the image processing apparatus; An information processing method characterized by executing the above.

13. A reading procedure for controlling reading of image data from a manuscript according to a user's instruction; A user feature acquisition procedure for acquiring the user's feature information; A reception procedure for receiving from the user information indicating a conversion process to be executed on the text included in the image data; A conversion request transmission procedure for transmitting the image data, the feature information, and the information indicating the conversion process to an information processing apparatus via a network; Execute the conversion process on the text extracted from the image data, and receive from the information processing apparatus a conversion result from a large language model that inputs information including an instruction that the result of the conversion process is suitable for the person related to the feature information; a conversion result reception procedure An output procedure for outputting the conversion result; A program characterized by causing a computer to execute the above.

14. In an information processing apparatus connected to an image processing apparatus via a network, A conversion request reception procedure for receiving from the image processing apparatus the image data read from a manuscript according to a user's instruction by the image processing apparatus, the user's feature information, and information indicating a conversion process to be executed on the text included in the image data; A character recognition procedure for extracting the text from the image data; Execute the conversion process on the text, and input information including an instruction that the result of the conversion process is suitable for the person related to the feature information into a large language model, and obtain a conversion result output by the large language model; a conversion procedure A conversion result transmission procedure for transmitting the conversion result to the image processing apparatus; A program characterized by causing the above to be executed.

Images