Programs, information processing devices, methods, and systems

Abstract

To improve the accuracy of updating web page content when web pages are automated. [Solution] This is a program to be executed by a computer equipped with a processor and memory. The program causes the processor to perform the following steps: a first acquisition step of acquiring first information regarding the content of a web page and second information regarding content that is scheduled to be updated; a first output step of inputting the acquired first and second information into a first generating AI model and causing the first generating AI model to output third information regarding the operation content of the web page; and an update step of operating the web page and updating the content based on the outputted third information.

Description

Technical Field

[0001] This disclosure relates to programs, information processing apparatuses, methods, and systems.

Background Art

[0002] Conventionally, technologies for automatically operating web pages have been researched and developed. For example, in Patent Document 1, among the input answers included in the answer generation data stored in the database, an input answer corresponding to a specified category and similar to a specified question among the answered questions is extracted as a reference answer, and the reference answer is input into an answer generation model to output an answer to the specified question. The technology disclosed in Patent Document 1 inputs the answer output by the answer generation model into an input form.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the technology disclosed in Patent Document 1 had room for improvement in the update accuracy of the content on the web page when the web page was automatically operated.

[0005] An object of this disclosure is to improve the update accuracy of the content on the web page when the web page is automatically operated.

Means for Solving the Problems

[0006] To solve the aforementioned problems, a program according to one aspect of this disclosure is a program to be executed by a computer having a processor and memory. The program causes the processor to execute: a first acquisition step of acquiring first information relating to the content of a web page and second information relating to content that is scheduled to be updated; a first output step of inputting the acquired first and second information into a first generating AI model and causing the first generating AI model to output third information relating to the operation content of the web page; and an update step of operating the web page and updating the content based on the output third information. [Effects of the Invention]

[0007] According to this disclosure, it is possible to improve the accuracy of updating content on a web page when the web page is operated automatically. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing an example of the overall configuration of System 1. [Figure 2] This block diagram shows an example configuration of the terminal device 10 shown in Figure 1. [Figure 3] This block diagram shows an example configuration of server 20 as shown in Figure 1. [Figure 4] Figure 3 shows the data structure of the tag management table 2021. [Figure 5] Figure 3 shows the data structure of the Input Schedule Table 2022. [Figure 6] This flowchart shows an example of the operation of the terminal device 10 when presenting an input form after automatic input. [Figure 7] This is a schematic diagram showing an example of the screen of display 141 when the input form after automatic input is displayed. [Figure 8] This is a schematic diagram showing other screen examples of display 141 when the input form after automatic input is displayed. [Figure 9]A block diagram showing the basic hardware configuration of Computer 90. [Modes for carrying out the invention]

[0009] The embodiments of this disclosure will be described below with reference to the drawings. In all the drawings illustrating the embodiments, common components are denoted by the same reference numerals, and repeated explanations are omitted. The following embodiments are not intended to unduly limit the content of this disclosure as described in the claims. Not all components shown in the embodiments are necessarily essential components of this disclosure. Also, each drawing is a schematic diagram and is not necessarily a strict illustration.

[0010] [1. Overview] The system according to this embodiment acquires first information regarding an input form displayed on a web page and second information regarding the items to be entered into the input form. The system according to this embodiment inputs the acquired first and second information into a multimodal generation AI model and causes the multimodal generation AI model to output third information regarding the operation content of the web page. Based on the output third information, the system according to this embodiment operates the web page and enters the aforementioned items into the input form.

[0011] [2. Overall System Configuration] Figure 1 is a block diagram showing an example of the overall configuration of System 1. System 1 is a system for providing an automatic input service (hereinafter referred to as the automatic input service) for input forms displayed on a web page. There are no particular limitations on the types of input forms to which the automatic input service can be applied; the automatic input service can be applied to various types of input forms.

[0012] Note that system 1 is not only applicable to the automatic input service, but can also be applied to services that automatically update various types of content on web pages. Examples of web page content include, for example, still images, videos, audio, text, etc. There are no particular limitations on each type of still image, video, audio, text, etc. that can be updated by system 1.

[0013] Also, the types of content for which system 1 can provide services are not limited to one type. System 1 can also be applied to services that automatically update multiple types of content, for example.

[0014] The system 1 shown in FIG. 1 includes, for example, a terminal device 10, a server 20, and an AI system 30. The terminal device 10, the server 20, and the AI system 30 are communicatively connected via, for example, a network 80.

[0015] In FIG. 1, an example in which system 1 includes one terminal device 10 is shown. However, for example, system 1 may include two or more terminal devices 10.

[0016] In FIG. 1, an example in which system 1 includes one server 20 is shown. However, for example, an aggregate of multiple devices may be regarded as one server 20. The way of distributing the multiple functions required to implement the server 20 to one or more hardware can be appropriately determined according to the processing capabilities of each hardware and / or the specifications required for the server 20, etc. Also, in FIG. 1, an example in which the server 20 is independent of the AI system 30 is shown. However, the AI system 30 may include the functions of the server 20.

[0017] In FIG. 1, an example in which the system 1 includes one AI system 30 is shown. However, the number of AI systems 30 included in the system 1 may be two or more. Also, in FIG. 1, an example in which the AI system 30 is independent of the server 20 is shown. However, the server 20 may include the functions of the AI system 30. That is, the server 20 may store the first generation AI model (details will be described later) included in the AI system 30.

[0018] In the present embodiment, an application 181 (details will be described later) is installed in the terminal device 10. Therefore, for example, when the terminal device 101 stores the first generation AI model, the server 20 and the AI system 30 may not be necessary, and the terminal device 10 alone can provide an automatic input service.

[0019] The terminal device 10 is, for example, an information processing device operated by a user. The user is, for example, a user of the automatic input service. The terminal device 10 is realized by, for example, a mobile terminal such as a smartphone or a tablet. In the present embodiment, it is assumed that the terminal device 10 is a smartphone. The terminal device 10 may also be realized by, for example, a stationary PC (Personal Computer), a laptop PC, or the like.

[0020] The terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, a storage 16, and a processor 19. The input device 13 is a device for receiving an input operation from the user (for example, a pointing device such as a touch panel, a touch pad, a mouse, etc., a keyboard, etc.). The output device 14 is a device for presenting information to the user (a display, a speaker, etc.). In the present embodiment, it is assumed that the terminal device 10 includes a touch panel in which the input device 13 and the output device 14 are integrated.

[0021] Server 20 is, for example, an information processing device for managing and operating an automatic input service, and is implemented by a computer connected to network 80. As shown in Figure 1, Server 20 includes a communication IF 22, an input / output IF 23, memory 25, storage 26, and a processor 29. The input / output IF 23 functions as an interface for an input device to receive input operations from the administrator of the automatic input service (hereinafter abbreviated as administrator) and an output device to output information to the administrator.

[0022] The AI ​​system 30 is, for example, a cloud server having a first generation AI model. The first generation AI model is, for example, one or more first multimodal generation AI models. The first multimodal generation AI model can process multiple types of data at once, such as still images, videos, audio, text, and numerical data. The first multimodal generation AI model is an example of a multimodal generation AI model according to one aspect of this disclosure.

[0023] There are no particular limitations on the type of first multimodal generation AI model included in the AI ​​system 30, but in this embodiment, the AI ​​system 30 is assumed to have a VLM (Vision Language Model) as the first multimodal generation AI model. The VLM understands both images and text and processes them integrally. The VLM then generates output data that correlates the images and text.

[0024] The AI ​​system 30 may include, for example, one or more LLMs (Large Language Models) as first-generation AI models. An LLM is a single-modal natural language model built by learning from large amounts of text data, and is used for many NLG (Natural Language Generation) tasks such as generating responses to specific questions, automatically generating sentences, and summarizing text. Examples of LLMs include the following:

[0025] • OpenAI: GPT-4 Google: Gemini 1.5 Flash ·Anthropic: Claude 3.5 Sonnet.

[0026] If the first generative AI model is one or more LLMs, both the first and second information consist solely of text.

[0027] The AI ​​system 30 inputs the first and second pieces of information, along with prompts, into the first multimodal generation model, and causes the first multimodal generation AI to output the third piece of information.

[0028] The first type of information is information about an input form displayed on a web page. This includes, for example, the page source (HTML code, etc.) of the web page displaying the input form, and screenshots / screen captures of the input form.

[0029] The second type of information concerns the items to be entered into the input form (hereinafter referred to as "planned input items"). This second type of information includes, for example, the URL of the webpage displaying the input destination (input form) for the planned input items (Uniform Resource Locator), a configuration file or CSV file recording the planned input items, an image showing the planned input items, text information or PDF data of the planned input items, etc. Details of the planned input items will be described later.

[0030] The third type of information concerns the operation of a web page, specifically, information concerning input operations on input forms. This third type of information includes, for example, the specification of HTML code elements and attributes using XPath or CSS selectors, the operation method (input, click, tap, etc.), and the items entered into the input form (input text, input value, etc.). The input items are those that have been transformed from planned input items into a form suitable for the input format and items of the input form through the first multimodal generation AI model.

[0031] When System 1 handles content other than input forms, the first information may be any information related to the content of the web page, and the second information may be any information related to the content that is scheduled to be updated (hereinafter referred to as "scheduled update content"). In this case, the first information may include, for example, the page source of the web page displaying the content, or screenshots / screen captures of the content. Depending on the type of content, the first information may not include the page source. The second information may include, for example, a configuration file or CSV file recording the scheduled update content, an image showing the scheduled update content, text information or PDF data of the scheduled update content, or the details of the update work (such as transitioning a still image screen display to a still image containing the scheduled update content, or editing a video to include the scheduled update content).

[0032] Herein, "updating content" as used herein does not refer only to altering the content itself, such as changing the input format of an input form, changing, deleting, or adding items to an input form, or editing still images, videos, or text. Rather, "updating content" as used herein also includes enabling some or all of the content's functions, such as filling out an input form, displaying a still image, or playing a video.

[0033] The prompts input to the first multimodal generation AI model are instruction statements that instruct the output of third information, and are hardcoded, for example, in a program stored in the AI ​​system 30. For example, if system 1 provides a service that automatically updates multiple types of content, the terminal device 10, server 20, or AI system 30 may have a prompt table that stores prompts for each type of content.

[0034] The AI ​​system 30 transmits the third piece of information output from the first multimodal generation AI model to the server 20.

[0035] Each information processing device, such as the terminal device 10, the server 20, and the AI ​​system 30, is composed of a computer 90 (see Figure 9) equipped with an arithmetic unit and a memory device. The basic hardware configuration of the computer 90 and the basic functional configuration of the computer 90 realized by this basic hardware configuration will be described later. Note that explanations of the terminal device 10, the server 20, and the AI ​​system 30 that overlap with the basic hardware configuration of the computer 90 and the basic functional configuration of the computer will be omitted.

[0036] <2.1 Terminal Device Configuration> Figure 2 is a block diagram showing an example configuration of the terminal device 10 shown in Figure 1. As shown in Figure 2, the terminal device 10 includes a communication unit 120, an input device 13, an output device 14, an audio processing unit 170, a microphone 171, a speaker 172, a camera 160, a position information sensor 150, an acceleration sensor 155, a storage unit 180, and a control unit 190. Each block included in the terminal device 10 is electrically connected, for example, by a bus.

[0037] The communication unit 120 performs processing such as modulation and demodulation processing for the terminal device 10 to communicate with an external device (for example, a server 20). The communication unit 120 performs transmission processing on the signal generated by the control unit 190 and transmits it to the external device. The communication unit 120 performs reception processing on the signal received from the external device and outputs it to the control unit 190.

[0038] The input device 13 is a device for the user to input instructions or information. The input device 13 can be implemented, for example, by a touch-sensitive device 131 on which instructions are input by touching the operating surface. If the terminal device 10 is a PC or the like, the input device 13 may be implemented by a reader, keyboard, mouse, etc. The input device 13 converts the instructions input by the user into electrical signals and outputs them to the control unit 190. The input device 13 may also include, for example, a receiving port that accepts electrical signals input from an external input device.

[0039] The output device 14 is a device for presenting information to the user. The output device 14 is implemented, for example, by a display 141. The display 141 displays various information according to the control of the control unit 190. The display 141 is implemented, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0040] The audio processing unit 170 performs, for example, digital-to-analog conversion processing of the audio signal. The audio processing unit 170 converts the signal provided from the microphone 171 into a digital signal and provides the converted signal to the control unit 190. The audio processing unit 170 also provides the audio signal to the speaker 172. The audio processing unit 170 is implemented, for example, by an audio processing processor. The microphone 171 receives an audio input and provides the audio signal corresponding to that audio input to the audio processing unit 170. The speaker 172 converts the audio signal provided from the audio processing unit 170 into audio and outputs the audio to the outside of the terminal device 10.

[0041] Camera 160 is an imaging device that captures images using visible light. In other words, camera 160 is a device that receives visible light using a photodetector and outputs image data as a shooting signal. Camera 160 captures subjects in a certain direction and within a certain shooting range relative to the terminal device 10 and outputs image data as a result of the capture. If camera 160 has a function that allows adjustment of the shooting range, or more precisely, the angle of view, camera 160 also outputs information regarding this angle of view. Such a function is called a zoom function.

[0042] The location information sensor 150 is a sensor that detects the position of the terminal device 10, and is generally a GNSS device, such as a GPS module. A GPS module is a receiving device used in a satellite positioning system. In a satellite positioning system, signals are received from at least three or four satellites, and based on the received signals, the current position of the terminal device 10, which is equipped with a GPS module, is detected in coordinate values. The location information sensor 150 may also detect the current position of the terminal device 10 from the position of a wireless base station to which the terminal device 10 is connected via the communication unit 120.

[0043] The acceleration sensor 155 is a sensor that detects the acceleration applied to the terminal device 10. Preferably, the acceleration sensor 155 has the function of detecting the tilt around each axis (X axis, Y axis, Z axis) of a three-dimensional coordinate system with the position of the terminal device 10 as the origin. An acceleration sensor 155 having such a function can detect the orientation of the terminal device 10, that is, its direction with respect to the X, Y, and Z axes, by detecting the gravitational acceleration of the Earth's gravity.

[0044] The storage unit 180 is implemented by the memory 15 and storage 16 shown in Figure 1, and stores data and programs used by the terminal device 10. The programs include, for example, a web browser application.

[0045] Furthermore, the memory unit 180 stores the application 181. The application 181 is dedicated application software for executing and managing the automatic input service. In this embodiment, the application 181 is assumed to be an RPA (Robotic Process Automation) agent.

[0046] An RPA agent is a software program used to automate the process of entering data into input forms. RPA agents mimic various user actions (such as inputting data, clicking, and reading information on the screen) to automatically perform routine tasks. Examples of operations that RPA agents can mimic include data entry, file movement and transfer, web scraping, and database searches.

[0047] The application 181 may, for example, be stored on the server 20. In this case, the application 181 runs in the background of other applications installed on the server 20. The user can access the application 181 on the server 20 using a web browser application installed on the terminal device 10. Alternatively, the application 181 may be added to the aforementioned web browser application as an extension.

[0048] The control unit 190 is implemented, for example, by the processor 19 reading a program (including an application 181) stored in the memory unit 180 and executing instructions contained in the program. The control unit 190 controls the operation of the terminal device 10. By operating according to the loaded program, the control unit 190 performs the functions of an operation reception unit 191, a transmission / reception unit 192, a presentation control unit 193, and an automatic processing unit 194.

[0049] The operation reception unit 191 processes instructions or information input from the input device 13. Specifically, the operation reception unit 191 receives instructions or information input from the touch-sensitive device 131. The transmission / reception unit 192 processes data for the terminal device 10 to send and receive data with an external device according to a communication protocol. Specifically, the transmission / reception unit 192 transmits instructions or information input from the user to the server 20 or AI system 30. The transmission / reception unit 192 receives information transmitted from the server 20 or AI system 30. The presentation control unit 193 controls the output device 14 to present various information to the user.

[0050] The automated processing unit 194 instructs the AI ​​system 30 to input the first information and the second information into the first multimodal generation AI model. The automated processing unit 194 also operates the web page based on the third information and inputs into the input form. If system 1 handles content other than the input form, the automated processing unit 194 operates the web page based on the third information and updates the content.

[0051] <2.2 Server Configuration> Figure 3 is a block diagram showing an example configuration of the server 20 shown in Figure 1. As shown in Figure 3, the server 20 performs the functions of a communication unit 201, a storage unit 202, and a control unit 203.

[0052] The communication unit 201 performs processing for the server 20 to communicate with an external device (e.g., terminal device 10). The storage unit 202 is implemented by memory 25 and storage 26 and stores data and programs used by the server 20. The programs include application programs such as web browser applications. The storage unit 202 stores, for example, a tag management table 2021 and an input schedule table 2022.

[0053] The tag management table 2021 is a table that stores tags assigned to planned input items. The planned input item table 2022 is a table that stores planned input items. Server 20 reads the planned input items contained in the second information received from the terminal device 10 or AI system 30 and assigns tags to them. Server 20 stores the assigned tags in the tag management table 2021 and stores the planned input items associated with the tags in the planned input item table 2022. Details of the tag management table 2021 and the planned input item table 2022 will be described later.

[0054] The control unit 203 is realized when the processor 29 reads a program stored in the memory unit 202 and executes the instructions contained in that program. The control unit 203 controls the operation of the server 20. By operating according to the loaded program, the control unit 203 performs the functions of the receive control module 2031, the transmit control module 2032, and the presentation control module 2033.

[0055] The receive control module 2031 controls the process by which the server 20 receives signals from an external device according to a communication protocol. The transmit control module 2032 controls the process by which the server 20 transmits signals to an external device according to a communication protocol. The presentation control module 2033 controls the process of presenting various information to the user.

[0056] [3 Data Structure] Figures 4 and 5 show the data structure of the tables stored by server 20. Note that Figures 4 and 5 are merely examples and do not exclude data not shown. Furthermore, even data listed in the same table may be stored in separate memory areas within the storage unit 202.

[0057] Figure 4 shows the data structure of the tag management table 2021. The tag management table 2021 shown in Figure 4 is a table that uses the tag ID as the key and contains tag data, administrator, and latest edit date. Whenever the server 20 reads the planned input items from newly received second information and assigns a tag, it stores information about that tag in the tag management table 2021.

[0058] The "Tag ID" field stores an identifier that uniquely identifies the tag assigned to the input item. The "Tag Data" field stores the tag. Specifically, for example, the "Tag Data" field stores text data such as words or short phrases that serve as tags. There are no particular limitations on the form of text data used as tags. In the example in Figure 4, data format names such as "File...", "List...", and "CSV..." are used as tags. The "Administrator" field stores the name of the administrator of the input item associated with the tag.

[0059] The item "Latest Edit Date" stores the latest date on which the input item associated with the tag was edited. In this embodiment, the server 20, for example, upon receiving a display request from the administrator, displays a list of tags stored in the tag management table 2021 on a display (not shown) on the server 20. For example, when the list of tags is displayed on the display, the server 20, upon receiving a tag selection operation from the administrator, reads the input item associated with the selected tag from the input item table 2022 and displays it on the display. In addition, in this embodiment, the input item stored in the input item table 2022 is editable, and the administrator can edit the input item displayed on the display as needed.

[0060] Furthermore, the tag management table 2021 does not necessarily have to contain at least one of the fields, for example, "Administrator" and "Last Edited Date". Also, for example, if the planned input items stored in the planned input items table 2022 are not editable, the "Last Edited Date" field becomes unnecessary. Moreover, it is not mandatory for the server 20 to display the list of tags and planned input items on the display.

[0061] Figure 5 shows the data structure of the Input Scheduled Items Table 2022. The Input Scheduled Items Table 2022 shown in Figure 5 is a table that has columns for User Information, Tag ID, Tag Data, and Input Scheduled Items, with User ID as the key. Each time Server 20 reads an Input Scheduled Item from newly received Second Information, it stores information about that Input Scheduled Item in the Input Scheduled Items Table 2022.

[0062] The "User ID" field stores an identifier to uniquely identify the user who provided the second information to the server 20. The "User Information" field stores user information (information about the user). There are no particular limitations on the content of the user information; in the example in Figure 5, the name, address, and two types of contact information (telephone number, mobile number, or email address, etc.) are stored in the "User Information" field. User information may, for example, be included in the second information. Alternatively, for example, the user may provide user information to the server 20 separately from the second information.

[0063] The "Tag ID" field stores an identifier that uniquely identifies the tag assigned to each planned input item. In the example in Figure 5, the identifier stored in the "Tag ID" field of the planned input item table 2022 is the same as the identifier stored in the "Tag ID" field of the tag management table 2021, but the two may be different. The "Tag Data" field stores the tag, and is the same as the tag stored in the "Tag Data" field of the tag management table 2021.

[0064] The item "Planned Input Items" is used to store the planned input items for each piece of information. There are no particular limitations on the content or number of planned input items; the server 20 stores various types of planned input items in the "Planned Input Items" item, depending on the type and format of the input form that the user plans to automate.

[0065] Examples of information to be entered include name, address, contact information, company name, company address, company contact information, department, purpose (summary), subject, and inquiry details. Of these examples, name, address, contact information, company name, company address, company contact information, and department may be included in the user information, for example. In other words, the name, address, etc., included as information to be entered in the second information may be the user's name, address, etc.

[0066] In the example in Figure 5, the "Planned Input Items" field in each row of the Input Scheduled Items Table 2022 consists of three types of fields: "Planned Input Items 1," "Planned Input Items 2," and "Planned Input Items 3." For each of the "Planned Input Items" fields in each row of the Input Scheduled Items Table 2022, the content stored in "Planned Input Items 1," "Planned Input Items 2," and "Planned Input Items 3" may be the same as or different from the "Planned Input Items" fields in other rows. Furthermore, the "Planned Input Items" field does not necessarily consist of three types of fields: "Planned Input Items 1," "Planned Input Items 2," and "Planned Input Items 3." For example, the "Planned Input Items" field in each row of the Input Scheduled Items Table 2022 may consist of a different number of fields depending on the type of input form.

[0067] [4 actions] This section describes an example of the operation of the terminal device 10 when presenting an input form after automatic input. Figure 6 is a flowchart showing an example of the operation of the terminal device 10 when presenting an input form after automatic input. As a prerequisite, it is assumed that the user has logged in from the login screen displayed on the display 141 and that the automatic input service function is available.

[0068] In step S11 shown in Figure 6, the terminal device 10 acquires the first information and the second information (first acquisition step).

[0069] Specifically, for example, the operation reception unit 191 receives input operations for second information from the user. As a result, the terminal device 10 acquires the second information. The automatic processing unit 194 searches for a web page based on the URL contained in the second information and acquires the first information from that web page. As a result, the terminal device 10 acquires the first information. Alternatively, the terminal device 10 may acquire the first information by having the operation reception unit 191 receive input operations for first information from the user.

[0070] In step S12, the terminal device 10 inputs the acquired first information and second information into the first generating AI model, causing the first generating AI model to output third information (first output step).

[0071] Specifically, for example, the transmitting / receiving unit 192 transmits the first information to the AI ​​system 30. The transmitting / receiving unit 192 also transmits the second information to the server 20.

[0072] The receiving control module 2031, for example, reads the planned input items from the received second information and assigns tags to them. The receiving control module 2031 stores the assigned tags in the tag management table 2021 and the planned input item table 2022. The receiving control module 2031 also stores the planned input items associated with the assigned tags in the planned input item table 2022.

[0073] The display control module 2033, for example, upon receiving a display request from the administrator, displays a list of tags stored in the tag management table 2021 on the server 20's display. For example, when the list of tags is displayed on the display, the display control module 2033, upon receiving a tag selection operation from the administrator, reads the input schedule items associated with the selected tag from the input schedule item table 2022 and displays them on the display. For example, when the input schedule items are displayed on the display, the display control module 2033, upon receiving an editing operation from the administrator, edits the input schedule items. Then, for example, the display control module 2033 displays the edited input schedule items on the display. For example, the display control module 2033 overwrites the input schedule items stored in the input schedule item table 2022 with the edited input schedule items.

[0074] The transmission control module 2032 transmits, for example, second information containing the edited input items to the AI ​​system 30. On the other hand, if it does not receive a display request from the administrator, the transmission control module 2032 transmits, for example, the second information received from the terminal device 10 to the AI ​​system 30 as is.

[0075] Furthermore, if the input items stored in the input item table 2022 are not editable, the receiving control module 2031 may, for example, send the second information received from the terminal device 10 directly to the AI ​​system 30. Alternatively, for example, the transmitting / receiving unit 192 may send the second information not only to the server 20 but also to the AI ​​system 30. In this case, the server 20 does not need to send the second information to the AI ​​system 30.

[0076] The AI ​​system 30 inputs the received first and second information, along with hardcoded prompts in the program, into the first generating AI model, specifically the first multimodal generating AI model. This causes the terminal device 10 to input the first and second information into the first generating AI model. The AI ​​system 30 then causes the first multimodal generating AI model to output the third information. This causes the terminal device 10 to output the third information from the first generating AI model. The AI ​​system 30 then transmits the third information to the terminal device 10.

[0077] In step S13, the terminal device 10 operates the web page based on the outputted third information and enters information into the input form (update step). If the content is not an input form, the terminal device 10 updates the content. Specifically, the automatic processing unit 194 operates the web page based on the received third information and enters the input items contained in the third information into the input form.

[0078] In step S11, for example, if the acquired second information includes URLs of multiple types of web pages, the terminal device 10 may acquire first information for each of the multiple types of web pages. In this case, in step S12, the terminal device 10 may input the first information for each of the multiple types of web pages into the first generating AI model and have the first generating AI model output third information for each of the multiple types of web pages. Then, in step S13, the terminal device 10 may, for example, operate each of the multiple types of web pages based on this third information and input the appropriate information for each input form. This series of processes can also be applied to content other than input forms.

[0079] As a result, System 1 can input the appropriate information for each of the multiple types of input forms simply by acquiring the second piece of information, enabling it to respond flexibly and adaptably to inputting into various types of input forms.

[0080] In step S14, the terminal device 10 presents the input form after automatic input to the user. Specifically, for example, when the operation reception unit 191 receives a display request from the user, the presentation control unit 193 displays the input form after automatic input on the display 141. However, the presentation control unit 193 may also display the input form after automatic input on the display 141 regardless of whether there is a display request from the user, for example, when the automatic processing unit 194 has finished operations based on the third information.

[0081] If the output device 14 is a printer, the presentation control unit 193 may, for example, print out a paper medium with the automatically entered input form printed on it from the output device 14. Furthermore, it is not mandatory for the presentation control unit 193 to have the output device 14 display the automatically entered input form.

[0082] The following describes examples of the display 141 screen when the input form after automatic input is displayed, with reference to Figures 7 and 8. Figure 7 is a schematic diagram showing an example of the display 141 screen when the input form after automatic input is displayed. Figure 8 is a schematic diagram showing another example of the display 141 screen when the input form after automatic input is displayed.

[0083] In the example shown in Figure 7, for example, the input form 1411 after automatic input is displayed on the display 141. The input form 1411 is in a format where the corresponding input items are directly entered into the fields of each item. The user can also edit the automatically entered items in the input form 1411 as needed. Furthermore, the information entered in the input form 1411 is sent to the management server of the company that has set up the input form 1411 on the web page (○○○ Corporation in Figure 7) when, for example, the submit button 1412 displayed on the display 141 is pressed.

[0084] Specifically, the display control unit 193, for example, when the operation reception unit 191 receives a display request from the user, displays the input form 1411 after automatic input on the display 141. Furthermore, when the operation reception unit 191 receives an editing operation from the user for input items while the input form 1411 after automatic input is displayed on the display 141, the display control unit 193 edits the input items. Then, the display control unit 193, for example, displays the edited input items on the display 141. In addition, when the input form 1411 after automatic input (or after editing) is displayed on the display 141, the transmission / reception unit 192 receives an operation from the user to press the send button 1412 and transmits the information entered in the input form 1411 to the server 20.

[0085] In the example shown in Figure 8, for instance, the input form 1421, after automatic input, is displayed on display 141. Of the items in input form 1421, the items "Purpose 1" and "Purpose 2" are configured so that a pull-down menu appears when the user clicks within the field. In the example shown in Figure 8, "Sales" is selected from the pull-down menu. Furthermore, of the items in input form 1421 other than "Purpose 1" and "Purpose 2," the corresponding input information is entered directly into the field. The user can also edit the automatically entered information in each item other than "Purpose 1" and "Purpose 2" as needed.

[0086] In the display area directly below the input form 1421 on display 141, there is a checkbox 14213 that can be checked to indicate agreement to the privacy policy. In the example shown in Figure 8, checkbox 14213 is checked. The information entered into the input form 1421 is sent to the management server of the company that has set up the input form 1421 on the web page (in Figure 8, "◇◇◇ Co., Ltd.") when, for example, the submit button 14214 displayed on display 141 is pressed.

[0087] Specifically, the display control unit 193, for example, when the operation reception unit 191 receives a display request from the user, displays the input form 1421 after automatic input on the display 141. Furthermore, when the operation reception unit 191 receives an editing operation from the user regarding the input items while the input form 1421 after automatic input is displayed on the display 141, the display control unit 193 edits the input items. The editing operation in the example shown in Figure 8 includes not only direct editing of text in fields, but also re-selection from pull-down menus and checking / unchecking checkboxes. The display control unit 193 then displays the edited input items on the display 141. In addition, when the input form 1211 after automatic input (or editing) is displayed on the display 141, the transmission / reception unit 192 receives an operation from the user to press the send button 1412 and transmits the information entered in the input form 1421 to the server 20.

[0088] Here, the examples shown in Figures 8 and 9 are screen examples where the second information is common and the second information includes both the URL for "○○○ Corporation" and the URL for "◇◇◇ Corporation". As in these examples, System 1 can automatically input common information into multiple types of input forms, even when it is necessary to input common information into each of them. The input information can be adjusted to the appropriate expression and format for each type of input form.

[0089] Please note that the screen examples in Figures 8 and 9 are merely illustrative. Various variations are possible regarding the display content and display modes of input forms 1411 and 1412.

[0090] [5 Summary] As described above, in this embodiment, the operation reception unit 191 receives an input operation for second information from the user. The automatic processing unit 194 searches for a web page based on the URL included in the second information and obtains the first information from the web page. The transmission / reception unit 192 transmits the first information to the AI ​​system 30 and the second information to the server 20. After the reception control module 2031 stores the planned input items included in the second information in the planned input item table 2022, the transmission control module 2032 transmits the second information to the AI ​​system 30. The AI ​​system 30 inputs the first information and the second information into the first multimodal generation AI model and outputs the third information from the first multimodal generation AI model. The AI ​​system 30 transmits the third information to the terminal device 10. The automatic processing unit 194 operates the web page based on the third information and inputs the input items included in the third information into the input form.

[0091] As a result, when a user simply inputs second information into the terminal device 10, the appropriate input items according to the fields and format of the input form are automatically entered. Therefore, the accuracy of input in input forms when a web page is operated automatically can be improved. This also applies when the content on the web page is content other than input forms, such as still images, videos, audio, and text, and the accuracy of updating the content on the web page when a web page is operated automatically can be improved.

[0092] Furthermore, when it is necessary to input common information for each of multiple types of input forms, the input information can be automatically entered by adjusting its expression and format to suit the type of input form. Therefore, by simply having the user input second information into the terminal device 10, the appropriate input information for each of the multiple types of input forms can be automatically entered, enabling flexible and adaptable responses to input into various types of input forms.

[0093] This also applies when the content on a webpage is not an input form, such as still images, videos, audio, or text. In other words, by simply inputting second information into the terminal device 10, the user can update the same type of content (any one of still images, videos, audio, or text) in a manner suitable for each of the multiple content types. This allows for flexible and adaptable responses to updating the same type of content in various forms.

[0094] [6 Variations] System 1 can adopt various configurations other than those of this embodiment. Several examples of modifications of System 1 will be described below.

[0095] <6.1 First Modification Example: Modifying the Input Form After Auto-Filling> The terminal device 10 may, for example, modify the input form after it has been automatically filled in.

[0096] Specifically, for example, the operation reception unit 191 may receive fourth information regarding the content to be modified in the input form after automatic input. As a result, the terminal device 10 may acquire the fourth information (second acquisition step). The transmission / reception unit 192 may, for example, transmit the acquired fourth information to the AI ​​system 30.

[0097] Here, "content to be modified in the input form after automatic input" specifically refers to content to be modified in the input fields (hereinafter referred to as "content to be modified"). There are no particular limitations on the types, formats, etc., of the various information included in the fourth information. The fourth information includes, for example, a configuration file or CSV file that records the content to be modified, an image that displays the content to be modified, text information or PDF data of the content to be modified, etc.

[0098] The AI ​​system 30 may, for example, input the received fourth information along with a prompt to the second generating AI model, causing the second generating AI model to output fifth information regarding the content of the web page modification operation. The prompt input to the second generating AI model is an instruction that instructs the output of the fifth information, and is, for example, hardcoded in a program stored in the AI ​​system 30. As a result, the terminal device 10 may cause the second generating AI model to output the fifth information (second output step). The AI ​​system 30 may, for example, transmit the outputted fifth information to the terminal device 10.

[0099] Here, the second generative AI model is a generative AI model different from the first generative AI model and is included in the AI ​​system 30. The second generative AI model may be, for example, an LLM or a multimodal generative AI model, but in the first modified example, it is assumed to be a VLM.

[0100] The fifth type of information specifically concerns the modification of input items. This fifth type of information includes, for example, the specification of HTML code elements and attributes using XPath or CSS selectors, the method of operation (input, click, tap, delete, etc.), and the re-entered items to be entered into the input form (re-entered text, re-entered value, etc.). The re-entered items are those that have been transformed from the items to be modified through the second multimodal generation AI model into a form suitable for the input format and items of the input form.

[0101] The automated processing unit 194 may, for example, modify the web page based on the outputted fifth information and correct the input form after automatic input. More specifically, the automated processing unit 194 may, for example, correct the input items in the input form after automatic input (in other words, re-enter items) based on the fifth information. As a result, the terminal device 10 may correct the input form after automatic input (correction step).

[0102] The above series of processes can also be applied to content other than input forms. That is, for example, terminal device 10 may acquire fourth information regarding the content that is scheduled to be modified in the updated content. Alternatively, for example, terminal device 10 may input the acquired fourth information into a second generation AI model (VLM) and have the second generation AI model output fifth information. Furthermore, terminal device 10 may modify the web page based on the output fifth information and modify the updated content.

[0103] Thus, according to the first modified example, the user simply inputs the fourth information into the terminal device 10, and the input is automatically corrected to the format desired by the user and to appropriate content according to the items and format of the input form. Alternatively, the user simply inputs the fourth information into the terminal device 10, and the updated content is automatically corrected to the format desired by the user and to appropriate content according to the nature of the content. This further improves input accuracy (update accuracy) while enhancing user convenience.

[0104] <6.2 Second Variation: Page transition before automatic input into the input form> The terminal device 10 may, for example, redirect the user to a web page displaying the input form before automatically filling in the input form.

[0105] Specifically, for example, the automatic processing unit 194 may analyze the first information received as input by the operation reception unit 191 and search for a web page. There are no particular limitations on the method of analyzing the first information; for example, analysis using a known rule-based method or analysis using a known trained model such as a neural network may be employed. As a result, the terminal device 10 may analyze the acquired first information and search for a web page (search step).

[0106] The automatic processing unit 194 may, for example, perform a transition operation to transition to the searched web page. The display control unit 193 may, for example, receive the transition operation from the automatic processing unit 194 and transition from another web page displayed on the display 141 before the transition to the web page searched by the automatic processing unit 194. The display control unit 193 may, for example, display the destination web page (displaying the input form before automatic input) on the display 141. As a result, the terminal device 10 may perform a transition operation to transition to the searched web page and transition to the searched web page, and display the input form on the web page on the screen (first transition step).

[0107] In the second modified example, for instance, the control unit 190 executes the search step and the first transition step by operating according to the loaded dedicated application. The dedicated application is application software for executing and managing the search step and the first transition step, and is installed on the terminal device 10 and stored in the storage unit 1801. The dedicated application may be, for example, a different RPA agent from application 181. Alternatively, application 181, which is an RPA agent, may function as the dedicated application.

[0108] Thus, according to this second modification, the user simply enters the second information into the terminal device 10, and the system automatically transitions to a webpage displaying the input form before automatic input. This allows the user to confirm whether the input form they wish to fill out is subject to automatic input simply by entering the second information into the terminal device 10. Furthermore, it eliminates the need for the user to manually search for the webpage, thus further enhancing user convenience.

[0109] Furthermore, the series of processes in the second modified example can also be applied to content other than input forms. That is, for example, the terminal device 10 may analyze the acquired first information and search for a web page to display the content. Alternatively, for example, the terminal device 10 may perform a display operation to display the searched web page on the screen.

[0110] <6.3 Third Modification: Automated Scraping> The terminal device 10 may, for example, perform automatic scraping with respect to content that is subject to automatic updates.

[0111] Specifically, for example, the operation reception unit 191 may accept input of sixth information from the user. The sixth information is information related to target information. The target information is information that is to be scraped and is related to the content that is subject to automatic updates. The sixth information may include, for example, the URL of the top page of the site that the user wishes to scrape, or the specification of information that the user wishes to search for (information that may become target information). As a result, the terminal device 10 may acquire the sixth information (third acquisition step).

[0112] The AI ​​system 30 may, for example, input the first information and the sixth information to the first generating AI model (first multimodal generating AI model) along with a prompt. In the third modified example, the prompt is an instruction that instructs the output of the third information, and is, for example, hardcoded in a program stored in the AI ​​system 30.

[0113] The AI ​​system 30 may, for example, output third information relating to the operations for performing scraping from the first generation AI model (first multimodal generation AI model). In the third modified example, the third information may include, for example, the specification of elements and attributes of HTML code (the top page of the site that the user wishes to scrape) using XPath or CSS selectors, operation methods such as clicking, and instructions to end scraping when the information desired by the user has been found (such as closing the page). As a result, the terminal device 10 may input the acquired first information and sixth information into the first generation AI model and output the third information from the first generation AI model (third output step).

[0114] The automatic processing unit 194 may, for example, operate on a web page containing content subject to automatic updating (hereinafter referred to as the first target web page) based on the third information. The automatic processing unit 194 may, for example, perform this operation to search for other web pages containing the target information (hereinafter referred to as the second target web page). If the automatic processing unit 194 finds the second target web page, for example, it may transition from the first target web page to the second target web page. The automatic processing unit 194 may, for example, extract the target information from the second target web page, which is the destination of the transition. As a result, the terminal device 10 may operate on the first target web page based on the outputted third information, transition from the first target web page to the second target web page, and extract the target information from the second target web page (second transition step).

[0115] Thus, according to this third modification, the user simply inputs the second and sixth pieces of information into the terminal device 10, and the terminal device 10 not only automatically updates the content but also automatically scrapes it. This further enhances user convenience.

[0116] <6.4 Fourth Variation: Content Update Using Other Apps> The second information is used when performing the content update work using application software. The terminal device 10 may update its content using, for example, other application software different from application 181 (hereinafter referred to as a collaborative application).

[0117] Specifically, for example, the operation reception unit 191 may accept input of information regarding the work content when performing content update work using a collaborative work application as input of second information. In the fourth modified example, the second information may include, for example, information indicating the work content of content update work performed collaboratively by the collaborative work application and application 181. Examples of information indicating the work content of content update work include "transferring data from a spreadsheet (collaborative work application) to an input form (content)". Another example of information indicating the work content of content update work is "sharing important information (content) from an email displayed on a web page with a chat application (collaborative work application)". Thus, the terminal device 10 may acquire information regarding the work content when performing content update work using a collaborative work application as second information.

[0118] The AI ​​system 30 may, for example, input the first information and the second information along with a prompt to the first generating AI model (first multimodal generating AI model), and cause the first generating AI model (first multimodal generating AI model) to output the seventh information. The seventh information is information about the operation content of the first target web page and the collaborative application. The seventh information may include, for example, screen XY coordinates for specifying the target of operation, operation method (input, click, etc.), input text / input value when the operation method is input, etc. In the fourth modified example, the prompt is an instruction sentence that instructs the output of the seventh information, and is, for example, hardcoded in a program stored in the AI ​​system 30. As a result, the terminal device 10 may cause the first generating AI model to output the seventh information.

[0119] The automated processing unit 194 may, for example, operate the first target web page and the collaborative application based on the seventh information. Examples of such operations include, for example, operations to transfer data from a spreadsheet to an input form (drag and drop), and operations to share important information in an email displayed on the web page with the chat application (click). When the automated processing unit 194 performs operations based on the seventh information, content is updated, such as data being transferred to the input form or important information being shared with the chat application. As a result, the terminal device 10 may operate the first target web page and the collaborative application based on the seventh information.

[0120] Thus, according to the fourth modified example, when a user performs content update work using a collaborative work application, the user simply inputs second information regarding the work content into the terminal device 10, and the content is automatically updated using the collaborative work application. This further enhances user convenience when operating multiple applications, including application 181, in parallel.

[0121] In addition, in the fourth modified example, the terminal device 10 may modify the content after the automatic update.

[0122] Specifically, for example, the AI ​​system 30 may input the fourth information along with a prompt to the second generating AI model (see the first modified example), and cause the second generating AI model to output the eighth information regarding the content modification operation after the automatic update. In this case, the prompt is an instruction that instructs the output of the eighth information, and is hardcoded, for example, in a program stored in the AI ​​system 30.

[0123] The eighth piece of information specifically concerns the modification operations performed on the first target webpage and the collaborative application to correct the content after it has been automatically updated. The eighth piece of information may include, for example, the screen XY coordinates for specifying the target of the modification operation, the operation method (input, click, etc.), and the input text / input value if the modification method is input.

[0124] The automated processing unit 194 may, for example, modify the first target web page and the collaborative application based on the outputted eighth information, thereby correcting the content after the automatic update.

[0125] [7. Basic Computer Hardware Configuration] Figure 9 is a block diagram showing the basic hardware configuration of computer 90. Computer 90 includes at least a processor 901, main memory 902, auxiliary storage 903, and a communication interface 991. These are electrically connected to each other by a communication bus 921.

[0126] The processor 901 is hardware for executing the instruction set written in a program. The processor 901 consists of an arithmetic unit, registers, peripheral circuits, etc.

[0127] Main memory 902 is used to temporarily store programs and data processed by programs, etc. For example, it is a volatile memory such as DRAM (Dynamic Random Access Memory).

[0128] Auxiliary storage device 903 refers to a storage device for saving data and programs. Examples include flash memory, HDD (Hard Disk Drive), magneto-optical disk, CD-ROM, DVD-ROM, and semiconductor memory.

[0129] The IF991 communication interface is an interface for inputting and outputting signals for communication with other computers via a network using wired or wireless communication standards.

[0130] A network consists of various mobile communication systems, such as the internet, LANs, and wireless base stations. For example, a network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks that can connect to the internet via designated access points (e.g., Wi-Fi®). When connecting wirelessly, communication protocols include, for example, Z-Wave®, ZigBee®, and Bluetooth®. When connecting via a wired connection, the network also includes connections made directly via USB (Universal Serial Bus) cables, etc.

[0131] Furthermore, by distributing all or part of each hardware configuration across multiple computers 90 and connecting them to each other via a network, a computer 90 can be virtually realized. Thus, the concept of computer 90 includes not only a computer 90 housed in a single enclosure or case, but also a virtualized computer system.

[0132] [8. Basic Functional Configuration of Computer 90] The functional configuration of the computer realized by the basic hardware configuration of computer 90 (Figure 9) will be explained. The computer comprises at least one functional unit: a control unit, a memory unit, and a communication unit.

[0133] Furthermore, the functional units of computer 90 can also be realized by distributing all or part of each functional unit across multiple computers 90 interconnected via a network. The concept of computer 90 includes not only a single computer 90 but also a virtualized computer system.

[0134] The control unit is realized when the processor 901 reads various programs stored in the auxiliary storage device 903, loads them into the main memory device 902, and executes processing according to those programs. The control unit can realize various functional units that perform information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0135] The memory unit is implemented by the main memory 902 and the auxiliary memory 903. The memory unit stores data, various programs, and various databases. The processor 901 can also reserve memory areas corresponding to the memory unit in the main memory 902 or the auxiliary memory 903 according to the program. The control unit can also cause the processor 901 to perform operations such as adding, updating, and deleting data stored in the memory unit according to the various programs.

[0136] A database, specifically a relational database, is used to manage and link together tabular data sets called masters, which are structurally defined by rows and columns. In a database, tables are called tables, masters are called masters, the columns of tables are called columns, and the rows of tables are called records. In a relational database, relationships can be established and linked between tables and masters.

[0137] Typically, each table and master has a primary key column to uniquely identify records, but setting a primary key column is not mandatory. The control unit can instruct the processor 901 to add, delete, or update records in specific tables and masters stored in the memory unit, according to various programs.

[0138] Furthermore, by storing data, various programs, and various databases in the memory unit, the information processing device and information processing system related to this disclosure can be considered to have been manufactured.

[0139] Furthermore, the databases and masters in this disclosure may include any data structures (lists, dictionaries, associative arrays, objects, etc.) in which information is structurally defined. Data structures also include data that can be considered as data structures by combining data with functions, classes, methods, etc., written in any programming language.

[0140] The communication unit is implemented by the communication IF991. The communication unit provides the functionality to communicate with other computers 90 via the network. The communication unit can receive information transmitted from other computers 90 and input it to the control unit. The control unit can cause the processor 901 to perform information processing on the received information according to various programs. The communication unit can also transmit information output from the control unit to other computers 90.

[0141] Furthermore, each of the above-mentioned configurations, functions, processing units, processing means, etc., may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. The present invention can also be implemented by software program code that realizes the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and the processor of that computer reads the program code stored in the storage medium. In this case, the program code read from the storage medium itself realizes the functions of the embodiments described above, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media used to supply such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs, optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, and the like.

[0142] Furthermore, the program code that implements the functions described in this embodiment can be implemented in a wide range of programming or scripting languages, such as assembler, C / C++, Perl, Shell, PHP, Java®, JavaScript, and TypeScript.

[0143] Furthermore, the program code of the software that realizes the functions of the embodiment may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the computer's processor may read and execute the program code stored in the storage means or storage medium.

[0144] The functions realized by the components described herein may be implemented in a circuit or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (Central Processing Units), conventional circuits, and / or combinations thereof, programmed to realize the functions described herein. A processor is considered a circuit or processing circuitry, including transistors and other circuits. A processor may be a programmed processor that executes a program stored in memory.

[0145] In this specification, circuitry, unit, and means are hardware programmed to perform or execute the functions described herein. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to perform or execute the functions described herein.

[0146] If the hardware is a processor that is considered to be a type of circuitry, then the circuitry, means, or unit is a combination of hardware and software used to constitute the hardware and / or processor.

[0147] Although several embodiments of this disclosure have been described above, these embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0148] [9. Addendum] The details described in each of the above embodiments are noted below.

[0149] <Note 1> A program for execution on a computer having a processor and memory, the program causing the processor to perform the following steps: a first acquisition step of acquiring first information about the content of a web page and second information about content that is scheduled to be updated in the content; a first output step of inputting the acquired first and second information into a first generating AI model and causing the first generating AI model to output third information about the operation content of the web page; and an update step of operating the web page and updating the content based on the output third information.

[0150] <Note 2> The program described in (Appendix 1) causes the processor to execute the following steps: a second acquisition step of acquiring fourth information regarding the content to be modified in the updated content; a second output step of inputting the acquired fourth information into a second generating AI model different from the first generating AI model, causing the second generating AI model to output fifth information regarding the content of the web page modification operation; and a modification step of modifying the web page based on the output fifth information and modifying the updated content.

[0151] <Note 3> The program described in (Appendix 1) or (Appendix 2) causes the processor to execute the following steps in the update step: a search step which involves inputting information into the input form, analyzing the acquired first information to search for a web page, and a first transition step which involves performing a transition operation to transition to the searched web page.

[0152] <Note 4> The first generative AI model is a multimodal generative AI model, and is one of the programs described in (Appendix 1) to (Appendix 3).

[0153] <Note 5> A program described in any of (Appendix 1) to (Appendix 4) that executes: a third acquisition step of obtaining sixth information related to the content and the target information to be scraped; a third output step of inputting the acquired first and sixth information into a first generating AI model and causing the first generating AI model to output third information; and a second transition step of manipulating a web page based on the outputted third information and transitioning from the web page to another web page containing the target information.

[0154] <Note 6> The second piece of information concerns the work involved when updating content using application software. In the first output step, the first generation AI model outputs the seventh piece of information concerning the operation of the web page and the application software. In the update step, the program described in any of (Appendix 1) to (Appendix 5) operates the web page and the application software based on the output seventh piece of information.

[0155] <Note 7> An information processing device comprising a control unit and a storage unit, wherein the control unit executes all steps in any of the programs described in (Appendix 1) to (Appendix 6).

[0156] <Note 8> A method to be executed on a computer comprising a processor and memory, wherein the processor executes all steps in any of the programs described in (Appendix 1) to (Appendix 6).

[0157] <Note 9> A system comprising means for executing all steps in any of the programs described in (Appendix 1) to (Appendix 6). [Explanation of symbols]

[0158] 1... System 10…Terminal device 120... Communications Department 13…Input device 14…Output device 15…Memory 16…Storage 19… Processor 20... Server 22...Communication IF 23…Input / Output Interface 25…Memory 2 hours… storage 29… Processor 30…AI system

Claims

1. A program to be executed on a computer having a processor and memory, The program is provided to the processor: A first acquisition step involves obtaining first information regarding the content of a web page and second information regarding the content that is scheduled to be updated. A first output step involves inputting the acquired first and second information into a first generating AI model and causing the first generating AI model to output third information relating to the operation content of the web page, An update step which involves manipulating the web page and updating the content based on the outputted third information, A program that executes the command.

2. A second acquisition step involves obtaining fourth information regarding the content that is scheduled to be corrected in the updated content, A second output step involves inputting the acquired fourth information into a second generation AI model different from the first generation AI model, and causing the second generation AI model to output fifth information regarding the content of the web page modification operation. A modification step involves modifying the web page based on the outputted fifth information and correcting the updated content, The program according to claim 1, which causes the processor to execute the following.

3. The aforementioned content is an input form, The second piece of information is information relating to the items to be entered into the input form, In the update step, input is made into the input form, The program according to claim 1, which causes the processor to perform a search step of analyzing the acquired first information to search for the web page, and a first transition step of performing a transition operation to transition to the searched web page.

4. The program according to claim 3, wherein the first generation AI model is a multimodal generation AI model.

5. A third acquisition step involves obtaining sixth information related to the content to be scraped, and A third output step involves inputting the acquired first information and the sixth information into the first generating AI model and causing the first generating AI model to output the third information, A second transition step involves manipulating the web page based on the outputted third information and transitioning from the web page to another web page containing the target information, The program according to claim 1 that causes the execution of the program.

6. The second piece of information is information regarding the work involved when updating the content using application software. In the first output step, the first generated AI model outputs seventh information relating to the operation content of the web page and the application software. The program according to claim 1, which in the update step operates the web page and the application software based on the outputted seventh information.

7. An information processing device comprising a control unit and a storage unit, wherein the control unit executes all steps in the program described in any one of claims 1 to 6.

8. A method to be performed on a computer comprising a processor and memory, wherein the processor performs all steps of a program according to any one of claims 1 to 6.

9. A system comprising means for executing all steps in the program described in any one of claims 1 to 6.

Images