Information processing method, program and information processing system

Abstract

To provide an information processing system, method and program that analyzes a focusing commodity / service from a website.SOLUTION: An information processing system includes user terminals mutually communicable via a network, and a server. An information processing method by the server includes: an image acquisition step of acquiring an image displayed on a display unit of the user terminal by an image acquisition unit; a text acquisition step of acquiring a first text displayed on the display unit of the user terminal by a text acquisition unit; an extraction step of extracting a second text from the acquired image by an extraction unit; an adjustment step of adjusting the second text on the basis of the first text by an adjustment unit; and an output step of outputting the adjusted text adjusted in the adjustment step by the adjustment unit.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to an information processing method, a program, and an information processing system. [Background technology]

[0002] Cited Document 1 discloses an information collection device that makes it possible to automatically organize and classify information obtained by browsing web pages according to its contents. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2004-310378 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above-mentioned prior art, although it is disclosed that texts posted on a website are analyzed, images posted on the website are not analyzed.

[0005] In consideration of the above, an object of the present invention is to provide an information processing method, a program, and an information processing system capable of analyzing corporate information such as a company's products and services, the culture and values ​​of the company, etc. from a website. [Means for solving the problem]

[0006] According to one embodiment, the information processing method is executed by an information processing device, and includes an image acquisition step of acquiring an image to be displayed on a display unit of the information processing device, a text acquisition step of acquiring a first text to be displayed on a display unit of the information processing device, an extraction step of extracting a second text from the image acquired in the image acquisition step, an adjustment step of adjusting the second text based on the first text, and an output step of outputting the adjusted text adjusted in the adjustment step.

[0007] According to a program of one embodiment, an information processing device is caused to execute an information processing method including an image acquisition step of acquiring an image to be displayed on a display unit of the information processing device, a text acquisition step of acquiring first text to be displayed on a display unit of the information processing device, an extraction step of extracting second text from the image acquired in the image acquisition step, an adjustment step of adjusting the second text based on the first text, and an output step of outputting adjusted text adjusted in the adjustment step.

[0008] According to one embodiment, the information processing system is executed by an information processing device, and includes an image acquisition step of acquiring an image to be displayed on a display unit of the information processing device, a text acquisition step of acquiring first text to be displayed on the display unit of the information processing device, an extraction step of extracting second text from the image acquired in the image acquisition step, an adjustment step of adjusting the second text based on the first text, and an output step of outputting the adjusted text adjusted in the adjustment step. Effect of the Invention

[0009] According to one embodiment, corporate information such as the company's products and services, the company's culture, values, etc. can be analyzed from the website. [Brief description of the drawings]

[0010] [Figure 1]FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a server according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a user terminal according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a functional configuration of a server according to an embodiment. [Diagram 5] FIG. 2 is a diagram illustrating an example of a functional configuration of a user terminal according to an embodiment. [Figure 6] FIG. 4 is a diagram illustrating an example of a website screen according to the embodiment. [Figure 7] 5 is a flowchart illustrating an example of processing executed by the information processing system according to the embodiment. [Figure 8] FIG. 13 is a diagram showing an example of a website screen including an advertisement according to an embodiment. [Figure 9] 5 is a flowchart illustrating an example of processing executed by the information processing system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An embodiment of an information processing system according to the present invention will be described below with reference to Figures 1 to 5. In each drawing, the same or equivalent components and parts are given the same reference numerals. Also, the dimensional ratios in the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

[0012] (System Overview) First, an overview of the information processing system 10 according to this embodiment will be described. The information processing system 10 according to this embodiment is a system for outputting a company's featured products and services from a website. In this embodiment, the information processing system 10 is a system for searching the website of a target company from an input character string, analyzing images and text (character strings) posted on the website, and outputting the featured products and services of the searched company.

[0013] (System Configuration) Fig. 1 is a diagram showing an example of a configuration of an information processing system 10 according to the present embodiment. As shown in Fig. 1, the information processing system 10 according to the present embodiment includes a user terminal 2 and a server 1, which are communicably connected to each other via a network N. The network N is, for example, a wired LAN (Local Area Network), a wireless LAN, the Internet, a public line network, a mobile data communication network, or a combination of these.

[0014] The user terminal 2 is an example of an information processing device that performs operations for inputting and displaying various information. The user terminal 2 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, a wearable device, or a combination of these.

[0015] The server 1 is an example of an information processing device that acquires information on an input sentence input from a user terminal 2, searches the web or the like for a website of a company to be analyzed based on the input sentence information, analyzes the website of the company, and outputs results of products and services to be focused on. The server 1 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, or a combination of these. The specific configuration and operation of the server 1 will be described later.

[0016] (Hardware configuration - Server) 2 is a block diagram showing a hardware configuration of the server 1. The server 1 includes a processor 201, a memory 202, a storage 203, and a communication I / F 204, which are communicatively connected to each other via a bus B.

[0017] The processor 201 controls each component of the server 1 and realizes the functions of the server 1 by expanding various programs stored in the storage 203 into the memory 202 and executing them. The programs executed by the processor 201 include, but are not limited to, an OS (Operating System) and various programs described below. The execution of these programs by the processor 201 realizes a part of the state visualization method according to this embodiment. The processor 201 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), or a combination of these.

[0018] The memory 202 is, for example, a Read Only Memory (ROM), a Random Access Memory (RAM), or a combination thereof. The ROM is, for example, a Programmable ROM (PROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), or a combination thereof. The RAM is, for example, a Dynamic RAM (DRAM), a Static RAM (SRAM), a Magnetoresistive RAM (MRAM), or a combination thereof.

[0019] The storage 203 stores an OS, various programs described below, and various data. The storage 203 is, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), a storage class memory (SCM), or a combination of these.

[0020] The communication I / F 204 is an interface for connecting the server 1 to external devices including the user terminal 2 and the image capturing device 16 via the network N and for controlling communication. The communication I / F 204 is, for example, an adapter conforming to Bluetooth (registered trademark), Wi-Fi (registered trademark), ZigBee (registered trademark), Ethernet (registered trademark), or optical communication (for example, Fibre Channel), but is not limited thereto.

[0021] (Hardware configuration - User terminal) 3 is a block diagram showing a hardware configuration of the user terminal 2. The user terminal 2 includes a processor 201, a memory 202, a storage 203, a communication I / F 204, an input / output I / F 205, an input device 206, and an output device 207, which are communicatively connected to each other via a bus B. The communication I / F 204 is an interface for connecting the user terminal 2 to an external device including the server 1 via a network N and controlling communication. The communication I / F 204 is, for example, an adapter conforming to Bluetooth (registered trademark), Wi-Fi (registered trademark), ZigBee (registered trademark), Ethernet (registered trademark), or optical communication (for example, Fibre Channel), but is not limited thereto.

[0022] The input / output I / F 205 is an interface for connecting an input device 206 and an output device 207 to the user terminal 2. The input device 206 is, for example, a mouse, a keyboard, a touch panel, a microphone, a scanner, a camera, various sensors, an operation button, or a combination of these. The output device 207 as a user interface is, for example, a display, a projector, a printer, a speaker, a vibrator, or a combination of these. In this embodiment, as an example, the output device 207 and the input device 206 are an integrally configured touch panel display.

[0023] In this embodiment, the program may be written in the memory 202 or the storage 203 during the manufacturing stage of the server 1, or may be provided to the server 1 via the network N. Also, the program may be provided to the server 1 via a non-transitory computer-readable recording medium such as a disk medium (not shown).

[0024] (Functional configuration - Server) Next, the functional configuration of the server 1 will be described. Fig. 4 is a diagram showing an example of the functional configuration of the server 1. When executing various programs, the server 1 realizes various functions using the above hardware resources. The server 1 has a communication unit 11, a storage unit 12, and a control unit 13 as the functional configuration realized by the server 1. Each functional configuration is realized by the processor 101 reading and executing a program 121 stored in the memory 102 or storage 103.

[0025] Control unit 13 includes image acquisition unit 131 that acquires images posted on a website, and text acquisition unit 132 that acquires text written on the website. Control unit 13 further includes extraction unit 133 that extracts text from the image acquired by image acquisition unit 131, adjustment unit 134 that adjusts the text extracted by extraction unit 133, and determination unit 135 that determines the size of the image acquired by image acquisition unit 131. These will be described in detail later.

[0026] (Functional configuration - User terminal) Next, the functional configuration of the user terminal 2 will be described. Fig. 5 is a diagram showing an example of the functional configuration of the user terminal 2. When executing various programs, the user terminal 2 realizes various functions using the above hardware resources. The user terminal 2 has a communication unit 21, a storage unit 22 in which a program 221 is stored, and a control unit 23 as functional configurations realized by the user terminal 2. Each functional configuration is realized by the processor 201 reading and executing the program 221 stored in the memory 202 or the storage 203. The control unit 23 includes an information acquisition control unit 231 that acquires information sent from the server 1, and a display unit 232 that displays the acquired information on the output device 207.

[0027] Fig. 6 is a schematic diagram showing an example of a company's website screen displayed on the user terminal 2. Fig. 7 is a diagram relating to processing executed by the information processing system 10.

[0028] The website screen includes text T1-T3 and images IM1-IM3 displayed on the website. The text T1-T3 includes information related to a brand, such as a company name, a product name, or a service name, and includes text to explain the information. The images IM1-IM3 are images of products, image images of services, or images including text.

[0029] The process executed by the information processing system 10 will be described with reference to the flowchart in Fig. 7. First, the control unit 13 of the server 1 identifies a company based on a character string input by the user via the input device 206, and in step S101, the image acquisition unit 131 acquires images from the website of the identified company (image acquisition step). Specifically, the image acquisition unit 131 acquires images IM1 to IM3 in Fig. 6. Next, in step S102, the text acquisition unit 132 acquires text (first text) written on the website (text acquisition step). Specifically, the text acquisition unit 132 acquires text T1 to T3 in Fig. 6.

[0030] Next, in step S103, the determination unit 135 determines the priority of the images IM1 to IM3 acquired in step S101 (determination step). Specifically, the determination unit 135 determines the priority according to the size, position, area, occupancy rate, etc. of the image. In the example of FIG. 6, the sizes of the images are IM1>IM2>IM3. Furthermore, the image IM1 is posted in a prominent position at the top of the website. For this reason, the determination unit 135 determines that the priority of the image IM1 is high. Note that, although the present embodiment describes a case where the website includes multiple images, if there is only one image, for example, step S103 may be omitted.

[0031] Next, in step S104, the extraction unit 133 extracts text (second text) from the image with high priority based on the priority determined in step S103 (extraction step). In this embodiment, the extraction unit 133 extracts text from the image IM1 determined by the determination unit 135 in step S103 to have high priority. The text extracted at this time is a general term, and is a name different from the actual product or service name of the company. For example, if an automobile manufacturer posts an image of an automobile, the extraction unit 133 extracts the general term automobile from the image, which is different from the name of the product of the company. In this embodiment, the extraction unit 133 extracts text from the image using, for example, a CLIP (Contrastive Language-Image Pre-training) model. However, the present invention is not limited to this, and text may be extracted from the image using another method.

[0032] Next, in step S105, the adjustment unit 134 adjusts the text extracted in step S104 (adjustment step). Specifically, the adjustment unit 134 adjusts the text to match the website based on the text described in the website acquired in step S102, that is, performs so-called fine tuning. In other words, as described above, the text extracted in step S104 is merely a text of general terms. Therefore, it is difficult to express the products and services that a company focuses on by simply extracting text from an image. Therefore, fine tuning is necessary based on the text described in the website. Then, in step S106, the adjustment unit 134 outputs the text adjusted in step S105 (adjusted text) (output step).

[0033] Here, an example of the fine-tuning method used in step S105 will be given. In this embodiment, for example, a VLM (Vision Language Model) is used as the base model. That is, the VLM combines a CLIP-based Vision Encoder and LLMs (Large Language Models) to enable the LLM to interpret visual information. Specifically, image embedding is linearly projected into a text embedding space. In this embodiment, PEFT (Parameter Efficient Fine Tuning) is used as a fine-tuning method for the base model. By freezing the parameters of the pre-trained LLM using PEFT including LoRA (Low-Rank Adaption), it is possible to efficiently fine-tune. By such fine-tuning, for example, if the target company is an automobile company, car images and car information can be learned from the website of the automobile company, and information on the car model and the car model can be obtained for the target image when inferring the information of the target company.

[0034] From the above, the company's website clearly shows the products and services that the company is focusing on. In other words, the company's website uses large images of the products and services that it is focusing on and places the images in prominent positions. Therefore, by extracting the text from such images, we can understand what the company is focusing on, and by fine-tuning based on the text on the website, the specific products and services can be clearly understood.

[0035] In this embodiment, the products and services that a company focuses on are analyzed from the company's website, but the company's website contains not only information about the products and services, but also information about the company's management philosophy, mission, values, etc. In this case, for example, after step S102, a process is inserted to determine whether the text acquired in step S102 is a text about the product or service, or a text about the company's management philosophy, mission, and values. This makes it possible to output the text according to whether it is a product or service, or a business philosophy, mission, or values ​​of the company, when fine-tuning is performed in step S105 and output in step S106. Note that the location where the process to determine whether the text acquired in step S102 is a text about the product or service, or a text about the company's management philosophy, mission, and values ​​is inserted may be changed as appropriate. By adding this process, it becomes possible to grasp the culture and values ​​of the company.

[0036] Next, Fig. 8 is a schematic diagram showing an example of a screen of a company's website displayed on the user terminal 2 when the website includes an advertisement. Fig. 9 is a diagram related to the process executed by the information processing system 10 when the website includes an advertisement.

[0037] As shown in Fig. 8, a company's website may include advertisements. In this case, step S107 is inserted between step S102 and step S103, and in step S107, the determination unit 135 determines whether or not the website includes advertisements (advertisement determination step). If the website includes advertisements (YES), in step S108, the determination unit 135 proceeds to step S103, excluding advertisement images. On the other hand, if advertisement images are not included (NO), the process proceeds to step S103, and the subsequent processes are the same as those shown in Fig. 7.

[0038] According to this embodiment, the text extracted from the images (excluding advertisements) posted on the company's website is fine-tuned based on the text posted on the website, so that it is possible to know what products and services the company is focusing on. In addition, even if not only information on products and services but also information on the company's management philosophy, mission, values, etc. is posted, a process is inserted to determine whether the text is related to products and services or the company's management philosophy, mission, and values, as described above. This makes it possible to use it in research such as competitive research and market research.

[0039] Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the scope that can achieve the object of the present invention are included in the present invention.

[0040] Also, for example, the above-mentioned series of processes can be executed by hardware or software. In other words, the functional configuration is merely an example and is not particularly limited. That is, it is sufficient that the information processing system is provided with a function capable of executing the above-mentioned series of processes as a whole, and there is no particular limit to what kind of functional block is used to realize this function. In addition, the location of the functional block is not particularly limited and may be arbitrary. For example, the functional block of the server may be transferred to a user terminal or the like. Conversely, the functional block of the user terminal may be transferred to a server or the like. In addition, one functional block may be configured by hardware alone, software alone, or a combination thereof.

[0041] For example, when a series of processes is executed by software, a program constituting the software is installed in a computer or the like from a network or a recording medium. The computer may be a computer incorporated in dedicated hardware. The computer may also be a computer capable of executing various functions by installing various programs, such as a server, a general-purpose smartphone, or a personal computer.

[0042] Furthermore, for example, a recording medium containing such a program may not only be constituted by a removable medium (not shown) that is distributed separately from the device main body in order to provide the program to the user, but may also be constituted by a recording medium that is provided to the user in a state in which it is pre-installed in the device main body.

[0043] In this specification, the steps of describing a program to be recorded on a recording medium include not only processes that are performed chronologically according to the order, but also processes that are not necessarily performed chronologically but are executed in parallel or individually. In addition, in this specification, the term "system" refers to an overall device that is composed of a plurality of devices, a plurality of means, etc. [Explanation of symbols]

[0044] 10 Information Processing Systems 2 User terminal (information processing device) 1 Server (information processing device) 131 Image Acquisition Unit 132 Text Acquisition Department 133 Extraction part 134 adjustment section

Claims

1. An information processing method executed by an information processing device, an image acquiring step of acquiring an image to be displayed on a display unit of the information processing device; a text acquisition step of acquiring a first text displayed on a display unit of the information processing device; an extraction step of extracting second text from the image acquired in the image acquisition step; adjusting the second text based on the first text; an output step of outputting the adjusted text obtained in the adjustment step; An information processing method comprising:

2. The method further includes a determination step of determining a priority of the image acquired in the image acquisition step according to a size of the image, a position, an area, and an occupancy rate of the image on the display unit, In the extracting step, the second text is extracted from the image determined to be a high priority image in the determining step. The information processing method according to claim 1 .

3. The method further includes an advertisement determination step of determining whether the image acquired in the image acquisition step is an advertisement, If the image is determined not to be an advertisement in the advertisement determination step, the priority of the image is determined in the determination step. The information processing method according to claim 2 .

4. In the information processing device, an image acquiring step of acquiring an image to be displayed on a display unit of the information processing device; a text acquisition step of acquiring a first text displayed on a display unit of the information processing device; an extraction step of extracting second text from the image acquired in the image acquisition step; adjusting the second text based on the first text; and an output step of outputting the adjusted text adjusted in the adjustment step.

5. An information processing system executed by an information processing device, an image acquiring step of acquiring an image to be displayed on a display unit of the information processing device; a text acquisition step of acquiring a first text displayed on a display unit of the information processing device; an extraction step of extracting second text from the image acquired in the image acquisition step; adjusting the second text based on the first text; an output step of outputting the adjusted text obtained in the adjustment step; An information processing system that performs the above.

Images