Collection device, collection method, and program

Abstract

To provide a collection device, a collection method, and a program with which it is possible to efficiently promote the process of collecting information on users of a plurality of services and storing the information in a database.SOLUTION: A collection device 100 comprises: a service record acquisition unit 110 for acquiring a service record from a service database 200 pertaining to a service to be processed; and a user record update unit 130 for updating a user record on the basis of the acquired service record. When starting update processing for updating user records, the user record update unit 130 aggregates the remaining work amount of update processing of a service whose update processing is already begun and not finished yet among a plurality of services, and determines, in accordance with the aggregated amount of the remaining work, whether to perform determination processing in the update processing based on the service record pertaining to the service to be processed.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a collection device, a collection method, and a program. [Background technology]

[0002] 2. Description of the Related Art There is known a technique for collecting and storing information on users of services provided on the Internet and utilizing the stored information.

[0003] For example, Patent Document 1 discloses an information processing device that collects browsing data of viewed content viewed by users from user terminals, adds the collected browsing data for each user to a list to be stored in a database, and adds the browsing data to the database at predetermined time intervals. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-164425 A Summary of the Invention [Problem to be solved by the invention]

[0005] Since this information processing device adds the collected browsing data of all users to the database, it takes time from when the browsing data is collected to when it is added to the database. Therefore, there is room for improvement in terms of efficiently carrying out the process of collecting user information and storing it in the database.

[0006] In addition, for businesses offering multiple services, the amount of user information they collect is becoming increasingly huge, so there is a demand for technology that can more efficiently collect user information and store it in a database.

[0007] The present invention has been made in consideration of the above-mentioned circumstances, and provides a collection device, a collection method, and a program that can efficiently carry out a process of collecting information on users of a plurality of services and storing the information in a database. [Means for solving the problem]

[0008] In order to solve the above problems, the collection device according to the present invention comprises: A collection device that collects user information from a plurality of services and stores the information in a user database, The user information for each of the plurality of services is stored in a service record stored in a service database for each of the services; The user database has a user record, and stores information derived from the service record of each of the plurality of services for each of the users. a service record acquisition unit that acquires the service record from the service database related to the service to be processed; a user record update unit that updates the user record based on the service record related to the service to be processed acquired by the service record acquisition unit; Equipped with The user record update unit, When starting an update process for updating the user record, tallying up the remaining amount of work for the update process of a service for which the update process has already started but not yet ended among the plurality of services; In the update process based on the service record related to the service to be processed, for each user, it is determined whether or not an update to the user record is necessary, and it is determined whether or not to perform a determination process to skip the update process for users for whom it is determined that an update is not necessary, based on the aggregated remaining amount of work. Effect of the Invention

[0009] According to the present invention, it is possible to provide a collection device, a collection method, and a program that can efficiently carry out the process of collecting information on users of a plurality of services and storing the information in a database. [Brief description of the drawings]

[0010] [Figure 1] FIG. 2 is an explanatory diagram showing cooperation between the collection device and other devices. [Diagram 2] FIG. 2 is a diagram illustrating a process executed by a collection device. [Diagram 3] FIG. 4 is a diagram illustrating an example of access log data. [Figure 4] FIG. 2 is a diagram showing an example of a service record generated by the service database. [Diagram 5] FIG. 2 is a diagram showing an example of a user record stored in a user database. [Figure 6] FIG. 2 is an explanatory diagram illustrating a functional configuration of a collection device. [Figure 7] FIG. 2 illustrates an example of a command queue included in the collection device. [Figure 8] 13 is a diagram illustrating a process of comparing a service record with a user record by an update instruction unit. FIG. [Figure 9] FIG. 2 is an explanatory diagram illustrating a physical configuration of a collection device. [Figure 10] 13 is a flowchart of a user record update process. [Figure 11] 13 is a flowchart of another example of a user record update process. [Figure 12] 13 is a flowchart of step S104 in the next update process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the embodiment is for illustrative purposes only and does not limit the scope of the present invention. Therefore, a person skilled in the art can adopt an embodiment in which each or all of these elements are replaced with an equivalent, and such an embodiment is also included in the scope of the present invention.

[0012] (Relationship between collection device and program) The collection device according to this embodiment collects user information for each service from a plurality of service databases that store user information including the access history of the user for each service provided on the Internet, and updates a user database that stores user information for each user for each of the plurality of services based on the collected user information. Here, user information is information that indicates the operation and attributes of each user. The collection device is composed of one or more server devices (server computers).

[0013] The collection device according to this embodiment is generally realized by making a computer execute a program, but it is also possible to make a dedicated electronic circuit execute the processing. In addition, as an intermediate form between a computer and a dedicated electronic circuit, it is also possible to configure the collection device according to this embodiment by applying a technology such as FPGA (Field Programmable Gate Array) that compiles a program into a design script for an electronic circuit and dynamically configures the electronic circuit based on the design script.

[0014] The collection device according to this embodiment is realized by one or more server computers that communicate with a database executing the functions realized by one or more server programs.

[0015] In general, the programs executed by the server computer can be recorded on computer-readable non-transitory information recording media such as compact disks, flexible disks, hard disks, magneto-optical disks, digital video disks, magnetic tapes, ROMs (Read Only Memory), EEPROMs (Electrically Erasable Programmable ROM), flash memories, semiconductor memories, etc. These information recording media can also be distributed and sold independently of the server computer.

[0016] In a server computer, a program recorded on a non-transitory information recording medium such as a flash memory or a hard disk is read into a temporary storage device called a random access memory (RAM), and then the instructions contained in the read program are executed by a central processing unit (CPU). However, in an architecture that allows for mapping and execution of ROM and RAM in a single memory space, the instructions contained in the program stored in the ROM are directly read and executed by the CPU.

[0017] Furthermore, the server program may be distributed and sold to server computers and terminal computers, etc. from a distribution server, etc. managed by the operator, via a temporary transmission medium such as a computer communication network, independently of the computer on which the program is executed.

[0018] In addition, when the collection device is composed of multiple computers, the programs running on the computers are multiple different server programs that have different functions and work together, and the combination of the multiple programs can be considered as a system program for realizing the collection device.

[0019] (Overall composition) 1 is an explanatory diagram showing the cooperation between the collection device and other devices. The following description will be given with reference to this diagram.

[0020] The collection device 100 is connected to a plurality of service databases 200a, 200b, 200c, and the like, and a user database 300, via a communication network 400. The communication network 400 may include various types of networks, such as a local area network (LAN), a wide area network (WAN) such as the Internet, a telecommunications network such as a public switched telephone network (PSTN), a wireless network, a public switched network, a satellite network, a cellular network, a public land mobile network (PLMN), a metropolitan area network (MAN), a private network, an ad-hoc network, an intranet, an optical fiber-based network, and / or a combination of these or other types of networks. In the following, the service databases 200a, 200b, 200c, and the like are collectively referred to as the service database 200.

[0021] The collecting device 100 is operated, for example, by a business operator that operates a plurality of service providing sites on the Internet. Here, an overview of the processing executed by the collecting device 100 will be described with reference to FIG. 2. As illustrated in FIG. 2, the collecting device 100 collects, from the service databases 200a, 200b, 200c, . . ., service records that store user information including an access history of a user accessing each service providing site using a user terminal. Based on the collected service records, the collecting device 100 updates the user records stored in the user database 300 that manages user information for each user and for each service. In this way, information based on (derived from) the service records is stored in the user records.

[0022] When updating a user record of a certain service stored in the user database 300, the collecting device 100 counts up the remaining work amount of the update process related to other services for which the update process has started but not yet been completed. Based on the counted remaining work amount, the collecting device 100 judges whether or not to perform a judgment process for judging whether or not an update is necessary by comparing the service record and the user record for each user. When the collecting device 100 decides to perform the judgment process, the collecting device 100 compares the service record collected from the service database 200 with the user record stored in the user database 300 for each user and judges whether or not an update is necessary for each user. The collecting device 100 skips the update process for the user record of the user for which it has been determined that the update is unnecessary, and updates only the user record of the user for which it has been determined that the update is necessary with the collected service record. On the other hand, when the collecting device 100 decides not to perform the judgment process, it updates the user records corresponding to all the service records collected from the service database 200.

[0023] Returning to FIG. 1, the service database 200 stores a service record that stores user information including an access history at each service providing site for each user. The service database 200 is realized by a storage device such as an HDD (hard disk drive) or SSD (solid state drive) provided in a service providing server (not shown) that provides each service. Each service providing server records access log data, which is log data of a user's access to each service providing site. Here, an example of access log data for a service with a service ID of "A001" is shown in FIG. 3. As illustrated in FIG. 3, the access log data includes information such as a "user ID" that uniquely identifies a user and an "access date and time" that indicates the date and time when the user accessed the service providing site. Note that the access log data is not limited to the example of FIG. 3 and may include any information. For example, activity information such as a product purchase history and an acquired point history may be accumulated. Also, log data including different information for each service may be recorded.

[0024] The service database 200 stores the results of tallying the access log data for any tallying period, such as daily, weekly, or monthly. FIG. 4 shows an example of the service database 200 for a service with a service ID of "A001". As shown in FIG. 4, the service database 200 is a table-format database, and the service records of each row constituting the service database 200 include information such as a "user ID" that uniquely identifies a user, and a "number of accesses" that indicates the number of times the user accessed the service providing site within the tallying period. The information stored in the user record is not limited to the example in FIG. 4, and any information may be stored. For example, information such as a service ID for uniquely identifying a service, the last access date and time within the tallying period, the number of points acquired by the user within the tallying period, the number of products purchased by the user within the tallying period, and the purchase amount may be stored.

[0025] Returning to FIG. 1, the user database 300 stores user records that store user information including the access history of each user in each of a plurality of services. The user database 300 is realized by a database server, a cloud server, or the like. An example of the user database 300 is shown in FIG. 5. As shown in FIG. 5, the user database 300 is a table-format database, and the user records of each row that constitute the user database 300 include information such as a "service ID" that is information that uniquely identifies a service, a "user ID" that is information that uniquely identifies a user, and an "access count" that indicates the number of accesses made by the user during a collection period. Note that the information stored in the user record is not limited to the example of FIG. 5, and may store information such as the last access date and time, the number of points acquired, the number of products purchased, and the purchase amount, as in the service database 200, or may store the user's name, address, and user attribute information.

[0026] (Functional configuration of the collection device) Next, a functional configuration of the collection device 100 will be described with reference to Fig. 6. The collection device 100 includes a service record acquisition unit 110, a service record storage unit 120, and a user record update unit .

[0027] The service record acquisition unit 110 acquires service records from the service databases 200a, 200b, 200c, etc. Specifically, the service record acquisition unit 110 acquires service records from each service database 200 intermittently, periodically, or when it is detected that a service record stored in each service database 200 has been updated, and stores the acquired service records in the service record storage unit 120. Depending on the number of service records stored in the service database 200, the service record acquisition unit 110 may acquire all the service records at once, or may acquire the service records in any unit number, such as 10 records at a time or 100 records at a time.

[0028] Returning to Fig. 6, the service record storage unit 120 stores the service record acquired by the service record acquisition unit 110. Specifically, the service record storage unit 120 stores the acquired service record in association with a service ID for uniquely identifying the service database 200 from which the service record was acquired. For example, as illustrated in Fig. 2, when a service record is acquired from the service database 200a, the service record storage unit 120 stores the acquired service record in association with the service ID "A001" of the service database 200a.

[0029] Returning to Figure 6, the user record update unit 130 includes a remaining work amount aggregation unit 131, a judgment unit 132, and an update instruction unit 133, and updates the user records stored in the user database 300 based on the service records acquired by the service record acquisition unit 110.

[0030] When starting a process of updating a user record based on a service record of a certain service, the remaining work amount tallying unit 131 tallies up the remaining work amount, which is the remaining work amount of the update process related to a service for which the update process has already started but is not yet completed, based on the update command queued in the command queue that stores the update command. Specifically, the remaining work amount tallying unit 131 tallies up the remaining work amount by any method, such as tallying up the number of services for which the update process is not yet completed, the predicted time until the update process is completed for which the update process is not yet completed, the number of records for which the update process is not yet completed, the queue length indicating the number of update commands queued in the command queue, etc. Details of the process of the remaining work amount tallying unit 131 will be described later.

[0031] The determination unit 132 determines whether or not to execute a determination process for determining whether an update is necessary by comparing the service record and the user record for each user based on the remaining work amount tallied by the remaining work amount tabulation unit 131. Specifically, the determination unit 132 determines to execute the determination process when the remaining work amount tallied by the remaining work amount tabulation unit 131 is equal to or greater than a set threshold, and determines not to execute the determination process when it is less than the threshold. The determination unit 132 notifies the update instruction unit 133 of the determination result.

[0032] The update instruction unit 133 generates an update command for updating the user record based on the determination result by the determination unit 132, and adds the command to a command queue that stores the update command. The command queue holds the update command in a FIFO (First in First out) list structure, for example, and is constructed in the storage unit 14 of the collection device 100. As illustrated in Fig. 7, the command queue includes information such as a "user ID" for uniquely identifying a user, a "service ID" for uniquely identifying a service, identification information for uniquely identifying the user information to be updated, and "user information: [value]" indicating the updated value.

[0033] Specifically, when the determination unit 132 notifies the update instruction unit 133 that the determination process is to be executed, the update instruction unit 133 accesses the user database 300 and reads out the latest user record. Next, as illustrated in FIG. 8, the update instruction unit 133 compares the service record and the user information of the user record for each user. When the user information of the user record is different from the user information of the service record, the update instruction unit 133 determines that an update is necessary, and generates an update command to update the user record of the user as illustrated in FIG. 7, and adds it to the end of the command queue. On the other hand, when the user information of the user record is the same as the user information of the service record, the update instruction unit 133 determines that an update is unnecessary, and skips without generating an update command to update the user record of the user. The user record update unit 130 takes out the update command at the top of the command queue, and sequentially executes the update process according to the update command.

[0034] On the other hand, if the judgment unit 132 notifies that the judgment process will not be executed, the update instruction unit 133 generates update commands for updating the corresponding user records for each of all service records acquired by the service record acquisition unit 110, and adds them sequentially to the end of the command queue.

[0035] (Physical configuration of the collector) The collection device 100 having the functional configuration described above physically comprises, as shown in FIG. 9, a CPU 11 that executes processing according to a program, a RAM 12 which is a volatile memory, a ROM 13 which is a non-volatile memory, a storage unit 14 which stores data, an input unit 15 which accepts input of information, a display unit 16 which visualizes and displays information, and a communication unit 17 which transmits and receives information, all of which are connected via an internal bus 99.

[0036] The CPU 11 executes various processes by reading out the programs stored in the storage unit 14 into the RAM 12 and executing them. The CPU 11 executes each process by a service record acquisition unit 110 and a user record update unit 130 as main functions provided by the programs.

[0037] The RAM 12 is used as a work area for the CPU 11. The ROM 13 stores a control program executed by the CPU 11 for the basic operation of the collection device 100, a BIOS (Basic Input Output System), and the like.

[0038] The storage unit 14 includes a hard disk drive, and stores the programs executed by the CPU 11 and various data used when the programs are executed. The storage unit 14 functions as a service record storage unit 120.

[0039] The input unit 15 is a user interface including a keyboard, a mouse, a communication device, etc. The display unit 16 is a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display that visualizes and displays information.

[0040] The communication unit 17 is a network termination device or a wireless communication device that connects to a network, and a serial interface or a LAN interface that connects to them.

[0041] (User record update process) Next, the operation of the collection device 100 will be described with reference to Fig. 10 and Fig. 11. Fig. 10 is a flowchart explaining the flow of a user record update process in which all service records are acquired from the service database 200 at once and the corresponding user records are updated. Fig. 11 is a flowchart explaining the flow of a user record update process in which an arbitrary number of service records are acquired sequentially from the service database 200 and the corresponding user records are updated. For example, depending on the settings of an administrator, the collection device 100 executes the user record update process according to the flowchart exemplified in Fig. 10 in the service database 200 with a small data size, and executes the user record update process according to the flowchart exemplified in Fig. 11 in the service database 200 with a large data size.

[0042] First, the user record update process in the case where all service records are acquired at once will be described with reference to Fig. 10. This user record update process starts when the collection device 100 receives an instruction to start the user record update process, for example.

[0043] The service record acquisition unit 110 determines whether or not the update of any of the service databases 200, among the service databases 200a, 200b, 200c, has been completed (step S101). Specifically, the service record acquisition unit 110 waits for a signal notifying that the update of any of the service databases 200 has been completed (hereinafter, referred to as an update completion signal). By receiving the update completion signal, the service record acquisition unit 110 detects that the service database 200 has been updated (step S101; Yes), and acquires the service database 200 for which the update has been completed (step S102). The service record acquisition unit 110 stores the acquired service database 200, exemplified in FIG. 4, in the service record storage unit 120.

[0044] On the other hand, if the service record acquisition unit 110 does not receive an update completion signal, it determines that none of the service databases 200 have been updated (step S101; No), and returns to step S101 to wait for the update completion signal. Note that the service record acquisition unit 110 may acquire the service databases 200a, 200b, 200c, ... intermittently or periodically, or may compare the previous acquisition date and time with the update date and time of the service database 200 to acquire a service database 200 that has been determined to have been updated.

[0045] In step S102, when the service database 200 is acquired, the remaining work amount tallying unit 131 tallies up the remaining work amount of the unfinished update processes (step S103). Specifically, the remaining work amount tallying unit 131 tallies up the remaining work amount by calculating the number of services whose update processes are unfinished, the predicted time until the unfinished update processes are completed, the number of records whose update processes are unfinished, a queue length indicating the number of update commands queued in the command queue, and the like, based on the update commands queued in the command queue illustrated in FIG.

[0046] When calculating the number of services for which update processing is not yet completed, the remaining work amount tallying unit 131 calculates the number of service IDs included in the command queue. When calculating a predicted time until an unfinished update processing is completed, the remaining work amount tallying unit 131 calculates the predicted time by multiplying the time required to execute update processing based on one update command by the number of update commands queued in the command queue. When calculating the number of records for which update processing is not yet completed, the remaining work amount tallying unit 131 calculates the number of combinations of service IDs and user IDs included in the command queue. When calculating a queue length, the remaining work amount tallying unit 131 calculates the number of update commands queued in the command queue.

[0047] Next, the determination unit 132 compares the service record and the user record for each user based on the counting result of the remaining work amount counting unit 131 in step S103, and determines whether or not to execute a determination process for determining whether an update is necessary (step S104). Specifically, the determination unit 132 determines to execute the determination process when the remaining work amount counted by the remaining work amount counting unit 131 exceeds a set threshold, and determines not to execute the determination process when the remaining work amount is equal to or less than the threshold. The determination unit 132 notifies the update instruction unit 133 of the determination result.

[0048] If the judgment unit 132 judges that the judgment process is to be executed (step S104; Yes), the update instruction unit 133 repeatedly executes the processes of steps S106 to S108 for all service records in the service database 200 acquired in step S102 (step S105).

[0049] In step S106, the update instruction unit 133 reads an unprocessed service record, and acquires a user record corresponding to the read service record from the user database 300 (step S106). Specifically, the update instruction unit 133 acquires a user record that matches the service ID associated with the read service record and the user ID stored in the read service record.

[0050] Next, in step S107, the update instruction unit 133 compares the user information in the service record with the user record to determine whether the user information is the same (step S107). As illustrated in FIG. 8, for example, when comparing the service record and user record of the service ID "A001" of the user ID "X0001", the access count "20" of the user ID "X0001" stored in the service record is compared with the access count "10" of the user ID "X0001" stored in the user record. If the user information in the user record is different from the user information in the service record (step S107; No), the update instruction unit 133 determines that an update is necessary, generates an update command to update the user record of the user, and adds it to the end of the command queue (step S108). Specifically, in the case of the user ID "X0001" mentioned above, a command is generated to update the access count "10" for the service ID "A001" in the user record to the access count "20" in the service record, and added to the end of the command queue.

[0051] On the other hand, if the user information in the user record is the same as the user information in the service record (step S107; Yes), the update instruction unit 133 determines that an update is not necessary and skips generating an update command to update the user record of the user.

[0052] 10, in step S109, the update instruction unit 133 determines whether or not the process of loop 1 has been executed for all the service records included in the service database 200 acquired in step S102 (step S109). If the update instruction unit 133 determines that there are unprocessed service records, it executes the process of loop 1 for the unprocessed service records, generates update instructions for updating the user records determined to require updating, and sequentially adds them to the end of the instruction queue.

[0053] On the other hand, if the update instruction unit 133 determines in step S109 that the process of loop 1 has been executed for all users, the process proceeds to step S110. The update instruction unit 133 saves in the storage unit 14 the total number of service records processed in the loop process of loop 1, that is, the number of records included in the service records acquired in step S101, and the skip number, which is the number of users for which it was determined in step S107 that the generation of an update command should be skipped (step S110), and returns to step S101.

[0054] Returning to step S104, if the judgment unit 132 judges not to execute the judgment process (step S104; No), the update instruction unit 133 generates update instructions for updating the user records of each user by the service records of all users included in the service records acquired by the service record acquisition unit 110, the number of which is equal to the number of all users, and sequentially adds them to the end of the instruction queue (step S111). After finishing adding the update instructions for all users to the instruction queue, the update instruction unit 133 returns to step S101.

[0055] The user record update unit 130 reads the update commands added by the update instruction unit 133 from the top, and updates the user records sequentially.

[0056] In the next update process, in step S104, the determination unit 132 may determine whether or not to execute the determination process by comparing (A) the time required for the update process when the determination process is not performed with (B) the time required for the update process when the determination process is performed, based on a skip rate calculated from the total number of service records and the skip count stored in step S110 of the previous update process. The time (A) is the time required for the update process based on all service records to be processed, and the time (B) is the sum of the time required for the determination process for all service records to be processed and the time required for the update process based only on service records determined to require updating. Details of the process in step S104 in this case will be described below with reference to FIG. 12.

[0057] First, the determination unit 132 acquires the total number of service records processed in the previous update process and the skip count from the storage unit 14 (step S1041). Next, the determination unit 132 calculates a skip rate, which is the ratio of the skip count to the total number of service records (step S1042).

[0058] Next, based on the skip rate calculated in step S1042, the judgment unit 132 calculates (A) the time required for the update process when the judgment process is not performed, and (B) the time required for the update process when the judgment process is performed, using the following equations (1) and (2), and judges whether the time (B) is less than the time (A) (step S1043).

[0059] (Time required for update processing when judgment processing is not performed) = (total number of service records to be processed) x (time required to update one service record) Formula (1)

[0060] (Time required for update processing when performing judgment processing) = ((total number of service records to be processed) x (time required for judgment processing for one service record)) + (total number of service records to be processed) x (1 - skip rate) x (time required for update processing per service record) Formula (2)

[0061] In the above formulas (1) and (2), the time required for the update process per service record is, for example, the time required for the process of generating an update command based on one service record, and the time required for the judgment process for one service record is, for example, the time required for the process of reading out a user record for one service record and comparing the service record with the user record. For example, in a preparatory stage before operating the collection device 100, the time required for the process of generating an update command based on one service record and the time required for the judgment process for one service record may be measured, and these times may be set using the measured values. Also, after step S110 or step S111, the operation log of the collection device 100 may be checked, each time may be calculated and stored, and at the time of the next judgment process, the time (A) and the time (B) may be obtained using the time required for the process of generating an update command based on one stored service record and the time required for the judgment process for one service record. In addition, the time required for the process of generating an update command based on one service record and the time required for the judgment process for one service record, which are calculated using the operation logs of the collection device 100 from multiple past operations, may be used to obtain estimated values ​​for each, using a maximum likelihood estimation method or the like, and the time (A) and the time (B) may be obtained using the estimated values ​​obtained.

[0062] In step S1043, it may be determined whether the skip rate calculated in step S1042 is greater than a threshold value. The threshold value may be set to any value, such as 80%, 90%, etc. Also, for example, the minimum skip rate that satisfies (A) the time required for update processing when the determination processing is not performed>(B) the time required for update processing when the determination processing is performed may be set as the threshold value, using the following formula (3).

[0063] (Total number of service records to be processed) x (Time required for update processing per service record) > ((Total number of service records to be processed) x (Time required for judgment processing for one service record)) + (Total number of service records to be processed) x (1 - skip rate) x (Time required for update processing per service record) Formula (3)

[0064] If the judgment unit 132 determines that the time required for the update processing when the judgment processing (B) is performed is less than the time required for the update processing when the judgment processing (A) is not performed (step S1043; Yes), it decides to perform the judgment processing (step S1044). If the judgment unit 132 determines that the time required for the update processing when the judgment processing (B) is performed is equal to or more than the time required for the update processing when the judgment processing (A) is not performed (step S1043; No), it decides not to perform the judgment processing (step S1045).

[0065] Next, a user record update process in the case where a service record is acquired for each unit number will be described with reference to Fig. 11. Note that since the process includes steps common to the flowchart illustrated in Fig. 10, differences will be mainly described.

[0066] In step S201, a determination is made as to whether or not updating of any of the service databases 200a, 200b, 200c, . . . has been completed by the same process as in step S101 (step S201).

[0067] Next, in step S202, the user record update section 130 repeatedly executes the processes of steps S203 to S212 for all the service records in the service database 200 for which it has been determined in step S201 that the update has been completed (step S202).

[0068] In step S203, the service record acquisition unit 110 acquires a unit number of unprocessed service records from the service database 200 (step S203).

[0069] Next, the collection device 100 executes steps S204 to S210, which are the same steps as steps S103 to S109, for each of the unit number of service records acquired in step S203. In step S210, the update instruction unit 133 determines whether or not the process of loop 3 has been executed for all the service records acquired in step S203, and repeatedly executes the process of loop 3 until it is determined that the process of loop 3 has been executed for all the service records.

[0070] Next, in step S211, the user record update unit 130 determines whether or not the process of loop 2 has been executed for all the service records in the service database 200 for which it was determined in step S201 that the update has been completed, and continues the loop until it is determined that the process of loop 2 has been executed for all the service records. 2 The process is repeated.

[0071] Next, the update instruction unit 133, by processing similar to that of step S110, stores in the memory unit 14 the total number of service records processed in the processing of loop 2 and the skip number, which is the number of records for which it was determined that the generation of an update command should be skipped (step S213), and returns to step S201.

[0072] As described above, when starting an update process for updating a user record, the collection device 100 counts up the workload of unprocessed update processes stored in the command queue, and determines whether or not to execute a determination process for determining the necessity of update for each user based on the counting result. If the workload of unprocessed update processes is large, the collection device 100 determines to execute the determination process, skips over the generation of update commands for users who do not need to be updated, and generates only update commands for users who need to be updated and adds them to the command queue. On the other hand, if the workload of unprocessed update processes is small, the collection device 100 determines not to execute the determination process, generates update commands for user records corresponding to each of all service records, and adds them to the command queue. Therefore, depending on the workload of unprocessed update processes that have already been started at the start of update processing for a certain service, it is determined whether to generate update commands for user records corresponding to each of all service records, or to generate update commands for only user records corresponding to service records that need to be updated, so that the process of collecting information on users of multiple services and storing them in a database can be efficiently carried out.

[0073] (Modification) In the above embodiment, the service database 200 has been described as being constructed inside the service providing server that provides each service, but this is not limited to this and the service database 200 may be constructed as an external database server, cloud server, etc. that is accessible from the service providing server via a network.

[0074] In the above embodiment, the user database 300 has been described as being realized by a database server, a cloud server, or the like, but may instead be realized by the storage unit 14 of the collection device 100.

[0075] In the above embodiment, the collection device 100 has been described as having one command queue, reading update commands from the single command queue, and executing update processing, but the collection device 100 may have multiple command queues, take update commands from each command queue, and execute multiple update processes in parallel. In that case, a service association may be set for each command queue, and the update command for each service may be added to each command queue based on the set association.

[0076] In addition, the programs executed by the collection device 100 and the like can be stored and distributed in computer-readable recording media such as CD-ROMs (Compact Disc Read Only Memory), DVDs (Digital Versatile Discs), MOs (Magneto-Optical Disks), USB memories, memory cards, etc. By installing such programs in a specific or general-purpose computer, the computer can be made to function as the collection device 100 in the above embodiment.

[0077] The above-mentioned program may also be stored in a disk device of a server device on a communication network such as the Internet, and downloaded to a computer by, for example, superimposing the program on a carrier wave. The above-mentioned processing can also be achieved by starting and executing the program while transferring it via the communication network. Furthermore, the above-mentioned processing can also be achieved by executing all or a part of the program on a server device, and executing the program while the computer transmits and receives information related to the processing via the communication network.

[0078] In addition, when the above-mentioned functions are shared and realized by the OS (Operating System) or by the OS working together with an application, etc., only the parts other than the OS may be stored on the above-mentioned recording medium and distributed, or may be downloaded to a computer.

[0079] Various aspects of the present disclosure are summarized below as appendices.

[0080] (Appendix 1) A collection device that collects user information from a plurality of services and stores the information in a user database, The user information for each of the plurality of services is stored in a service record stored in a service database for each of the services; The user database has a user record, and stores information derived from the service record of each of the plurality of services for each of the users. a service record acquisition unit that acquires the service record from the service database related to the service to be processed; a user record update unit that updates the user record based on the service record related to the service to be processed acquired by the service record acquisition unit; Equipped with The user record update unit, When starting an update process for updating the user record, tallying up the remaining amount of work for the update process of a service for which the update process has already started but not yet ended among the plurality of services; In the update process based on the service record related to the service to be processed, it is determined whether or not an update to the user record is necessary for each user, and it is determined whether or not a determination process for skipping the update process for a user for which it is determined that an update is not necessary is performed according to the totalized remaining amount of work. Collection device.

[0081] (Appendix 2) the user record update unit starts the update process by adding, to a command queue, an update command for updating the user record for each user based on the service record acquired by the service record acquisition unit. 2. A collection device as described in claim 1.

[0082] (Appendix 3) the user record update unit counts at least one of the number of the services for which the update process is not yet completed, the estimated time required for the update process that is not yet completed, the number of the user records for which the update process is not yet completed, and the length of the command queue, based on the command queue in which the update commands are accumulated, and determines whether or not to perform the determination process, based on the counting result; 10. A collection device as described in claim 2.

[0083] (Appendix 4) When it is determined that the determination process is to be performed, the user record update unit acquiring, from the user database, a user record associated with the service record acquired by the service record acquisition unit, and comparing the service record with the user record for each user to determine, for each user, whether or not the user record needs to be updated; 4. A collection device according to any one of claims 1 to 3.

[0084] (Appendix 5) When it is determined that the determination process is to be performed and the determination process is executed, the user record update unit: recording a total number of the service records acquired by the service record acquisition unit and a skip count of the service records skipped by the determination process; calculating a skip rate indicating a ratio of the skip count to a total number of the recorded service records in the next update process for the service to be processed, and determining whether or not to perform the determination process according to the calculated skip rate; 5. A collection device according to any one of claims 1 to 4.

[0085] (Appendix 6) The user record update unit, based on the calculated skip rate, (A) calculating a time required for the update process based on all the service records to be processed, and (B) performing the determination process on all the service records to be processed and performing the update process based only on the service records determined to require updating, and deciding to perform the determination process when it is determined that the time (B) is shorter than the time (A); 6. A collection device as described in claim 5.

[0086] (Appendix 7) When it is determined not to perform the determination process, updating the user record based on the service records of all users included in the service record acquired by the service record acquisition unit; 7. A collection device according to any one of claims 1 to 6.

[0087] (Appendix 8) A computer that is a collection device that collects user information from a plurality of services and stores the collected information in a user database, acquiring a service record storing information of the user for each of the plurality of services from a service database related to the service to be processed; updating a user record stored in the user database based on the acquired service record related to the service to be processed, the user record including information derived from the service record of each of the plurality of services for each user; Including, The step of updating the user record includes: When starting an update process for updating the user record, tallying up the remaining amount of work for the update process of a service for which the update process has already started but not yet ended among the plurality of services; In the update process based on the service record related to the service to be processed, it is determined whether or not an update to the user record is necessary for each user, and it is determined whether or not a determination process for skipping the update process for a user for which it is determined that an update is not necessary is performed according to the totalized remaining amount of work. Collection methods.

[0088] (Appendix 9) A computer that is a collection device that collects user information from a plurality of services and stores the collected information in a user database, A process of acquiring a service record storing information of the user for each of the plurality of services from a service database related to the service to be processed; updating a user record stored in the user database based on the acquired service record related to the service to be processed, the user record including information derived from the service record of each of the plurality of services for each user; Run the command, The process of updating the user record includes: When starting an update process for updating the user record, tallying up the remaining amount of work for the update process of a service for which the update process has already started but not yet ended among the plurality of services; In the update process based on the service record related to the service to be processed, it is determined whether or not an update to the user record is necessary for each user, and it is determined whether or not a determination process for skipping the update process for a user for which it is determined that an update is not necessary is performed according to the totalized remaining amount of work. program.

[0089] Various embodiments and modifications of the present disclosure are possible without departing from the broad spirit and scope of the present disclosure. The above-described embodiments are for explaining the present disclosure and do not limit the scope of the present disclosure. That is, the scope of the present disclosure is indicated by the claims, not the embodiments. Various modifications made within the scope of the claims and the scope of the disclosure equivalent thereto are considered to be within the scope of the present disclosure. [Industrial Applicability]

[0090] The present invention can be suitably adopted in a collection device, a collection method, and a program that collect information on users of a plurality of services from a service database and store the information in a user database. [Explanation of symbols]

[0091] 100 collection device, 200, 200a, 200b, 200c service database, 300 user database, 400 communication network, 110 service record acquisition unit, 120 service record storage unit, 130 user record update unit, 131 remaining work amount calculation unit, 132 judgment unit, 133 update instruction unit, 11 CPU, 12 RAM, 13 ROM, 14 storage unit, 15 input unit, 16 display unit, 17 communication unit, 99 internal bus

Claims

1. A collection device that collects user information from a plurality of services and stores the information in a user database, The user information for each of the plurality of services is stored in a service record stored in a service database for each of the services; The user database has a user record, and stores information derived from the service record of each of the plurality of services for each of the users. a service record acquisition unit that acquires the service record from the service database related to the service to be processed; a user record update unit that updates the user record based on the service record related to the service to be processed acquired by the service record acquisition unit; Equipped with The user record update unit, When starting an update process for updating the user record, tallying up the remaining amount of work in the update process of a service for which the update process has already started but not yet ended among the plurality of services; In the update process based on the service record related to the service to be processed, it is determined whether or not an update to the user record is necessary for each user, and it is determined whether or not a determination process for skipping the update process for a user for which it is determined that an update is not necessary is performed according to the totalized remaining amount of work. Collection device.

2. the user record update unit starts the update process by adding, to a command queue, an update command for updating the user record for each user based on the service record acquired by the service record acquisition unit. The collection device of claim 1 .

3. the user record update unit counts at least one of the number of the services for which the update process is not yet completed, the estimated time required for the update process that is not yet completed, the number of user records for which the update process is not yet completed, and the length of the command queue, based on the command queue in which the update commands are accumulated, and determines whether or not to perform the determination process, based on the counting result; The collection device of claim 2 .

4. When it is determined that the determination process is to be performed, the user record update unit: acquiring, from the user database, a user record associated with the service record acquired by the service record acquisition unit, and comparing the service record with the user record for each user to determine, for each user, whether or not the user record needs to be updated; A collection device according to any one of claims 1 to 3.

5. When it is determined that the determination process is to be performed and the determination process is executed, the user record update unit: recording a total number of the service records acquired by the service record acquisition unit and a skip count of the service records skipped by the determination process; calculating a skip rate indicating a ratio of the skip count to a total number of the recorded service records in the next update process for the service to be processed, and determining whether or not to perform the determination process according to the calculated skip rate; A collection device according to claim 1 or 2.

6. The user record update unit, based on the calculated skip rate, (A) calculating a time required for the update process based on all the service records to be processed, and (B) performing the determination process on all the service records to be processed and performing the update process based only on the service records determined to require updating, and if it is determined that the time required for (B) is shorter than the time required for (A), determining to perform the determination process; The collection device of claim 5.

7. When it is determined not to perform the determination process, updating the user record based on the service records of all users included in the service record acquired by the service record acquisition unit; A collection device according to claim 1 or 2.

8. A computer that is a collection device that collects user information from a plurality of services and stores the collected information in a user database, acquiring a service record storing information of the user for each of the plurality of services from a service database related to the service to be processed; updating a user record stored in the user database based on the acquired service record related to the service to be processed, the user record including information derived from the service record of each of the plurality of services for each user; Including, The step of updating the user record includes: When starting an update process for updating the user record, tallying up the remaining amount of work in the update process of a service for which the update process has already started but not yet ended among the plurality of services; In the update process based on the service record related to the service to be processed, it is determined whether or not an update to the user record is necessary for each user, and it is determined whether or not a determination process for skipping the update process for a user for which it is determined that an update is not necessary is performed according to the totalized remaining amount of work. Collection methods.

9. A computer that is a collection device that collects user information from a plurality of services and stores the collected information in a user database, A process of acquiring a service record storing information of the user for each of the plurality of services from a service database related to the service to be processed; updating a user record stored in the user database based on the acquired service record related to the service to be processed, the user record including information derived from the service record of each of the plurality of services for each user; Run the command, The process of updating the user record includes: When starting an update process for updating the user record, tallying up the remaining amount of work in the update process of a service for which the update process has already started but not yet ended among the plurality of services; In the update process based on the service record related to the service to be processed, it is determined whether or not an update to the user record is necessary for each user, and it is determined whether or not a determination process for skipping the update process for a user for which it is determined that an update is not necessary is performed according to the totalized remaining amount of work. program.

Images