Information processing device and mobile terminal

Abstract

To solve the problem in which conventionally, it has not been possible to provide appropriate information about an operational status of a store to one or more mobile devices.SOLUTION: An information processing device 1 comprises: an image receiving unit 121 which receives an image captured during movement from each of two or more mobile devices; a status acquisition unit which acquires status information for identifying an operational status of a store in the image; an identifier acquisition unit 132 which acquires a store identifier for identifying a store corresponding to the status information; a time acquisition unit 133 which acquires time information on a time corresponding to the operational status; a status accumulation unit 134 which accumulates the status information in a store management unit 111 for storing status information of each of two or more stores, in association with the time information and the store identifier; and a status transmission unit 141 which transmits the status information associated with each of one or more pieces of time information and with the store identifier to one or more mobile devices 2. With this configuration, the information processing device can provide one or more mobile terminals with appropriate information about the operational status of the store.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device or the like that acquires the business status of a store and transmits the acquired information to one or more mobile terminals. [Background technology]

[0002] BACKGROUND ART Conventionally, there has been an in-vehicle device that outputs information indicating the size of a queue at a facility based on an image captured by an imaging unit mounted on the vehicle (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-081052 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the prior art, it was not possible to provide appropriate information on the business status of a store to one or more mobile terminals. In particular, in the prior art, it was not possible to provide the statistical results obtained by statistically processing the store's status information to one or more mobile terminals. Furthermore, in the prior art, it was not possible to provide the statistical results obtained by statistically processing the store's status information for each of two or more time periods to one or more mobile terminals. [Means for solving the problem]

[0005] The information processing device of the first invention is an information processing device comprising: an image receiving unit that receives images taken while moving from two or more mobile terminals; a status acquisition unit that acquires status information specifying the business status of the store in the image; an identifier acquisition unit that acquires a store identifier that identifies the store corresponding to the status information; a time acquisition unit that acquires time information related to the time corresponding to the business status; a store management unit that stores status information for two or more stores; a status accumulation unit that accumulates the status information in association with the time information and the store identifier; and a status transmission unit that transmits status information associated with one or more pieces of time information and associated with the store identifier to one or more mobile terminals.

[0006] With this configuration, appropriate information on the business status of popular stores, newly opened stores, and the like can be provided to one or more mobile terminals.

[0007] In addition, the information processing device of the second invention is an information processing device in which, compared to the first invention, the status acquisition unit acquires status information corresponding to two or more different pieces of time information for one store, acquires status information corresponding to a store identifier that identifies the one store and pairs with the two or more pieces of time information from the store management unit, performs statistical processing on the two or more pieces of status information, and acquires statistical results, and a statistical transmission unit transmits the statistical results corresponding to the one store identifier.

[0008] With this configuration, it is possible to provide statistical results obtained by statistically processing store status information.

[0009] Furthermore, in the information processing device of the third invention, compared to the second invention, the time information is information that identifies a time period, the statistical processing unit performs statistical processing on the status information corresponding to each of two or more time periods to obtain statistical results, and the statistical transmission unit transmits the statistical results for each of one or more time periods.

[0010] With this configuration, it is possible to provide statistical results obtained by statistically processing store status information for each time period.

[0011] Furthermore, the mobile terminal of the fourth invention is a mobile terminal that includes a terminal storage unit in which store conditions, which are conditions related to a store, are stored, a terminal receiving unit that receives status information or statistical results corresponding to a store identifier, a terminal judgment unit that determines whether the store identified by the store identifier satisfies the store conditions, and a terminal output unit that outputs the status information or statistical results only if the terminal judgment unit determines that the store conditions are met.

[0012] With this configuration, it is possible to obtain status information or statistical results for stores that meet the stored store conditions.

[0013] Furthermore, in the mobile terminal of the fifth invention, compared to the fourth invention, the terminal storage unit stores route information that identifies the route to the destination, the store condition is that the store is located on or near the route, and the terminal judgment unit acquires the store location that pairs with the store identifier and uses the store location to judge whether the store condition is met.

[0014] With this configuration, it is possible to obtain status information or statistical results for stores on or around the route to the destination. [Effects of the Invention]

[0015] According to the information processing device of the present invention, appropriate information on the business status of stores such as popular stores and newly opened stores can be provided to one or more mobile terminals. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram of an information system A according to the first embodiment. [Figure 2] An example of a block diagram for Information System A [Figure 3] 1 is a flowchart illustrating an example of the operation of the information processing device 1. [Figure 4] 10 is a flowchart illustrating an example of a status acquisition process. [Figure 5] 10 is a flowchart illustrating an example of an identifier acquisition process. [Figure 6] 10 is a flowchart illustrating an example of a transmission process. [Figure 7] 10 is a flowchart illustrating an example of a transmission information acquisition process. [Figure 8] 10 is a flowchart illustrating an example of the operation of the mobile terminal 2. [Figure 9] 10 is a flowchart illustrating an example of an output process. [Figure 10] An example of a terminal management table [Figure 11] An example of a store management table [Figure 12] An example of the statistical management table [Figure 13] A diagram showing an example of an image [Figure 14] A diagram showing an example of a display [Figure 15] An example of a computer system block diagram DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, embodiments of an information processing device and the like will be described with reference to the drawings. Note that components with the same reference numerals in the embodiments perform similar operations, and therefore repeated description may be omitted.

[0018] (Embodiment 1) 1 is a conceptual diagram of an information system A according to this embodiment. The information system A includes an information processing device 1 and one or more mobile terminals 2.

[0019] The information processing device 1 is a server that acquires situation information specifying the business status of a store using images captured while moving, and provides the situation information to one or more mobile terminals 2. The information processing device 1 is, for example, a cloud server or an ASP server, but the type is not important.

[0020] The mobile terminal 2 is a terminal that transmits images captured while moving to the information processing device 1. The mobile terminal 2 is also a terminal that receives and outputs status information related to the business status of a store. The mobile terminal 2 is usually mounted on a moving object. The moving object is, for example, a car, a truck, a bicycle, or a train, but it can also be a person. In other words, the mobile terminal 2 can also be a terminal carried by a person. The mobile terminal 2 is, for example, a navigation terminal, a drive recorder, a smartphone, a personal computer, or a tablet terminal, but the type is not important.

[0021] Furthermore, the information processing device 1 and one or more mobile terminals 2 can communicate with each other via a network such as the Internet.

[0022] 2 is a block diagram of an information system A according to the present embodiment. The information processing device 1 includes a storage unit 11, a receiving unit 12, a processing unit 13, and a transmitting unit 14. The storage unit 11 includes a store management unit 111. The receiving unit 12 includes an image receiving unit 121. The processing unit 13 includes a status acquisition unit 131, an identifier acquisition unit 132, a time acquisition unit 133, a status accumulation unit 134, a statistical processing unit 135, and a transmission information acquisition unit 136. The transmitting unit 14 includes a status transmission unit 141 and a statistical transmission unit 142.

[0023] The mobile terminal 2 includes a terminal storage unit 21, a terminal reception unit 22, a terminal processing unit 23, a terminal transmission unit 24, a terminal reception unit 25, and a terminal output unit 26. The terminal processing unit 23 includes a terminal determination unit 231.

[0024] Various types of information are stored in the storage unit 11 constituting the information processing device 1. The various types of information are, for example, one or more pieces of store information, which will be described later, or one or more pieces of terminal information.

[0025] The terminal information is information relating to the mobile terminal 2 for which status information is to be provided. The terminal information includes, for example, a terminal identifier, a terminal position, and route information.

[0026] The terminal identifier is information that identifies the mobile terminal 2. The terminal identifier is, for example, the ID of the mobile terminal 2. The terminal location is information that specifies the location of the mobile terminal 2. The terminal location is, for example, (latitude, longitude) or (latitude, longitude, altitude). The route information is information that specifies the route set by the mobile terminal 2. The route is the route to the destination. The route information is, for example, a collection of two or more pieces of location information. The location information is, for example, (latitude, longitude) or (latitude, longitude, altitude).

[0027] The store management unit 111 stores one or more pieces of store information. Store information is information related to a store. The store information has, for example, a store identifier and one or more store attribute values. The store information has, for example, one or more pieces of situation information and one or more statistical results. It is preferable that the situation information corresponds to time information. It is preferable that the statistical results correspond to time information. The store attribute values ​​are, for example, the store name, store location, and store type.

[0028] The store identifier is information that identifies a store, such as a store ID or a store name.

[0029] The status information is information that specifies the business status of a store. The status information is, for example, information indicating whether there is a line, the number of people in the line, the number of people in the store, or information indicating whether the store is open.

[0030] The statistical results are information resulting from statistical processing of situation information. Examples of statistical results include the degree of congestion at a store, the average number of people in line, and the percentage of people queuing (which can also be a probability or likelihood). It is preferable that the statistical results be associated with time information indicating the time period.

[0031] Time information is information relating to time. For example, time information is information specifying a time period, but it may also be information specifying the time of day. A time period may be one hour or 30 minutes. The length of time within a time period does not matter. It is preferable that the time interval between two or more time periods is constant, but it does not have to be constant.

[0032] The store location is information that identifies the location of the store. The store location has, for example, one or more (latitude, longitude).

[0033] The store type is information that identifies the type of store, such as "ramen shop," "restaurant," "sushi shop," or "bakery."

[0034] The receiving unit 12 receives various types of information and instructions, such as images, transmission instructions, or terminal information, which will be described later.

[0035] The transmission instruction is an instruction to transmit status information or statistical results. The transmission instruction includes, for example, one or more pieces of location information. The transmission instruction includes, for example, route information. The route information is information that identifies a route set in the mobile terminal 2.

[0036] The terminal information is information about the mobile terminal 2. The terminal information includes, for example, a terminal identifier and route information. The terminal information includes, for example, a terminal identifier and a terminal location.

[0037] The image receiving unit 121 receives images taken during movement from each of two or more mobile terminals 2. It is preferable that the terminal taking the images is the mobile terminal 2, but it may be a terminal other than the mobile terminal 2. The terminal other than the mobile terminal 2 is, for example, a drive recorder (not shown).

[0038] The image is preferably a still image, but may be a video. Note that the image received by the image receiving unit 121 is usually associated with location information. The location information is usually (latitude, longitude) or (latitude, longitude, altitude). The location information associated with the image is usually information that can identify the location of the terminal that captured the image. The image received by the image receiving unit 121 is usually associated with time information. The time information here is, for example, the time. The time is the time when the image was captured.

[0039] The processing unit 13 performs various processes. For example, the processing unit 13 performs the following store information addition process.

[0040] The store information addition process is a process of adding store information to the store management unit 111. The processing unit 13 generates a unique store identifier. The processing unit 13 also acquires location information paired with the received image as the store location. The processing unit 13 also acquires the store name and store type using the received image. Next, the processing unit 13 creates store information having the acquired store identifier, store name, store location, and store type, and adds the store information to the store management unit 111. Note that the processing unit 13 preferably adds the store information to the store management unit 111 only when store information including the store name acquired from the received image does not exist in the store management unit 111. Also, if the store is so crowded that the situation information acquired by the situation acquisition unit 131 described below satisfies a congestion condition (e.g., "the number of people in line is above a threshold" or "the congestion level is above a threshold"), it is preferable to add the store information of the store to the store management unit 111. Note that the congestion condition is a condition for determining that the store is crowded.

[0041] The status acquisition unit 131 acquires status information that specifies the business status of the store in the image, using the image received by the image receiving unit 121. The status acquisition unit 131 preferably acquires the status information in association with time information. The status acquisition unit 131 preferably acquires the status information in association with a store identifier.

[0042] The situation acquisition unit 131, for example, performs image recognition processing on the image received by the image receiving unit 121 to recognize the store and acquire situation information that identifies the business status of the store. The situation acquisition unit 131 acquires the number of people in line from the image, for example, using the technology described in Patent Document 1. The situation acquisition unit 131, for example, performs image recognition processing on the image received by the image receiving unit 121 to detect people in front of the store and acquire the number of people in line, which is the number of people in front of the store. The situation acquisition unit 131, for example, performs character recognition processing on the image received by the image receiving unit 121 to acquire one or more character strings, acquire the class of each character string, and acquire the character string corresponding to the class, which is the store name, as the store name. Note that the technology for acquiring the class of a character string is a publicly known technology, such as using machine learning prediction processing, and therefore detailed description thereof will be omitted. The situation acquisition unit 131, for example, estimates the store type from the store name. Such a technology is also a publicly known technology, such as using machine learning prediction processing, and therefore detailed description thereof will be omitted.

[0043] The situation acquisition unit 131 may create a prompt including the image received by the image receiving unit 121, pass the prompt to the generation AI, and acquire situation information from the generation AI. The generation AI is an AI with a large-scale language model, and when a prompt is given, it outputs an answer corresponding to the prompt. The generation AI is, for example, ChatGPT (registered trademark), but is not limited thereto.

[0044] For example, the situation acquisition unit 131 reads a prompt template "Please tell me in bullet points whether the store in the image below is open or not, the type of store, and the number of people in line!" from the storage unit 11, composes a prompt having the prompt template and an image placed below the prompt template, passes the prompt to the generation AI, and acquires situation information "<whether open or not><type of store><number of people in line>" from the generation AI. Note that the situation information is, for example, "whether open or not: open, type of store: ramen shop, number of people in line: 4."

[0045] For example, the situation acquisition unit 131 reads a prompt template "Please tell me in bullet points whether the store in the image below is open or not, the name of the store, the type of store, and the number of people in line!" from the storage unit 11, composes a prompt having the prompt template and an image placed below the prompt template, passes the prompt to the generation AI, and acquires situation information "<whether open or not><name of store><type of store><number of people in line>" from the generation AI. Note that the situation information is, for example, "whether open or not: open, name of store: Ramen XY, type of store: ramen store, number of people in line: 4."

[0046] It is preferable that the situation acquisition unit 131 acquires situation information corresponding to two or more different pieces of time information for one or two or more stores. The images from which such two or more pieces of situation information are acquired are usually images transmitted from two or more mobile terminals 2.

[0047] The identifier acquisition unit 132 acquires a store identifier that identifies a store corresponding to the situation information. For example, the identifier acquisition unit 132 acquires location information associated with the received image, and acquires from the store management unit 111 a store identifier that is paired with location information that matches the location information or location information whose distance is within a threshold.

[0048] For example, the identifier acquisition unit 132 uses the type of store acquired by the situation acquisition unit 131 and the location information corresponding to the received image to acquire from the store management unit 111 a store identifier that pairs with the type of store and with location information that matches the location information or location information whose distance is within a threshold.

[0049] For example, the identifier acquisition unit 132 uses the name of the store acquired by the situation acquisition unit 131 and the location information corresponding to the received image to acquire from the store management unit 111 a store identifier that is paired with the name of the store and is paired with location information that matches the location information or location information whose distance is within a threshold.

[0050] The time acquisition unit 133 acquires time information related to the time corresponding to the business status. For example, when the image receiving unit 121 receives an image, the time acquisition unit 133 acquires the current time from a clock (not shown) and acquires time information (for example, a time zone) based on the current time. For example, the time acquisition unit 133 acquires the time paired with the image received by the image receiving unit 121 and acquires time information (for example, a time zone) based on the time.

[0051] The status accumulation unit 134 accumulates the status information acquired by the status acquisition unit 131 in the store management unit 111 in association with the time information acquired by the time acquisition unit 133 and the store identifier acquired by the identifier acquisition unit 132 .

[0052] The statistical processing unit 135 acquires from the store management unit 111 situation information that corresponds to a store identifier that identifies one or more stores and pairs with two or more pieces of time information, performs statistical processing on the two or more pieces of situation information, and acquires statistical results. The statistical results are, for example, the congestion level and the average number of people in line. Note that the larger the number of people in line, the larger the congestion level acquired by the statistical processing unit 135. The statistical processing unit 135 calculates the congestion level, for example, using an increasing function that uses the number of people in line as a parameter. The statistical processing unit 135, for example, references a correspondence table consisting of two or more pieces of correspondence information that have widths of people in line and congestion levels, determines the width of the number of people in line that includes the acquired number of people in line, and acquires the congestion level that pairs with the width from the correspondence table.

[0053] The statistical processing unit 135 performs statistical processing on the situation information associated with each time period for, for example, one or more stores, for each of two or more time periods, and obtains statistical results.

[0054] The transmission information acquisition unit 136 acquires transmission information, which is information to be transmitted to the mobile terminal 2 that has transmitted the transmission instruction or terminal information.

[0055] The transmission information acquisition unit 136 acquires, from the store management unit 111, for example, status information of stores in the vicinity of the location information of the mobile terminal 2 contained in the transmission instruction or the terminal information.

[0056] The transmission information acquisition unit 136 acquires, from the store management unit 111, for example, status information of stores on or around the route specified by the route information of the mobile terminal 2 included in the transmission instruction or terminal information.

[0057] The transmission information acquisition unit 136 acquires, for example, status information of stores on or around the route specified by the route information of the mobile terminal 2 contained in the transmission instruction or the terminal information, and status information of a time period during which the mobile terminal 2 is predicted to pass while traveling, from the store management unit 111. Details of the processing of the transmission information acquisition unit 136 will be described later using the flowchart in FIG. 7.

[0058] The transmitter 14 transmits various types of information, such as status information, statistical results, and transmission information.

[0059] The status transmission unit 141 transmits status information associated with one or more pieces of time information and associated with a store identifier to one or more mobile terminals 2.

[0060] The statistics transmission unit 142 transmits the statistical results associated with one store identifier to one or more mobile terminals 2.

[0061] The statistics transmission unit 142 transmits the statistical results for one or more time periods to one or more mobile terminals 2.

[0062] The statistics transmission unit 142 transmits the statistical results for one or more time periods at one or more stores to one or more mobile terminals 2.

[0063] Store conditions are stored in the terminal storage unit 21 that constitutes the mobile terminal 2. Store conditions are conditions related to stores. Store conditions are, for example, conditions related to the set route or conditions related to the user's preferences. Store conditions are, for example, "located on the set route," "distance or travel time from the set route is within a threshold," and "store type = ramen shop."

[0064] The terminal storage unit 21 stores, for example, route information that specifies a route to a destination. The destination information contained in the route information is information set by the user. The process for storing route information is a well-known navigation technique, and therefore, a description thereof will be omitted.

[0065] The terminal storage unit 21 stores, for example, a map database. The map database is a database used, for example, for navigation. The map database has, for example, information on one or more stores. The store information may include statistical results and situation information for each time period. The store information may also be called POI information.

[0066] The terminal reception unit 22 receives various instructions and information, such as a transmission instruction and a destination.

[0067] The means for inputting various instructions and information may be any means, such as a touch panel, keyboard, mouse, or menu screen.

[0068] The terminal processing unit 23 performs various types of processing. For example, the various types of processing are processing to convert received information, instructions, etc. into information, instructions, etc. with a structure to be transmitted. For example, the various types of processing are processing to convert received information into information with a structure to be output.

[0069] When the terminal reception unit 22 receives a destination, the device processing unit 23 acquires route information from the current location to the destination, and stores the route information in the terminal storage unit 21. In other words, the device processing unit 23 performs a process of setting the route.

[0070] The device processing unit 23 stores, for example, situation information associated with the store identifier received by the terminal receiving unit 25 or statistical results associated with the store identifier in the map database. The device processing unit 23 stores, for example, situation information for each time period associated with the store identifier received by the terminal receiving unit 25 or statistical results for each time period associated with the store identifier in the map database. This allows the user to obtain information that estimates the degree of congestion at the store for each time period.

[0071] The terminal determination unit 231 determines whether the store identified by the store identifier satisfies the store conditions. For example, the terminal determination unit 231 obtains the store location paired with the store identifier and uses the store location to determine whether the store conditions are satisfied.

[0072] The terminal transmitting unit 24 transmits various types of information, instructions, etc. to the information processing device 1. The various types of information, instructions, etc. are images and transmission instructions.

[0073] The terminal receiving unit 25 receives status information associated with the store identifier or statistical results associated with the store identifier. Such status information may be status information indicating the current status or status information predicting the future. The statistical results may be information paired with time information corresponding to the current time, or information paired with two or more pieces of time information.

[0074] The terminal receiving unit 25 receives status information of one or more pieces of time information (for example, time zones) associated with the store identifier, or statistical results of one or more pieces of time information (for example, time zones) associated with the store identifier.

[0075] The terminal output unit 26 outputs the status information or the statistical results. The status information or the statistical results are associated with the store identifier.

[0076] The terminal output unit 26 outputs the status information in association with, for example, one or more pieces of time information. The terminal output unit 26 outputs the statistical result in association with, for example, one or more pieces of time information.

[0077] It is preferable that the terminal output unit 26 outputs the status information or statistical results only when the terminal determination unit 231 determines that the store conditions are met.

[0078] Here, output usually means display on a display, but it may also be a concept that includes projection using a projector, printing on a printer, sound output, transmission to an external device, storage on a recording medium, and handing over of processing results to other processing devices or other programs.

[0079] The storage unit 11, the store management unit 111, and the terminal storage unit 21 are preferably non-volatile recording media, but may also be realized as volatile recording media.

[0080] There is no restriction on the process by which information is stored in the storage unit 11 etc. For example, information may be stored in the storage unit 11 etc. via a recording medium, information transmitted via a communication line etc. may be stored in the storage unit 11 etc., or information input via an input device may be stored in the storage unit 11 etc.

[0081] The receiving unit 12, the image receiving unit 121, and the terminal receiving unit 25 are usually realized by wireless or wired communication means, but may also be realized by means for receiving broadcasts.

[0082] The processing unit 13, status acquisition unit 131, identifier acquisition unit 132, time acquisition unit 133, status accumulation unit 134, statistical processing unit 135, device processing unit 23, and device determination unit 231 can usually be realized by a processor, memory, etc. The processing procedures of the processing unit 13, etc. are usually realized by software, and the software is recorded on a recording medium such as a ROM. However, they may also be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type is not important.

[0083] The transmitting unit 14, the status transmitting unit 141, the statistics transmitting unit 142, and the terminal transmitting unit 24 are usually realized by wireless or wired communication means, but may also be realized by broadcasting means.

[0084] The terminal reception unit 22 can be realized by a device driver for an input means such as a touch panel or a keyboard, or control software for a menu screen.

[0085] The terminal output unit 26 may or may not include an output device such as a display, a speaker, etc. The terminal output unit 26 may be realized by driver software for an output device, or by a combination of driver software for an output device and the output device, etc.

[0086] Next, an example of the operation of the information processing device 1 will be described with reference to the flowchart of FIG.

[0087] (S301) The image receiving unit 121 determines whether or not an image has been received from the mobile terminal 2. If an image has been received, the process proceeds to S302, and if not, the process proceeds to S309.

[0088] (S302) The situation acquisition unit 131 acquires a still image. If the image received in S301 is a moving image, one still image is acquired from the moving image.

[0089] (S303) The status acquisition unit 131 acquires status information, etc. Details of the status acquisition process will be described later.

[0090] (S304) The identifier acquisition unit 132 acquires a store identifier. Details of this identifier acquisition process will be described later.

[0091] (S305) The time acquisition unit 133 acquires the time paired with the image received in S301, or acquires the current time from a clock (not shown). Next, the time acquisition unit 133 acquires time information that identifies the time zone corresponding to the acquired time.

[0092] (S306) The situation accumulation unit 134 accumulates the situation information acquired in S303 in the store management unit 111 in association with the store identifier acquired in S304 and the time information acquired in S305.

[0093] (S307) The processing unit 13 determines whether the status information satisfies the transmission condition. If the transmission condition is satisfied, the process proceeds to S308, and if not, the process returns to S301. The transmission condition is, for example, that the status information is information indicating that the area is so crowded that it satisfies a predetermined condition. Such a transmission condition is, for example, "there is a queue," "the number of people in the queue is equal to or greater than a threshold," or "the degree of congestion is equal to or greater than a threshold." The transmission condition may be the same as the congestion condition described above.

[0094] (S308) Status transmission unit 141 transmits the status information and the like to one or more mobile terminals 2. Details of this transmission process will be described later.

[0095] (S309) The statistical processing unit 135 determines whether or not to perform statistical processing. If statistical processing is to be performed, the process proceeds to S310, and if not, the process proceeds to S319. Statistical processing may be performed, for example, when a predetermined time (for example, 12:00 on the first day of each month) arrives or when an instruction from the administrator is received.

[0096] (S310) The statistical processing unit 135 assigns 1 to the counter i.

[0097] (S311) The statistical processing unit 135 determines whether or not the i-th store information exists in the store management unit 111. If the i-th store information exists, the process proceeds to S312; if not, the process returns to S301.

[0098] (S312) The statistical processing unit 135 assigns 1 to the counter j.

[0099] (S313) The statistical processing unit 135 determines whether or not there is time information identifying the jth time period. If there is jth time information, the process proceeds to S314; if there is not, the process proceeds to S318. What kind of time information exists in the time information identifying the time period is determined in advance.

[0100] (S314) The statistical processing unit 135 acquires from the store management unit 111 all pieces of situation information corresponding to the i-th store information and the j-th time information.

[0101] (S315) The statistical processing unit 135 performs statistical processing on all the situation information acquired in S314, and acquires statistical results (for example, the average number of people in a queue, the probability that a queue exists, and the degree of congestion).

[0102] (S316) The statistical processing unit 135 stores the statistical results acquired in S315 in the store management unit 111 in association with the i-th store information and the j-th time information.

[0103] (S317) The statistical processing unit 135 increments the counter j by 1. The process returns to S313.

[0104] (S318) The statistical processing unit 135 increments the counter i by 1. The process returns to S311.

[0105] (S319) The receiving unit 12 determines whether or not a transmission instruction has been received from the mobile terminal 2. If a transmission instruction has been received, the process proceeds to S320, and if not, the process proceeds to S322.

[0106] (S320) The transmission information acquisition unit 136 acquires the transmission information. Details of the transmission information acquisition process will be described later.

[0107] (S321) The transmitter 14 transmits the transmission information acquired in S320 to the mobile terminal 2 that has transmitted the transmission instruction. Return to S301.

[0108] (S322) The receiving unit 12 determines whether or not terminal information has been received from the mobile terminal 2. If terminal information has been received, the process proceeds to S323; if not, the process returns to S301. Note that the terminal information received here may be part of the terminal information stored in the storage unit 11. The terminal information may be, for example, a terminal identifier and a terminal position, or a terminal identifier and route information.

[0109] (S323) The processing unit 13 stores the terminal position and / or route information received in S322 in the storage unit 11 in pairs with the terminal identifier of the mobile terminal 2 that has transmitted the terminal information.

[0110] (S324) The processing unit 13 determines whether or not to transmit information to the mobile terminal 2 that has transmitted the terminal information. If the information is to be transmitted, the process proceeds to S20, and if the information is not to be transmitted, the process returns to S301. Note that the processing unit 13 may always determine to transmit information.

[0111] In the flowchart of FIG. 3, the process ends when the power is turned off or an interrupt occurs to end the process.

[0112] Next, an example of the status acquisition process in S303 will be described with reference to the flowchart in FIG.

[0113] (S401) The situation acquisition unit 131 recognizes a store in an image by image recognition processing.

[0114] (S402) The status acquisition unit 131 inspects the inside or surrounding area of ​​the store recognized in S401, determines whether the store is open for business, and acquires the determination result.

[0115] (S403) If the store is open, proceed to S404, and if not, proceed to S405.

[0116] (S404) The situation acquisition unit 131 inspects the inside of the store recognized in S401 and / or the surrounding area of ​​the store, and acquires the number of people.

[0117] (S405) The status acquisition unit 131 uses the acquired information to create status information, and returns to the upper level processing.

[0118] 4, as described above, the situation acquisition unit 131 may acquire situation information using a generation AI. Also, in the flowchart of FIG. 4, the situation acquisition unit 131 may acquire the number of people in line even when it determines that the store is closed. In other words, S403 may not be present, and the process may proceed from S402 to S404.

[0119] Next, an example of the identifier acquisition process in S304 will be described with reference to the flowchart in FIG.

[0120] (S501) The identifier acquisition unit 132 acquires location information paired with a received image.

[0121] (S502) The identifier acquisition unit 132 uses the location information acquired in S501 to search for a store location close to the location information and acquires the store name paired with the store location. Note that a store location close to the location information is a store location whose distance from the location information is within a threshold.

[0122] (S503) The identifier acquisition unit 132 determines whether or not the store name was acquired in S502. If the store name was acquired, the process returns to the upper process, and if not, the process proceeds to S504.

[0123] (S504) The identifier acquisition unit 132 determines whether the situation information acquired in S303 satisfies the congestion condition. If the congestion condition is satisfied, the process proceeds to S505, and if not, the process proceeds to S514. The congestion condition is a condition for determining that congestion is occurring. Examples of the congestion condition include the number of people in line being equal to or greater than a threshold, or the existence of a line.

[0124] (S505) The identifier acquisition unit 132 assigns 1 to a counter i.

[0125] (S506) The identifier acquisition unit 132 determines whether the i-th SNS post exists in the SNS server (not shown). If the i-th SNS post exists, the process proceeds to S507; if not, the process proceeds to S513.

[0126] (S507) The identifier acquisition unit 132 acquires a posting location, which is location information paired with the i-th SNS post.

[0127] (S508) The identifier acquisition unit 132 determines whether the posting location acquired in S507 satisfies a location condition (distance is within a threshold) with respect to the location information acquired in S501. If the location condition is satisfied, proceed to S509; if not, proceed to S512.

[0128] (S509) The identifier acquisition unit 132 performs morphological analysis on the i-th SNS post to acquire the store name. Note that there are cases where the store name cannot be acquired here.

[0129] (S510) The identifier acquisition unit 132 determines whether or not the store name was acquired in S509. If the store name was acquired, the process proceeds to S511; if not, the process proceeds to S512.

[0130] (S511) The identifier acquisition unit 132 temporarily stores the store name in a buffer (not shown).

[0131] (S512) The identifier acquisition unit 132 increments the counter i by 1. The process returns to S506.

[0132] (S513) The identifier acquisition unit 132 acquires the most common store name among the store names temporarily stored in a buffer (not shown), and returns to the upper-level processing.

[0133] (S514) The identifier acquisition unit 132 assigns “NULL” to the store name, and returns to the upper-level process.

[0134] Next, an example of the transmission process in S308 will be described with reference to the flowchart in FIG.

[0135] (S601) The processing unit 13 acquires the location information of the store. Note that the location information of the store here may be location information paired with the received image.

[0136] (S602) The processing unit 13 assigns 1 to the counter i.

[0137] (S603) The processing unit 13 determines whether or not the terminal information of the i-th mobile terminal exists in the storage unit 11. If the terminal information of the i-th mobile terminal exists, the process proceeds to S604, and if not, the process returns to the upper processing.

[0138] (S604) The processing unit 13 determines whether the terminal information of the i-th mobile terminal includes route information transmitted from the mobile terminal 2. If the route information exists, the process proceeds to S605; if not, the process proceeds to S606.

[0139] (S605) The processing unit 13 acquires the latest location information and route information from the terminal information of the i-th mobile terminal, and determines whether or not a location specified by the store's location information exists on or near the route from the latest location to the destination. If the location exists, the process proceeds to S607; if not, the process proceeds to S608.

[0140] (S606) The processing unit 13 acquires the latest location information from the information of the i-th mobile terminal, and determines whether the location specified by the location information is in the vicinity of the location specified by the store's location information. If it is in the vicinity, proceed to S607; if it is not in the vicinity, proceed to S608. Note that whether two locations are in the vicinity is usually determined by whether the distance between the two locations is within a threshold. The distance may be a physical distance or a temporal distance (travel time).

[0141] (S607) The status transmission unit 141 transmits the status information accumulated in S306 to the i-th mobile terminal 2, paired with the store identifier.

[0142] (S608) The processing unit 13 increments the counter i by 1. The process returns to S603.

[0143] Next, an example of the transmission information acquisition process of S320 will be described with reference to the flowchart of FIG.

[0144] (S701) The transmission information acquisition unit 136 assigns 1 to a counter i.

[0145] (S702) The transmission information acquisition unit 136 determines whether or not time information identifying the i-th time period exists. If time information identifying the i-th time period exists, the process proceeds to S703; if not, the process returns to the upper level process.

[0146] (S703) The transmission information acquisition unit 136 uses the route information of the mobile terminal 2 and the current terminal location to acquire predicted location information, which is location information that identifies the location of the mobile terminal 2 during the time period indicated by the i-th time information.

[0147] (S704) The transmission information acquisition unit 136 assigns 1 to a counter j.

[0148] (S705) The transmission information acquisition unit 136 determines whether the j-th store information exists in the store management unit 111. If the j-th store information exists, the process proceeds to S706; if not, the process proceeds to S712.

[0149] (S706) The transmission information acquisition unit 136 acquires the store location included in the j-th store information.

[0150] (S707) The transmission information acquisition unit 136 determines whether the store location satisfies the location condition for the location information acquired in S703 (for example, whether the distance is within a threshold). If the location condition is satisfied, proceed to S708; if not, proceed to S711.

[0151] (S708) The transmission information acquisition unit 136 acquires, from the store management unit 111, the statistical results for the ith time slot corresponding to the jth store information.

[0152] (S709) The transmission information acquisition unit 136 determines whether the statistical results acquired in S708 satisfy the congestion condition (the condition for identifying congestion). If the congestion condition is satisfied, the process proceeds to S710; if not, the process proceeds to S711.

[0153] (S710) The transmission information acquisition unit 136 acquires the store identifier etc. contained in the j-th store information. The store identifier etc. may be all or part of the store information.

[0154] (S711) The transmission information acquisition unit 136 increments the counter j by 1. The process returns to S705.

[0155] (S712) The transmission information acquisition unit 136 increments the counter i by 1. The process returns to S702.

[0156] It should be noted that the determination process in S709 may be omitted in the flowchart of Fig. 7. That is, the transmission information acquisition unit 136 may acquire the store identifier etc. contained in the j-th store information even if the congestion condition is not satisfied.

[0157] Next, an example of the operation of the mobile terminal 2 will be described with reference to the flowchart of FIG.

[0158] (S801) The terminal reception unit 22 determines whether or not a destination has been received from the user. If a destination has been received, the process proceeds to S802, and if a destination has not been received, the process proceeds to S805.

[0159] (S802) The device processing unit 23 acquires the current location, searches for a route from the current location to the destination, acquires route information, and sets a route to the destination using the route information.

[0160] (S803) The terminal processing unit 23 composes a transmission instruction including the route information acquired in S802. The transmission instruction preferably includes the current terminal location.

[0161] (S804) The terminal transmitting unit 24 transmits the transmission instruction to the information processing device 1. The process returns to S801.

[0162] (S805) The terminal receiving unit 25 determines whether or not one or more pieces of store information have been received from the information processing device 1. If store information has been received, the process proceeds to S806, and if not, the process proceeds to S809.

[0163] (S806) The device processing unit 23 stores situation information and the like in the terminal storage unit 21 in association with the store identifier and time information contained in each of the one or more pieces of store information received in S805. This preferably updates the map database in the terminal storage unit 21. Furthermore, if the store identifier contained in the store information received in S805 does not exist in the map database, the device processing unit 23 adds new store information to the map database. Preferably, such store information includes situation information corresponding to one or more pieces of time information or statistical results corresponding to one or more pieces of time information. The situation information and the like may be situation information, statistical results, or a combination of situation information and statistical results.

[0164] (S807) The terminal determination unit 231 performs processing to output status information, etc. Return to S801. Details of this output processing will be described later.

[0165] (S808) The device processing unit 23 acquires an image. The device processing unit 23 may acquire the image by taking a photograph, or may acquire the image from a drive recorder (not shown).

[0166] (S809) The device processing unit 23 stores the acquired image in the device storage unit 21.

[0167] (S810) The terminal processing unit 23 determines whether or not to transmit an image. If an image is to be transmitted, the process proceeds to S811, and if not, the process proceeds to S813.

[0168] An image is sent when, for example, the similarity with the previously sent image is below a threshold, or when a time greater than or equal to the threshold (for example, 5 seconds or more) has passed since the previous image was sent.

[0169] (S811) The terminal transmitting unit 24 transmits the image. The process returns to S801.

[0170] (S812) The terminal processing unit 23 acquires the current terminal location.

[0171] (S813) The device processing unit 23 acquires a terminal identifier from the terminal storage unit 21. The terminal transmission unit 24 transmits the terminal position acquired in S812 to the information processing device 1 in association with the terminal identifier.

[0172] In the flowchart of FIG. 8, the process ends when the power is turned off or an interrupt occurs to end the process.

[0173] Next, an example of the output process in S807 will be described with reference to the flowchart in FIG.

[0174] (S901) The terminal processing unit 23 assigns 1 to a counter i.

[0175] (S902) The terminal processing unit 23 determines whether the i-th store information is present in the received store information. If the i-th store information is present, the process proceeds to S903; if not, the process returns to the upper processing.

[0176] (S903) The terminal determination unit 231 determines whether the i-th store information matches the output condition. If the output condition is met, the process proceeds to S904, and if not, the process proceeds to S905. The output condition is, for example, that "the store location included in the store information is a position within the area of ​​the map being output by the terminal output unit 26" or "the store type included in the store information is included in the user preference information in the terminal storage unit 21."

[0177] (S904) The terminal output unit 26 outputs the situation information, etc., contained in the ith store information. It is preferable that the terminal output unit 26 outputs the situation information, etc., contained in the ith store information on the map output by the terminal output unit 26. Note that the situation information, etc., may be situation information paired with one or more pieces of hourly information, may be statistical results paired with one or more pieces of hourly information, or may be situation information and statistical results paired with one or more pieces of hourly information.

[0178] (S905) The terminal processing unit 23 increments the counter i by 1. The process returns to S902.

[0179] A specific example of the operation of the information system A in this embodiment will be described below. Here, it is assumed that the mobile terminal 2 is a navigation terminal. The navigation terminal acquires images from a drive recorder installed in the same mobile object, for example, and transmits the images to the information processing device 1.

[0180] Currently, the storage unit 11 of the information processing device 1 stores a terminal management table shown in Fig. 10. The terminal management table is a table that manages information about one or more terminals. The terminal management table includes an "ID," a "terminal identifier," a "terminal location," and "route information."

[0181] When a route is set, each of two or more mobile terminals 2 transmits route information paired with a terminal identifier to the information processing device 1, and also sequentially transmits the latest terminal location paired with the terminal identifier to the information processing device 1. Then, the information processing device 1 receives the route information and the terminal location, and stores the route information and the terminal location paired with the terminal identifier in the terminal management table (FIG. 10). Note that the terminal information may not include route information.

[0182] The store management unit 111 of the information processing device 1 also stores a store management table shown in FIG. 11. The store management table is a table for managing information on one or more stores. The store management table includes an "ID," a "store identifier," a "store attribute value," and "status information." The "store attribute value" includes a "store name," a "store location," and a "store type." The "status information" includes a "time," a "business flag," a "number of people," and a "congestion level." The "business flag" can be either "1" for open or "0" for closed. The "number of people" is the number of people in line, or the total number of people inside the store and the number of people in line. The "congestion level" is information indicating the degree of congestion, and is a natural number between "1" and "5" in this case. The "congestion level" is information based on the number of people; the higher the number of people, the higher the congestion level.

[0183] Each of the two or more mobile terminals 2 transmits to the information processing device 1, while traveling, an image captured by a drive recorder, paired with a terminal identifier.

[0184] The image receiving unit 121 of the information processing device 1 receives the image. Furthermore, the situation acquisition unit 131 acquires the business flag and the number of people based on the image by the above-mentioned processing. Furthermore, the situation acquisition unit 131 acquires the congestion level using the number of people. Furthermore, the identifier acquisition unit 132 acquires the store identifier of the store in the image by the above-mentioned processing. Furthermore, the time acquisition unit 133 acquires the time by the above-mentioned processing. Then, in response to receiving the image, the situation accumulation unit 134 accumulates the situation information corresponding to the image (here, "time," "business flag," "number of people," and "congestion level") in the store management table.

[0185] The store management unit 111 also stores the statistical management table shown in FIG. 12. The statistical management table is a table that manages the statistical results of one or more stores. The statistical management table includes an "ID," a "store identifier," and "number of people per time period, and congestion level per time period." The "number of people per time period" is the average number of people in line per time period. The "congestion level per time period" is the average congestion level per time period.

[0186] The statistical processing unit 135 detects that it is time to perform statistical processing, performs statistical processing on the status information of each store in FIG. 11, and calculates the average number of people and the average congestion level for each time period.

[0187] In the above situation, it is assumed that while traveling along the set route, the mobile terminal "T01" acquires the image shown in FIG.

[0188] Next, the image receiving unit 121 of the information processing device 1 receives the image of Fig. 13 from the mobile terminal 2. Next, the situation acquisition unit 131 performs the situation acquisition process described using the flowchart of Fig. 4 on the image, and acquires the situation information "<time> T1 <business flag> 1 <number of people> 31 <congestion level> 5". Note that the situation acquisition unit 131 acquires the current time "T1", for example, from a clock (not shown).

[0189] Next, the identifier acquisition unit 132 attempts to acquire a store identifier from the store management table (FIG. 11) by the identifier acquisition process described using the flowchart in FIG. 5, but is unable to acquire the store identifier. In other words, it is assumed that this store is a new store that is not stored in the store management table. For example, it is assumed that the identifier acquisition unit 132 performs character recognition on the image in FIG. 13 and acquires the store name "RAMEN AB" from the image.

[0190] The processing unit 13 also generates a unique store identifier "S78". The processing unit 13 also generates location information "P S78 " is acquired as the store location. In addition, the processing unit 13 acquires the store type "ramen shop" from the image or the store name "RAMEN AB". Next, the processing unit 13 acquires "<Store Identifier>S78 <Store Name>RAMEN AB <Store Location>P S78 Add a record with "<Store Type> Ramen Shop" to the store management table (Figure 11).

[0191] In addition, the status accumulation unit 134 accumulates the status information acquired by the status acquisition unit 131, "<time> T1 <business flag> 1 <number of people> 31 <crowding level> 5", in the store management table (Figure 11), in pair with the store identifier "S78".

[0192] Next, the processing unit 13 determines that the acquired status information satisfies a transmission condition (for example, "the congestion level is equal to or greater than a threshold value"). Next, the status transmission unit 141 transmits the status information, the store name, and the store location to one or more mobile terminals 2 that are located on the route to which the store "RAMEN AB" is set, by the transmission process described using the flowchart in FIG.

[0193] Then, the one or more mobile terminals 2 receive the situation information and the like, and display "RAMEN AB" on the map on the screen of the mobile terminal 2. An example of such a display is shown in FIG.

[0194] In the above specific example, it is preferable that the information processing device 1 transmits the statistical results and latest status information of each store to the mobile terminal 2 that requires the information of the store. It is also preferable that the mobile terminal 2 receives and outputs the statistical results and latest status information.

[0195] Furthermore, the mobile terminal 2 that has received the store information for the store "RAMEN AB" stores the store information in the map database.

[0196] As described above, according to this embodiment, appropriate information on the business status of popular stores, newly opened stores, and the like can be provided to one or more mobile terminals.

[0197] Furthermore, according to this embodiment, it is possible to provide statistical results obtained by statistically processing store status information.

[0198] Furthermore, according to this embodiment, it is possible to provide statistical results obtained by statistically processing store status information for each time period.

[0199] Furthermore, according to this embodiment, it is possible to obtain status information or statistical results for stores that satisfy the stored store conditions.

[0200] Furthermore, according to this embodiment, it is possible to obtain status information or statistical results of stores on the route to the destination or in the vicinity of the route.

[0201] Each process in this embodiment is realized by software and a computer system 300 configured with the hardware shown in FIG. 15, for example. This software can be distributed by software download, etc. It can also be distributed by recording it on a recording medium such as a CD-ROM. The software realizing the information processing device 1 in this embodiment is the following program. That is, this program causes a computer to function as an image receiving unit that receives images taken while moving from two or more mobile terminals, a status acquiring unit that acquires status information specifying the business status of a store in the images, an identifier acquiring unit that acquires a store identifier that identifies the store corresponding to the status information, a time acquiring unit that acquires time information related to the time corresponding to the business status, a store management unit that stores status information for each of two or more stores, a status accumulating unit that associates the time information with the store identifier and accumulates the status information, and a status transmitting unit that transmits the status information associated with one or more pieces of time information and associated with the store identifier to one or more mobile terminals.

[0202] In FIG. 15, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0203] In FIG. 15, computer 301 includes, in addition to CD-ROM drive 3012, MPU 3013, bus 3014 connected to CD-ROM drive 3012 etc., ROM 3015 for storing programs such as boot-up programs, RAM 3016 connected to MPU 3013 for temporarily storing instructions for application programs and providing temporary storage space, and hard disk 3017 for storing application programs, system programs, and data.

[0204] The computer that executes the program may be a single computer or a plurality of computers, that is, it may perform centralized processing or distributed processing.

[0205] The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these modifications are also included within the scope of the present invention. [Industrial Applicability]

[0206] As described above, the information processing device 1 according to the present invention has the effect of being able to provide appropriate information about the business status of a store to one or more mobile terminals, and is useful as a server or the like that provides store information. [Explanation of symbols]

[0207] A. Information Systems 1. Information processing equipment 2. Mobile terminals 11 Storage area 12 Receiving section 13 Processing section 14 Transmitter 21 Terminal storage section 22 Terminal Reception 23 Terminal processing section 24 Terminal transmitter 25 Terminal receiving section 26 Terminal Output Unit 111 Store Management Department 121 Image receiving unit 131 Status Acquisition Unit 132 Identifier Acquisition Unit 133 Time acquisition part 134 Status Accumulation Unit 135 Statistical Processing Unit 136 Transmission information acquisition unit 141 Status Transmission Unit 142 Statistics Transmission Unit 231 Terminal Determination Unit

Claims

1. an image receiving unit that receives images taken while moving from each of the two or more mobile terminals; a status acquisition unit that acquires status information that identifies the business status of the store in the image; an identifier acquisition unit that acquires a store identifier that identifies a store corresponding to the situation information; a time acquisition unit that acquires time information relating to a time corresponding to the business status; a store management unit that stores status information of each of two or more stores, a status accumulation unit that accumulates the status information in association with the time information and the store identifier; an information processing device comprising: a status transmitting unit that transmits the status information associated with one or more pieces of time information and associated with a store identifier to one or more mobile terminals;

2. The situation acquisition unit Acquire situation information corresponding to two or more different pieces of time information for one store; a statistical processing unit that acquires, from the store management unit, situation information that corresponds to a store identifier that identifies the one store and that is paired with two or more pieces of time information, performs statistical processing on the two or more pieces of situation information, and acquires statistical results; The information processing device according to claim 1 , further comprising a statistics transmission unit that transmits the statistical results associated with the one store identifier.

3. The time information is information that identifies a time period, The statistical processing unit performing statistical processing on the situation information corresponding to each of the two or more time periods, and obtaining statistical results; The statistics transmission unit 3. The information processing device according to claim 2, wherein the statistical results for one or more time periods are transmitted.

4. a terminal storage unit for storing store conditions, which are conditions related to the store; a terminal receiving unit for receiving status information or statistical results associated with the store identifier; a terminal determination unit that determines whether the store identified by the store identifier satisfies the store condition; a terminal output unit that outputs the status information or the statistical results only when the terminal determination unit determines that the store conditions are met.

5. The terminal storage unit stores route information that identifies a route to a destination, the store condition is that the store is located on the route or in the vicinity of the route; The terminal determination unit The mobile terminal according to claim 4 , wherein a store location paired with the store identifier is acquired, and the store location is used to determine whether the store condition is satisfied.

Images