Information processing system and information processing method

Abstract

To reduce a load of information processing.SOLUTION: An information processing system includes a server device, and a plurality of terminal devices. The plurality of terminal devices include a first calculation part, a transmission part, and a setting part. The first calculation part executes first calculation processing including detection of a change in an image acquired by a photographing part. The transmission part executes transmission processing of transmitting transmission information indicating the change in the detected image to the server device. The setting part sets a cycle to execute at least one of the first calculation processing and the transmission processing, on the basis of detection frequency of the change of the image during a fixed period. The server device includes a reception part and a second calculation part. The reception part receives the transmission information from each of the plurality of terminal devices. The second calculation part executes second calculation processing by using the received transmission information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] FIELD An embodiment of the present invention relates to an information processing system and an information processing method. [Background technology]

[0002] A people flow analysis system has been proposed that has multiple edge terminals (terminal devices) capable of acquiring images and analyzes people flow over a wide area. In such a system, it is considered that the frequency (probability) of detection of the detection target is high in places with many people, and the frequency of detection of the detection target is low in places with few people. If such circumstances are not taken into consideration, the load of information processing such as calculation processing and communication processing may become excessively heavy. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2001-326903 A [Patent Document 2] JP 2007-272733 A [Patent Document 3] JP 2012-004720 A [Patent Document 4] JP 2015-002553 A [Patent Document 5] Patent No. 7074244 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide an information processing system and an information processing method that can reduce the load of information processing. [Means for solving the problem]

[0005] An information processing system according to an embodiment includes a server device and a plurality of terminal devices. The plurality of terminal devices include a first calculation unit, a transmission unit, and a setting unit. The first calculation unit executes a first calculation process including detection of a change in an image acquired by an imaging unit. The transmission unit executes a transmission process for transmitting transmission information indicating the detected change in the image to the server device. The setting unit sets a period for executing at least one of the first calculation process and the transmission process based on a detection frequency of the change in the image within a certain period. The server device includes a receiving unit and a second calculation unit. The receiving unit receives the transmission information from each of the plurality of terminal devices. The second calculation unit executes a second calculation process using the received transmission information. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram of a people flow analysis system according to an embodiment. [Diagram 2] Block diagram of an edge terminal. [Diagram 3] FIG. 2 is a block diagram of a server device. [Figure 4] 5 is a flowchart of a period setting process according to the embodiment. [Diagram 5] FIG. 4 is a sequence diagram of an analysis process according to the embodiment. [Figure 6] FIG. 11 is a diagram illustrating an example of an operation in the first modified example. [Figure 7] FIG. 11 is a diagram illustrating an example of an operation in Modification 2. [Figure 8] FIG. 13 is a diagram illustrating an example of an operation in Modification 3. [Figure 9] FIG. 13 is a diagram illustrating an example of an operation in Modification 4. [Figure 10] FIG. 13 is a diagram illustrating an example of the operation in Modification 5. [Figure 11] FIG. 2 is a hardware configuration diagram of the apparatus according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of an information processing system according to the present invention will be described in detail with reference to the accompanying drawings.

[0008] In the following, as an example of an information processing system, a system including a plurality of edge terminals (an example of terminal devices) and a server device, in which a people flow analysis system that analyzes people flow using images acquired (captured) by the plurality of edge terminals is described. Applicable systems are not limited to people flow analysis systems, and may be any other system. For example, the detection target is not limited to people flow, and may be other objects such as vehicles, or a specific person. That is, the following embodiment may be applied to a system that tracks objects.

[0009] As described above, in the people flow analysis system, the frequency at which a detection target (person) is detected (hereinafter, detection frequency) may vary, and if such variation is not taken into consideration, the load of information processing may become excessively heavy. The detection frequency is, for example, the number of times a detection target is detected within a certain period of time, or a value obtained by dividing this number by the total number of detection processes within the certain period of time (detection probability). The information processing includes, for example, calculation processing executed by each edge terminal, communication processing between each edge terminal and a server device (including information transmission processing), and calculation processing executed by the server device.

[0010] For example, assume that an edge terminal with a low detection frequency (including when the detection target is not detected) and an edge terminal with a high detection frequency are configured to perform arithmetic processing on the obtained images in the same manner. In such a configuration, as an extreme example, arithmetic processing is performed to detect people flow on an image captured in an area where no people are present, and a transmission process is further performed to transmit the result of the arithmetic processing (also called the arithmetic result) to a server device. Such a situation is wasteful from the viewpoint of both arithmetic processing and communication, and as a consequence of such waste, the load on information processing (arithmetic processing, communication processing) becomes excessively heavy. Furthermore, as a result of the increased load, an increase in power consumption and an increase in heat generation in the entire system may occur.

[0011] As a technology for reducing the load on a moving object tracking system using captured images, a technology (comparative example) has been proposed that checks the expected arrival time of the tracked object for each camera position, and returns an image of the object from the camera where it is expected to arrive, thereby limiting the cameras to be tracked, and reducing the processing load. With such a technology, in order to prevent overlooking an object, it is necessary to start transmission at an early time with a considerable margin in mind, and as a result, the reduction in the information processing load is only slight.

[0012] In the information processing system of the embodiment, the operation cycle of the information processing, which is at least one of the calculation processing and the transmission processing, is set based on the detection frequency of image changes in a certain period of time. This reduces the load of at least one of the calculation processing and the transmission processing (communication processing), and as a result, the power consumption or heat generation of the system can be reduced.

[0013] Fig. 1 is a block diagram showing an example of a configuration of a people flow analysis system 10 according to an embodiment. As shown in Fig. 1, the people flow analysis system 10 includes a server device 100, a plurality of edge terminals 200a and 200b, and a control device 300, which are connected via a network 400.

[0014] The edge terminals 200a and 200b are devices that perform calculation processing to obtain information for analyzing people flow, and processing to transmit information including the calculation results to the server device 100. Since the edge terminals 200a and 200b have similar functions, they are simply referred to as edge terminals 200 when there is no need to distinguish between them. The number of edge terminals 200 is not limited to two, and may be three or more.

[0015] The server device 100 is a device that executes a calculation process for analyzing people flow using information transmitted from a plurality of edge terminals 200.

[0016] The control device 300 is a device capable of communicating with the server device 100 and for controlling the operation of the server device 100 in response to instructions from a user (such as an administrator). The control device 300 may be connected to the server device 100 by a connection means other than the network 400 (such as another network or a communication line).

[0017] The control device 300 can be configured, for example, as a personal computer and a server device different from the server device 100. The control device 300 may transmit an instruction to the server device 100 in response to an operation by a user (such as an administrator), or may transmit an instruction to the server device 100 by a control program. The control device 300 may transmit an instruction to the server device 100 to transmit a result of arithmetic processing by the server device 100. The server device 100 transmits the result of arithmetic processing to the control device 300 in response to this instruction or independently of the instruction (for example, periodically).

[0018] The network 400 may be of any type, and may be, for example, the Internet, a local area network (LAN), etc. The network 400 may be any of a wired network, a wireless network, and a combination of a wired network and a wireless network.

[0019] Fig. 2 is a block diagram showing an example of a functional configuration of the edge terminal 200. As shown in Fig. 2, the edge terminal 200 includes a calculation unit 201 (first calculation unit), a transmission unit 202, a reception unit 203, a setting unit 204, a control unit 205, a storage unit 221, and an imaging unit 222.

[0020] The storage unit 221 stores various information used by the edge terminal 200. For example, the storage unit 221 stores information indicating the operation cycle (hereinafter simply referred to as cycle) of the arithmetic processing by the arithmetic unit 201, and the result of the arithmetic processing by the arithmetic unit 201. The storage unit 221 can be configured with any commonly used storage medium, such as a flash memory, a memory card, a RAM (Random Access Memory), a HDD (Hard Disk Drive), and an optical disk.

[0021] The imaging unit 222 is an element that captures (obtains) an image, and can be configured with imaging elements such as a CCD (Charge Coupled Device) and a CIS (CMOS image sensor). The edge terminal 200 may not include the imaging unit 222 inside, and may be configured to obtain an image captured by the imaging unit 222 from, for example, an imaging unit 222 provided outside the edge terminal 200.

[0022] In this embodiment, the imaging unit 222 of each of the multiple edge terminals 200 makes it possible to analyze the flow of people over a wide area.

[0023] The calculation unit 201 executes a calculation process OP_A (first calculation process) on an image captured by the imaging unit 222. The calculation process OP_A may be any process, for example, a process including detection of a change in the captured image. The calculation unit 201 may execute the calculation process OP_A using information received from the server device 100 or another edge terminal 200.

[0024] The arithmetic processing OP_A is executed periodically (intermittently) according to, for example, a set cycle. For example, the arithmetic unit 201 executes the arithmetic processing OP_A to detect a change in the image, that is, the movement of the object, by calculating the difference between an image obtained by the imaging unit 222 at a specific time TA determined by the set cycle and an image at another time TB. The other time TB is, for example, a time immediately before or after the time TA.

[0025] The calculation unit 201 may execute the calculation process OP_A in accordance with an instruction from the server device 100 received by the receiving unit 203, separately from the periodic execution.

[0026] As described above, the information indicating the cycle of the arithmetic processing OP_A is stored, for example, in the storage unit 221. The arithmetic unit 201 refers to the information stored in the storage unit 221 as necessary and determines the cycle at which the arithmetic processing OP_A is operated.

[0027] The calculation result by the calculation unit 201 may be immediately transmitted to the server device 100 or to another edge terminal 200, or may be stored in the storage unit 221 without being immediately transmitted.

[0028] The transmission unit 202 transmits various information to external devices such as the server device 100 and other edge terminals 200. For example, the transmission unit 202 executes a transmission process of transmitting, to the server device 100, transmission information indicating a change in the image detected by the calculation unit 201.

[0029] The transmitting unit 202 may transmit the transmission information in accordance with a set cycle, similar to the arithmetic unit 201. The cycle may be the same as the cycle set in the arithmetic unit 201. That is, a common cycle may be set in the arithmetic unit 201 and the transmitting unit 202. For example, when the result of the arithmetic process OP_A is immediately transmitted to the server device 100, a common cycle may be set. The cycle of the transmission process may be a value different from the cycle set in the arithmetic unit 201. For example, when transmitting information including one or more arithmetic results stored in the storage unit 221, the cycle of the transmission process may be set to be larger than the cycle of the arithmetic unit 201 (arithmetic process OP_A).

[0030] The transmitting unit 202 may transmit the transmission information under the control of the control unit 205 in response to an instruction from the server device 100. The transmitting unit 202 may transmit, to another edge terminal 200, information indicating the period set in the own device.

[0031] The receiving unit 203 receives various information from external devices such as the server device 100 and other edge terminals 200 via the network 400. For example, the receiving unit 203 receives the calculation results of the other edge terminals 200 and the cycles of the calculation processes OP_A performed by the other edge terminals 200. The receiving unit 203 also receives instructions from the server device 100.

[0032] The setting unit 204 sets the cycles of the calculation process OP_A and the transmission process. For example, the setting unit 204 sets the cycle of at least one of the calculation process OP_A and the transmission process based on the detection frequency of image changes in a certain period. In the following, a case where a common cycle is set for the transmission process and the calculation process OP_A will be described as an example. This cycle may also be referred to as the cycle of information processing.

[0033] The setting unit 204 may set the information processing period of its own device using at least one of the information processing period of the other edge terminals 200 and an instruction from the server device 100.

[0034] An example of a method for setting a period using the detection frequency will be described below. First, the setting unit 204 calculates a detection frequency that indicates the frequency at which a change in an image is detected. The detection frequency is, for example, the number of times a detection target (change in an image) is detected within a certain period of time, or a value (detection probability) obtained by dividing this number by the total number of detection processes within the certain period of time. The certain period of time is, for example, a period going back a certain amount of time from the current time.

[0035] The setting unit 204 calculates the information processing period according to the calculated detection frequency. For example, in an edge terminal 200 with a high detection frequency, it is desirable to execute the calculation process OP_A or the transmission process more frequently in order to prevent missed detection of the target movement. Therefore, the setting unit 204 sets the period to be shorter as the detection frequency is higher. In other words, the setting unit 204 sets the period so that the value becomes smaller as the detection frequency is higher.

[0036] On the other hand, in the edge terminal 200 where the detection frequency is low, it is not necessary to set the information processing cycle short. In fact, if an excessively short cycle is set, the information processing load may become excessively heavy. Therefore, the setting unit 204 sets the cycle longer as the detection frequency decreases. By such processing, it is possible to reduce the information processing load while avoiding missed detection.

[0037] For example, the setting unit 204 compares the detection frequency with a threshold value TH_A (first threshold value), and if the detection frequency is greater than the threshold value TH_A, sets the period to a value VA (first value), and if the detection frequency is equal to or less than the threshold value TH_A, sets the period to a value VB (second value) greater than the value VA.

[0038] The values ​​VA and VB depend on the size of the imaging range of the imaging unit 222, but for example, the value VA set for an edge terminal 200 with a high detection frequency is about several tens of milliseconds, and the value VB set for an edge terminal 200 with a low detection frequency is about several hundreds of milliseconds to several seconds. Note that these values ​​are merely examples and are not limited to these.

[0039] Also, the method of setting the period according to the detection frequency is not limited to the above, and any other method may be used. For example, the setting unit 204 may set the period to one of (n+1) values ​​according to the result of comparing the detection frequency with n (n is an integer of 2 or more) threshold values ​​TH_A. Also, the setting unit 204 may calculate the period according to the detection frequency using a calculation formula for calculating the period from the detection frequency, or correspondence information (such as a table) that associates the detection frequency with the period.

[0040] The setting unit 204 sets the calculated period as the period of information processing. As a result, for example, when the detection frequency is low (for example, when there is no person at all), the period can be set to a large value, and as a result, the load of information processing can be reduced. Note that even when only the period of the arithmetic processing OP_A or the period of the transmission processing is set, rather than the periods of both, it is possible to reduce the load of at least the processing for which the period is set.

[0041] The control unit 205 controls the overall various processes performed by the edge terminal 200. For example, the control unit 205 controls the calculation unit 201 to execute a calculation process OP_A at a set cycle. The control unit 205 may control the various processes according to an instruction from the server device 100. For example, when the server device 100 instructs the control unit 205 to transmit a calculation result to another edge terminal 200, the control unit 205 controls the transmission unit 202 to transmit the calculation result performed by the calculation unit 201 of the own device to the instructed other edge terminal 200.

[0042] The imaging unit 222 may be capable of performing operations such as panning, tilting, and zooming. When such an imaging unit 222 is used, the control unit 205 may control the panning, tilting, and zooming of the imaging unit 222 so that a desired imaging area can be imaged.

[0043] Each of the above units (the calculation unit 201, the transmission unit 202, the reception unit 203, the setting unit 204, and the control unit 205) is realized, for example, by one or more processors. For example, each of the above units may be realized by having a processor such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit) execute a program, that is, by software. Each of the above units may be realized by a processor such as a dedicated IC (Integrated Circuit), that is, by hardware. Each of the above units may be realized by a combination of software and hardware. When multiple processors are used, each of the above units may be realized by any of the multiple processors.

[0044] Fig. 3 is a block diagram showing an example of a functional configuration of the server device 100. As shown in Fig. 3, the server device 100 includes a calculation unit 101 (a second calculation unit), a transmission unit 102, a reception unit 103, a control unit 104, and a storage unit 121.

[0045] The storage unit 121 stores various information used by the server device 100. For example, the storage unit 121 stores information indicating the time when the calculation unit 101 executes calculation processing, and transmission information received from the edge terminal 200. The transmission information may be added with information identifying the edge terminal 200 that is the sender. The storage unit 121 can be configured with any commonly used storage medium, such as a flash memory, a memory card, a RAM, a HDD, or an optical disk.

[0046] The receiving unit 103 receives various information from external devices such as the edge terminal 200 and the control device 300. For example, the receiving unit 103 receives transmission information transmitted from the edge terminal 200 via the network 400. The receiving unit 103 may receive information indicating the period of the arithmetic processing OP_A or the like by the edge terminal 200 from the edge terminal 200. Note that the information received by the receiving unit 103 is not limited to the above.

[0047] The calculation unit 101 executes a calculation process OP_B (second calculation process) using the received transmission information. The calculation process OP_B may be any process, for example, a process for determining the time transition of people flow, and a process for systematically organizing the dependency of people flow on time or place.

[0048] The calculation unit 101 may execute the arithmetic process OP_B periodically, or may execute the arithmetic process OP_B based on an instruction received from the control device 300. The calculation unit 101 may execute the arithmetic process OP_B by referring to information stored in the storage unit 121 that indicates the time when the arithmetic process is to be executed.

[0049] The calculation result by the calculation unit 101 may be immediately transmitted to the edge terminal 200 or the control device 300, or may be stored in the storage unit 121 without being immediately transmitted.

[0050] The transmission unit 102 transmits various information to external devices such as the edge terminal 200 and the control device 300. For example, the transmission unit 102 transmits the result of the arithmetic processing OP_B, information stored in the storage unit 121, and the like to the edge terminal 200 or the control device 300.

[0051] The transmitting unit 102 may transmit information such as an instruction received from the control device 300 to the edge terminal 200, or may transmit information received from one edge terminal 200 to another edge terminal 200.

[0052] The control unit 104 controls the overall various processes performed by the server device 100. For example, the control unit 104 controls the calculation unit 101 to execute a calculation process OP_B at a set time. The control unit 104 may control the various processes according to instructions from the control device 300. The control unit 104 may control the transmission unit 102 to transmit an instruction to the edge terminal 200 to transmit a result of the calculation process.

[0053] Each of the above units (the calculation unit 101, the transmission unit 102, the reception unit 103, and the control unit 104) is realized, for example, by one or more processors. For example, each of the above units may be realized by having a processor such as a CPU or GPU execute a program, that is, by software. Each of the above units may be realized by a processor such as a dedicated IC, that is, by hardware. Each of the above units may be realized by using a combination of software and hardware. When multiple processors are used, each of the above units may be realized by any of the multiple processors.

[0054] The server device 100 may be physically configured as one device, or may be physically configured as multiple devices. For example, the server device 100 may be built on a cloud environment.

[0055] Next, a description will be given of a period setting process by the people flow analysis system 10 according to the embodiment. Fig. 4 is a flowchart showing an example of the period setting process according to the embodiment.

[0056] The cycle setting process is a process for setting a cycle of information processing, and is executed by the setting unit 204 of each edge terminal 200. As described above, the calculation unit 201 detects a change in the image (movement of the target) in the imaging range of the imaging unit 222 by calculating the difference between the image at time TA and the image at time TB obtained by the imaging unit 222 for each set cycle. The cycle setting process is executed using the detection results (accumulated empirical values) obtained by the calculation unit 201 during a certain period of time.

[0057] That is, the setting unit 204 calculates a detection frequency at which a change in an image is detected within a certain period of time (step S101). The setting unit 204 calculates a period according to the calculated detection frequency (step S102). The setting unit 204 sets the calculated period as a period of information processing (at least one of the arithmetic processing OP_A and the transmission processing) (step S103).

[0058] The cycle setting process may be executed at any timing, for example, when instructed by a user (such as an administrator) or periodically.

[0059] For example, the detection frequency may change over time in each edge terminal 200. If the cycle setting process is configured to be executed periodically, it becomes possible to update the cycle to be set in response to the changed detection frequency.

[0060] Next, a description will be given of an analysis process performed by the people flow analysis system 10 according to the embodiment. Fig. 5 is a sequence diagram showing an example of the analysis process according to the embodiment.

[0061] The imaging unit 222 of the edge terminal 200 captures an image of the imaging range (step S201). The calculation unit 201 executes a calculation process OP_A including a process of detecting a change in the image using the captured image (step S202). The calculation process OP_A is executed at a period set by the period setting process. When a change in the image is detected, the transmission unit 202 transmits transmission information indicating the change in the image to the server device 100 (step S203).

[0062] The edge terminal 200 repeatedly executes the processes of steps S201 to S203. Although only one edge terminal 200 is shown in Fig. 5, the processes shown in Fig. 5 are executed in each of the multiple edge terminals 200 included in the people flow analysis system 10.

[0063] The receiving unit 103 of the server device 100 receives the transmission information transmitted from the edge terminal 200 (step S204). The calculation unit 101 executes a calculation process OP_B using the received transmission information (step S205). The server device 100 repeatedly executes the processes of steps S204 to S205.

[0064] By the above-mentioned process, for example, the load of the information processing (the calculation process OP_A, the transmission process) is prevented from becoming excessively heavy due to the cycle being set too short, and the detection failure of the target movement due to the cycle being set too long is also prevented. Furthermore, by executing the cycle setting process at an appropriate timing, the cycle can be appropriately updated even if the detection frequency fluctuates.

[0065] (Variation 1) In the above embodiment, the information processing cycle of each edge terminal 200 is determined based on the detection frequency of the detection target (change in image). This reduces the load of information processing. However, the movement of the target, i.e., change in the image, may occur even in the imaging range of the imaging unit 222 of the edge terminal 200 with a low detection frequency. Therefore, if the information processing cycle is set long in the edge terminal 200 with a low detection frequency, the movement of the target may not be detected. If a detection miss occurs, a situation may arise in which information required for people flow analysis, for example, cannot be obtained.

[0066] Therefore, in the first modification, when the period is set to a long value (for example, value VB) and a change in the image is detected, the setting unit 204 sets the period to a short value (for example, value VA). This suppresses missed detection of the movement of the object in the imaging range of the imaging unit 222.

[0067] In the edge terminal 200 with a low detection frequency, even if a change in the image, i.e., the movement of the object in the imaging range of the imaging unit 222, is detected, it is possible that the movement of the object, i.e., the change in the image, will not be detected after a short time has passed. Therefore, in this modified example, if a state in which no change in the image is detected continues for a specific time, the information processing cycle is reset to a long value again. That is, the setting unit 204 sets the cycle to a short value (e.g., value VA) and if a state in which no change in the image is detected continues for a specific time, it sets the information processing cycle to a long value again (e.g., value VB). This makes it possible to further reduce the load of information processing.

[0068] The specific time period depends on the size of the imaging range of the imaging unit 222, and can be, for example, several seconds to several tens of seconds. Note that these values ​​are merely examples and are not limited to these.

[0069] Fig. 6 is a diagram showing a schematic example of an operation in this modified example. Here, an example of the operation of one edge terminal 200a in states 601 to 603 corresponding to three times is described. Note that Fig. 6 shows only the imaging unit 222 of the edge terminal 200a.

[0070] First, assume that the edge terminal 200a, which has a low detection frequency, detects a change in the image, that is, the movement of an object (person P01), as shown in state 601. In addition, in Fig. 6, the person P01 is shown moving in the right direction as indicated by the arrow. In this case, the setting unit 204 of the edge terminal 200a resets the period to a shorter period.

[0071] As a result, it becomes possible to capture the movement of the target (person P01) in detail in state 602. Then, as shown in state 603, when the target (person P01) goes out of the imaging range of the imaging unit 222 of the edge terminal 200a and the state where the movement of the target is not detected continues for a specific time, the setting unit 204 of the edge terminal 200a resets the period to be longer again.

[0072] (Variation 2) In the first modification, an example has been described in which the information processing cycle of the edge terminal 200 in which a change in an image has been detected is reset. In the second modification, the information processing cycles of the other edge terminals 200 are also reset.

[0073] When a change in an image is detected in any edge terminal 200, this means that the movement of an object has been detected in the imaging range of the imaging unit 222 of that edge terminal 200. For this reason, it is expected that a change in an image will also be detected in the imaging range of the imaging unit 222 of another edge terminal 200 close to that edge terminal 200 after a short time has passed.

[0074] Therefore, in this modification, when a change in an image is detected in any edge terminal 200, the information processing cycle of the edge terminals 200 adjacent to that edge terminal 200 is set to be short.

[0075] For example, when a change in an image is detected, the transmitting unit 202 of the edge terminal 200 (first terminal device) has a long value (e.g., value VB) set for the period and transmits a request to the edge terminal 200 (second terminal device) whose distance from the edge terminal 200 is equal to or less than a threshold (distance threshold) to set the period to a short value (e.g., value VA) among the other edge terminals 200. The setting unit 204 of the edge terminal 200 that receives the request sets the information processing period of the edge terminal 200 to a short value in accordance with the request.

[0076] This makes it possible to suppress missed detection of the movement of a target in the imaging range of the imaging unit 222 of the edge terminal 200 adjacent to the edge terminal 200 where a change in the image has been detected. This is an advantage that cannot be obtained when there is only one edge terminal 200.

[0077] An example of a method for determining a nearby edge terminal 200 will be described below. For example, assume a situation in which the imaging range of the imaging unit 222 is a long and narrow area such as a road or a passageway, and the imaging ranges of the imaging units 222 of a plurality of edge terminals 200 are arranged in a line. In such a situation, the nearby edge terminal 200 can be determined as the edge terminal 200 adjacent to the edge terminal 200 in which a change in the image has been detected.

[0078] Also, assume a situation in which the imaging range of the imaging unit 222 is an area other than a long and narrow area such as a square or a room, and the imaging ranges of the imaging units 222 of the edge terminals 200 are scattered. In such a situation, the nearby edge terminal 200 can be determined as the edge terminal 200 having the imaging range of the imaging unit 222 within a specific distance from the edge terminal 200 in which a change in the image has been detected.

[0079] If information indicating the direction of the object's movement can be obtained from the image, among the multiple edge terminals 200, an edge terminal 200 that is installed in the same direction as the direction of the object's movement may be determined to be the adjacent edge terminal 200.

[0080] The specific distance depends on the size of the imaging range of the imaging unit 222, and can be, for example, several meters to several tens of meters. Note that these values ​​are merely examples and are not limited to these.

[0081] Fig. 7 is a diagram showing a schematic example of an operation in this modified example. Here, an example of the operation of five edge terminals 200a to 200e arranged in a row will be described. Note that Fig. 7 shows only the imaging unit 222 of each edge terminal 200. The number of edge terminals 200 is not limited to five.

[0082] Assume that the edge terminal 200c in the center detects a change in the image, that is, the movement of the target (person P01). In this case, the periods of the three edge terminals 200b, 200c, and 200d included in the setting target 701 are re-set to shorter periods. That is, the transmission unit 202 of the edge terminal 200c transmits a request to the edge terminals 200b and 200d adjacent to the edge terminal 200c to set the periods to shorter values.

[0083] The edge terminals 200 included in the setting target 701 are not limited to the three shown in Fig. 7. For example, either of the edge terminals 200a and 200e at both ends in Fig. 7 may be included in the targets for which the cycle is set to be short.

[0084] (Variation 3) In variant example 2, the information processing cycle of an edge terminal 200 close to an edge terminal 200 in which a change in an image has been detected, i.e., an edge terminal 200 in which a change in the image obtained by the imaging unit 222 is expected to be detected after a short period of time, is set to be short.

[0085] For example, if an edge terminal 200 that detects a change in an image immediately transmits a request to a nearby edge terminal 200 to set the period to a short value, the period of the nearby edge terminal 200 may be set too quickly. In other words, there is a possibility that information processing may be performed unnecessarily.

[0086] Therefore, in the third modification, when a change in an image is detected in any of the edge terminals 200, the time when the object will arrive in the imaging range of the imaging unit 222 of the other edge terminal 200 is predicted in consideration of the content of the change. Then, the information processing period of the other edge terminal 200 is set according to the predicted time. The content of the change is, for example, at least one of the direction of the object's movement and the speed of the object's movement.

[0087] For example, in this modification, the setting unit 204 predicts the time when a change in an image is detected in a nearby edge terminal 200 (e.g., an edge terminal 200 whose distance from the own device is equal to or less than a threshold value). Furthermore, the transmission unit 202 transmits a request to set the period to a short value (e.g., value VA) at the predicted time.

[0088] This prevents the cycle from being set too short, i.e., prevents the information processing load from becoming excessive. This is an advantage that cannot be obtained when there is only one edge terminal 200.

[0089] Any method may be used to predict the arrival time, but for example, a method that considers at least one of the changes in the direction of the object's movement detected along with the change in the image and the speed of the object's movement may be applied. For example, the setting unit 204 uses the direction of the object's movement to determine an edge terminal 200 that is predicted to be reached by the object among the nearby edge terminals 200. The setting unit 204 also uses the speed of the object's movement and the distance to the determined edge terminal 200 to calculate the time it takes for the object to reach the determined edge terminal 200. The setting unit 204 adds the calculated time to the current time to calculate a predicted value for the time the object will arrive.

[0090] Since prediction may include errors, the setting unit 204 may output a time slightly earlier than the predicted time as the final predicted value, taking into account the error (margin). In this case, it is not possible to completely prevent the cycle from being set too early, but it is possible to suppress it. In other words, it is not possible to reduce the load of information processing to a minimum, but it is possible to reduce it to a value close to that minimum.

[0091] In the technology of the comparative example, for example, a camera where the arrival of a tracked object is predicted is switched from a stopped state to an operating state. Therefore, it is necessary to increase the time margin required to start operating. In contrast, in this modified example, each edge terminal 200 is not stopped, but operates at least for a long period. Therefore, the time margin for switching the period can be set to a smaller value compared to the technology of the comparative example.

[0092] Here, the time to be advanced in consideration of the error (margin) depends on the imaging range of the imaging unit 222 or the accuracy of the prediction of the arrival time, and can be, for example, several seconds to tens of seconds. Note that these values ​​are examples and are not limited to these.

[0093] Fig. 8 is a diagram showing a schematic example of an operation in this modified example. Here, an example of the operation of three edge terminals 200a to 200c arranged in a row will be described. Note that Fig. 8 shows only the imaging unit 222 of each edge terminal 200. The number of edge terminals 200 is not limited to three.

[0094] Assume that the edge terminal 200b installed in the center detects a change in the image, i.e., the movement of the target (person P01), and that the target is moving in the right direction in Fig. 8. In this situation, it is expected that the target will enter the imaging range of the imaging unit 222 of the edge terminal 200c after a short time has passed. Therefore, the setting unit 204 of the edge terminal 200b predicts that time, and the transmission unit 202 transmits a request to the edge terminal 200c to set the cycle to a shorter time at the predicted time.

[0095] (Variation 4) In the above embodiment and each modified example, a situation may occur in which edge terminals 200 with a long cycle and edge terminals 200 with a short cycle are mixed. Furthermore, as a special situation, a situation may occur in which the information processing cycle is set to be long for all of the edge terminals 200 having the imaging range of the imaging unit 222 in a predetermined specific area (hereinafter, area RA).

[0096] In such a special situation, the detection of the movement of the object in the area RA may be delayed, resulting in the object movement being missed. Therefore, it is desirable to prevent the occurrence of such a special situation.

[0097] Here, as described above, the multiple edge terminals 200 can be configured to mutually transmit and receive information indicating the information processing period. That is, the edge terminals 200 can grasp the periods of the other edge terminals 200. Therefore, when the special situation as described above occurs, any of the edge terminals 200 in the area RA can reset the information processing period of its own device to a shorter period.

[0098] For example, among multiple edge terminals 200 installed in a predetermined area RA, the setting unit 204 of a specific edge terminal 200-S (first terminal device) sets the period of the own device to a short value (e.g., value VA) when the period of the own device is set to a long value (e.g., value VB) and when the period of one or more edge terminals 200 other than the edge terminal 200-S installed in the area RA is set to a long value (e.g., value VB).

[0099] The edge terminal 200-S may be specified in any manner, but is, for example, predetermined as a device for which the period is set to be shorter than that of the other edge terminals 200 among the edge terminals 200 installed in the area RA.

[0100] By performing such processing, it is possible to avoid the occurrence of the special situation described above. Therefore, it is possible to avoid or suppress the failure to detect the movement of the object in the area RA. This is an advantage that cannot be obtained when there is only one edge terminal 200.

[0101] The special situation described above occurs when no movement of an object is detected in the area RA for a certain period of time, i.e., when no flow of people is detected. The edge terminal 200-S for which a short period is set is preferably an edge terminal 200 having a location within the imaging range of the imaging unit 222 where the movement of an object is expected to be detected sooner thereafter. By determining the edge terminal 200-S in this way, the flow of people in the area RA can be detected sooner.

[0102] The location where the object movement is expected to be detected earlier may be defined in any manner, but may be defined, for example, as follows. If the area RA is a road, the location corresponds to a branch point (a location with a branch road) If the area RA is a passageway, the locations corresponding to both ends of the passageway, the locations corresponding to the entrances and exits of rooms facing the passageway, the locations corresponding to other structures (stairs, elevators, escalators, etc.) leading to the passageway, or the locations corresponding to branching points. If the area RA is, for example, a room, the location of the entrance / exit

[0103] Fig. 9 is a diagram showing a schematic example of an operation in this modified example. Here, the area RA is an aisle 911, and an example of the operation of five edge terminals 200a to 200e installed in a row in the aisle 911 will be described. Note that Fig. 9 shows only the imaging unit 222 of each edge terminal 200. The number of edge terminals 200 is not limited to five.

[0104] 9, the period is set to a long value (for example, value VB).

[0105] The imaging region of the imaging unit 222 of the edge terminal 200b is close to the branch point 912. In such a case, the period of the edge terminal 200b is re-set to be shorter. That is, since the edge terminal 200b is close to the branch point 912 where the movement of the target is expected to be detected earlier, the edge terminal 200b is determined as the edge terminal 200-S. Then, the setting unit 204 of the edge terminal 200b sets the period of the edge terminal 200b to a short value (for example, the value VA) because the period of the edge terminal 200b is set to a long value (for example, the value VB) and is set to a long value (for example, the value VB) relative to the other edge terminals 200a, 200c to 200e installed in the area RA.

[0106] The number of edge terminals 200-S whose periods are to be shortened is not limited to one, and may be two or more. In the example of Fig. 9, the periods of one or both of the edge terminals 200a and 200c adjacent to the edge terminal 200b may be shortened together with the period of the edge terminal 200b.

[0107] (Variation 5) Depending on the area where the flow of people is detected, for example, the detection frequency may be extremely low during a specific time period on a specific day of the week. The extremely low detection frequency means, for example, that the detection frequency is lower than a threshold value TH_B (second threshold value). The specific time period on a specific day of the week corresponds to a period in which the detection frequency is predicted to be lower than the threshold value TH_B.

[0108] Examples of specific time periods on specific days of the week include early morning on holidays, daytime on weekdays, and nighttime on any day of the week, etc. These are just examples, and the days of the week or time periods with extremely low detection frequency are not limited to these.

[0109] In this modified example, when the detection frequency is assumed to be extremely low, information processing, including periodic operation, is stopped for any of the edge terminals 200. In other words, the number of edge terminals 200 that are in operation is thinned out.

[0110] For example, at least some of the control units 205 of the edge terminals 200 stop the arithmetic processing OP_A and the transmission processing during a period in which the detection frequency is predicted to be lower than the threshold value TH_B.

[0111] This makes it possible to further reduce the load of information processing compared to a configuration in which information processing is performed at long intervals without stopping, which is an advantage that cannot be obtained when there is only one edge terminal 200.

[0112] However, as described in variant example 4, if the operation of the edge terminal 200 is stopped, the detection of people flow within the area RA may be delayed if the operation of the edge terminal 200 includes a location within the imaging range of the imaging unit 222 where the movement of objects within the area RA is expected to be detected earlier.

[0113] Therefore, the edge terminal 200 to be stopped is preferably an edge terminal 200 other than the edge terminal 200 having a place in the imaging range of the imaging unit 222 where the movement of the target is expected to be detected earlier in the region RA. In other words, the edge terminal 200 to be stopped is preferably an edge terminal 200 having a place in the imaging range of the imaging unit 222 where the movement of the target is expected to be detected relatively slowly in the region RA.

[0114] For example, the control unit 205 of the edge terminal 200 that has been predetermined as a target for suspension suspends the arithmetic processing OP_A and the transmission processing during a period in which the detection frequency is predicted to be lower than the threshold value TH_B.

[0115] This allows the load of information processing to be further reduced compared to a configuration in which information processing is performed over a long period without stopping, and the flow of people is detected without delay, which is an advantage that cannot be obtained when there is only one edge terminal 200.

[0116] Fig. 10 is a diagram showing a schematic example of an operation in this modified example. Here, as in Fig. 9, the area RA is the passage 911, and an example of the operation of five edge terminals 200a to 200e installed in a row in the passage 911 will be described. Note that Fig. 10 shows only the imaging unit 222 of each edge terminal 200. The number of edge terminals 200 is not limited to five.

[0117] 10 is a day of the week or a time period when the frequency of detecting changes in the image, that is, the movement of the object, is extremely low. The imaging range of the imaging unit 222 of the edge terminal 200b includes the branch point 912. The imaging ranges of the imaging units 222 of the edge terminals 200a and 200c on both sides of the edge terminal 200b are close to the branch point 912.

[0118] Therefore, the control units 205 of these edge terminals 200a, 200b, and 200c do not stop the information processing operations. On the other hand, the control units 205 of the two edge terminals 200 included in the stop target 1001, that is, the two edge terminals 200d and 200e whose imaging ranges of the imaging units 222 are far from the branch point 912, stop the information processing operations.

[0119] 10, an example has been described in which the stop target 1001 includes the edge terminals 200d and 200e, but the stop target is not limited to this. For example, the stop target 1001 may include the edge terminals 200a, 200c, 200d, and 200e other than the edge terminal 200b having the imaging area of ​​the imaging unit 222 in front of the branch point 912. The stop target 1001 may include only the edge terminal 200e that is the farthest from the branch point 912 in FIG. 10.

[0120] In the above embodiment and each of the modified examples, mutual transmission and reception of information indicating the information processing cycle between the multiple edge terminals 200 can be performed by communication via the server device 100, or can be performed by direct communication not via the server device 100. In a configuration in which direct communication is performed between the multiple edge terminals 200 not via the server device 100, the communication load of the people flow analysis system 10 as a whole can be further reduced.

[0121] As described above, according to the embodiment and the modified example, it is possible to reduce the load of information processing.

[0122] Next, the hardware configuration of the devices (server device, edge terminal, control device) of the embodiment will be described with reference to Fig. 11. Fig. 11 is an explanatory diagram showing an example of the hardware configuration of the devices of the embodiment.

[0123] The device of the embodiment includes a control device such as a CPU 51, storage devices such as a ROM (Read Only Memory) 52 and a RAM 53, a communication I / F 54 that connects to a network and performs communication, and a bus 61 that connects each unit.

[0124] The programs executed by the device of the embodiment are provided in a state that they are pre-installed in the ROM 52 or the like.

[0125] The programs executed by the devices of the embodiments may be configured to be provided as a computer program product by being recorded in an installable or executable format on a computer-readable recording medium such as a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk Recordable), or a DVD (Digital Versatile Disk).

[0126] Furthermore, the program executed by the device of the embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the program executed by the device of the embodiment may be provided or distributed via a network such as the Internet.

[0127] The programs executed by the apparatus of the embodiment can cause a computer to function as each unit of the apparatus described above. In this computer, the CPU 51 can read the programs from a computer-readable storage medium onto a main storage device and execute the programs.

[0128] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0129] 10 People Flow Analysis System 100 Server device 200a, 200b, 200c, 200d, 200e Edge Terminals 300 Control device 400 Network 101, 201 Arithmetic section 102, 202 Transmitter 103, 203 Receiver 104, 205 Control section 121, 221 Storage section 204 Setting section 222 Imaging unit

Claims

1. An information processing system including a server device and a plurality of terminal devices, Each of the plurality of terminal devices a first calculation unit that executes a first calculation process including detection of a change in an image acquired by the imaging unit; a transmission unit that executes a transmission process to transmit transmission information indicating the detected change in the image to the server device; a setting unit that sets a cycle for executing at least one of the first calculation process and the transmission process based on a detection frequency of a change in the image within a certain period of time; The server device includes: a receiving unit that receives the transmission information from each of the plurality of terminal devices; A second calculation unit that executes a second calculation process using the received transmission information. Information processing system.

2. the setting unit sets the period so that the period has a smaller value as the detection frequency increases. The information processing system according to claim 1 .

3. The setting unit is If the detection frequency is greater than a first threshold, the period is set to a first value, and if the detection frequency is equal to or less than the first threshold, the period is set to a second value greater than the first value; when the period is set to the second value and the change is detected, setting the period to the first value; The information processing system according to claim 1 .

4. The transmission unit of a first terminal device included in the plurality of terminal devices, when the period is set to the second value and the change is detected, a request to set the period to the first value is transmitted to a second terminal device, the second terminal device being located at a distance from the first terminal device equal to or less than a threshold, among a plurality of terminal devices other than the first terminal device included in the plurality of terminal devices; The information processing system according to claim 3 .

5. The setting unit predicts a time when a change in the image will be detected by the second terminal device; the transmitter transmits a request to set the period to the first value at the predicted time.

5. The information processing system according to claim 4.

6. The setting unit is if the period is set to the first value and no change is detected for a specified time, then setting the period to the second value. The information processing system according to claim 3 .

7. The setting unit of a first terminal device included in the plurality of terminal devices installed in a predetermined specific area, When the period is set to the second value and the period is set to the second value for one or more of the terminal devices other than the first terminal device installed in the specific area, the period of the first terminal device is set to the first value. The information processing system according to claim 3 .

8. The first terminal device is predetermined as a device that sets the period to the first value in priority to one or more of the terminal devices other than the first terminal device. The information processing system according to claim 7.

9. One or more of the terminal devices other than the first terminal device installed in the specific area, a control unit that stops the first calculation process and the transmission process during a period in which the detection frequency is predicted to be lower than a second threshold value. The information processing system according to claim 7.

10. Some of the plurality of terminal devices a control unit that stops the first calculation process and the transmission process during a period in which the detection frequency is predicted to be lower than a second threshold value. The information processing system according to claim 1 .

11. The transmission unit is transmitting information indicating the set cycle to the other terminal devices; The information processing system according to claim 1 .

12. An information processing method executed in an information processing system including a server device and a plurality of terminal devices, A first calculation step in which the plurality of terminal devices execute a first calculation process including detection of a change in an image acquired by an imaging unit; a transmission step in which the terminal devices execute a transmission process of transmitting transmission information indicating the detected change in the image to the server device; a setting step of setting a cycle in which the plurality of terminal devices execute at least one of the first arithmetic process and the transmission process based on a detection frequency of a change in the image within a certain period of time; a receiving step in which the server device receives the transmission information from each of the plurality of terminal devices; a second calculation step in which the server device executes a second calculation process using the received transmission information; An information processing method comprising:

Images