Information Processing System

Abstract

To efficiently execute distributed processing by a plurality of apparatuses.SOLUTION: An information processing system includes a plurality of information processing apparatus. Each of the plurality of information processing apparatuses includes a reception unit, a detection unit, an execution unit, a determination unit, and a transmission unit. The reception unit receives an input of processing data. The detection unit detects whether a type of the processing data is a target type subjected to first processing or a non-target type not subjected to the first processing. The execution unit executes the first processing on the processing data when the type of the processing data is the target type. The determination unit determines another information processing apparatus that processes data of the non-target type to be a transmission destination apparatus. The transmission unit transmits at least part of the processing data to the transmission destination apparatus.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] An embodiment of the present invention relates to an information processing system, an information processing method, an information processing device, and a program. [Background technology]

[0002] A system in which multiple IoT (Internet of Things) devices (edge ​​devices) perform distributed processing of various data is known. For example, a technology has been proposed that determines the processing to be executed by a robot, which is a type of IoT device, and the processing to be executed by a monitoring server, based on acquired image data and at least one of tracking operating environment information, communication quality, and load information. With this configuration, distributed processing can be executed even between devices with different processing capabilities.

[0003] In addition, a technology has been proposed that implements processing on lightweight virtual machines and distributes the virtual machines to each IoT device to improve overall processing scalability. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2017 / 038100 [Non-patent literature]

[0005] [Non-Patent Document 1] R. Morabito, V. Cozzolino, AY Ding, N. Beijar and J. Ott, "Consolidate IoT Edge Computing with Lightweight Virtualization," in IEEE Network, vol. 32, no. 1, pp. 102-111, Jan.-Feb. 2018, doi: 10.1109 / MNET.2018.1700175. Summary of the Invention [Problem to be solved by the invention]

[0006] An object of the present invention is to provide an information processing system, an information processing method, an information processing device, and a program that are capable of more efficiently executing distributed processing by a plurality of devices. [Means for solving the problem]

[0007] An information processing system according to an embodiment includes a plurality of information processing devices. The plurality of information processing devices include a receiving unit, a detecting unit, an executing unit, a determining unit, and a transmitting unit. The receiving unit receives input of processing data. The detecting unit detects whether the type of the processing data is a target type that is a target type of the first processing or a non-target type that is not a target type of the first processing. The executing unit executes the first processing on the processing data when it is detected that the type of the processing data is the target type. The determining unit determines another information processing device that processes data of the non-target type as a destination device. The transmitting unit transmits at least a portion of the processing data to the destination device. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram of an information processing system according to an embodiment of the present invention. [Diagram 2] Block diagram of an IoT device. [Diagram 3] Block diagram of the server. [Figure 4] FIG. 4 is a diagram showing an example of a data structure of correspondence information. [Diagram 5] 4 is a flowchart of information processing according to the first embodiment. [Figure 6] 11 is a flowchart of information processing by a destination device. [Figure 7] FIG. 1 is a diagram showing an example of the configuration of an information processing system. [Figure 8] A diagram showing an example of distributed processing using two IoT devices. [Figure 9] A diagram showing an example of distributed processing using three IoT devices. [Figure 10] FIG. 13 is a block diagram of an IoT device according to a second embodiment. [Figure 11] 10 is a flowchart of information processing according to the second embodiment. [Figure 12] FIG. 2 is a hardware configuration diagram of an apparatus according to the first or second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of an information processing apparatus according to the present invention will be described in detail with reference to the accompanying drawings.

[0010] As described above, the following systems exist as systems that realize distributed processing using a plurality of devices. (S1) A system that distributes processing between IoT devices (robots) and servers (monitoring servers, etc.) (S2) System using virtual machine

[0011] In a system of (S1), the server may share the processing. For this reason, it is necessary to prepare a server with a large processing capacity and higher performance. However, preparing a high-performance device may lead to an increase in costs. In addition, the need to transmit data to the server arises, and network traffic increases. Since servers are not usually configured to specialize for a specific purpose, the power consumption when each process is executed on the server tends to be higher than the power consumption when the process is executed on a device specialized for a specific purpose. Therefore, in a system of (S1) in which processing is distributed to servers, the power consumption of the entire system may increase.

[0012] Furthermore, when there is data that is undesirable to be transmitted over a public network due to privacy and security requirements (for example, data with a low level of abstraction, such as image data captured by a camera), it may not be possible to adopt a configuration using a server as in (S1).

[0013] In addition, in the configuration of (S1), if an IoT device does not have the capability to process a new function that needs to be implemented, the server needs to have that capability, which further increases the load on the server.

[0014] Another measure that takes into account the need to implement new functions is to design devices in advance to realize advanced functions, assuming the addition of new functions. For example, one possible method is to provide a margin of computing power in computing devices such as a central processing unit (CPU) and a graphics processing unit (GPU). However, this method requires the preparation of a higher performance device, and problems such as the device size increasing in order to provide scalability for device performance may arise.

[0015] In the (S2) system, each piece of hardware must have the computing power to run a virtual machine. Also, each piece of hardware must be designed to have an OS (operating system) environment in which a virtual machine can run, or a virtual machine execution environment.

[0016] However, for example, an IoT device highly specialized for a specific purpose, particularly an IoT device implemented in a lightweight manner, may not have the ability to execute a virtual machine and may not be applicable to a system such as (S2). Implementing an IoT device so that it can execute a virtual machine may result in a complex configuration in order to improve the computing performance and support the execution of an OS.

[0017] In addition, a configuration using a virtual machine as in (S2) may not be able to effectively utilize the capabilities of an IoT device that is lightweight and specialized for a specific purpose. For example, even if an IoT device with specific hardware for AI (Artificial Intelligence) processing such as person authentication is realized (commercialized, etc.), the device is configured to run a virtual machine without adopting this IoT device. Since a device specialized for a specific purpose that can reduce power consumption is not used, efficiency cannot be achieved in terms of power consumption.

[0018] Furthermore, in the system of (S2), processing is distributed on a virtual machine basis. Therefore, in order to have other devices execute processing, it is necessary to transmit data including the data to be processed (hereinafter referred to as processing data) and the virtual machine image, which increases the communication load. There is also a method of storing the virtual machine image in the device in advance, but in that case, a storage device (data storage) for storing the virtual machine image is required, which complicates the configuration of the IoT device.

[0019] Therefore, in the information processing system according to the embodiment, when a process (non-target process) other than a predetermined specific process (first process) targeted by an IoT device becomes necessary, another IoT device specialized in the non-target process is determined, and data is transmitted (transferred) to the device (destination device) to have the device perform the non-target process instead. For example, each IoT device is constructed as a device specialized in executing a specific process defined for each device. The non-target process is a process other than the specific process, such as a process that the IoT device cannot handle and a process that cannot be efficiently executed by the IoT device. At least some of the multiple IoT devices 100 are defined as specific processes that are different from each other.

[0020] (First embodiment) In the information processing system according to the first embodiment, a server selects another IoT device that is specialized in a specific process that is not a target process for a certain IoT device.

[0021] Fig. 1 is a block diagram showing an example of the configuration of an information processing system 10 according to the present embodiment. As shown in Fig. 1, the information processing system 10 has a configuration in which a plurality of IoT devices 100a, 100b, and 100c (examples of information processing devices) and a server 200 are connected via a network 300.

[0022] Since the IoT devices 100a, 100b, and 100c have the same configuration, when there is no need to distinguish between them, they are simply referred to as the IoT device 100. The number of IoT devices 100 is not limited to three, and may be two or four or more.

[0023] The server 200 may be configured as one physical device, or may be constructed in a cloud environment.

[0024] The network 300 is, for example, a network such as a LAN (Local Area Network) or the Internet, etc. The network 300 may be either a wired network or a wireless network.

[0025] Fig. 2 is a block diagram showing an example of the configuration of the IoT device 100. As shown in Fig. 2, the IoT device 100 includes a reception unit 101, a detection unit 102, a determination unit 103, a type notification unit 104, a reception unit 105, a selection unit 106, an execution unit 107, a transmission unit 108, and a storage unit 121.

[0026] The IoT device 100 is also connected to a sensor 150. The IoT device 100 and the sensor 150 may be connected in any form, for example, a connection by a dedicated line or a connection via a network 300. The sensor 150 corresponds to a device (input device) that acquires processing data processed by the IoT device 100. A plurality of types of sensors 150 may be connected to the IoT device 100.

[0027] The sensor 150 may be any type of sensor, such as a camera (imaging device), a temperature sensor, a humidity sensor, a vibration sensor, and an acoustic sensor. The sensor 150 may be provided in the IoT device 100. In the following, an example in which the sensor 150 is a camera that captures image data (video stream, still image, etc.) is mainly described.

[0028] The receiving unit 101 receives input of processing data from the sensor 150. For example, if the sensor 150 is a camera, the receiving unit 101 receives image data captured by the camera as the processing data.

[0029] The detection unit 102 detects the type of processing data. For example, the detection unit 102 detects whether the type of the accepted processing data is a type that is a target of a specific process defined for the own device (hereinafter, a target type) or a type that is not a target of the specific process (hereinafter, a non-target type).

[0030] If the type of the processing data is a target type, the detection unit 102 inputs the processing data to the selection unit 106. If the type of the processing data is a non-target type, the detection unit 102 inputs the detection result to the determination unit 103, and also inputs the processing data or a part of the processing data to the transmission unit 108 in preparation for transmission to another IoT device 100.

[0031] The detection unit 102 may execute a recognition process for recognizing a data type (class) of the processing data, and detect the type of the processing data based on the result of the recognition process. For example, the detection unit 102 executes an object recognition process for recognizing (detecting) vehicles, people, transport robots, and other objects of unknown types from image data captured by a camera.

[0032] As an example, a case will be described where the identification process is a vehicle identification process. The vehicle identification process is, for example, a process of recognizing a license plate number by image recognition of image data, and determining whether or not the vehicle is permitted to pass (permitted vehicle). If a vehicle is recognized as a result of the recognition process, the detection unit 102 determines that the type of the processed data is a target type. If an object other than a vehicle (a person, a transport robot, or other unknown object type) is recognized as a result of the recognition process, the detection unit 102 determines that the type of the processed data is a non-target type.

[0033] When a vehicle is recognized, the detection unit 102 transmits the processing data (image data) or a part of the data (partial data) to the selection unit 106. The partial data is, for example, data corresponding to a part of the processing data that is used in processing by the execution unit 107. For example, data of an area of ​​the image data that includes the recognized object (for example, a vehicle) may correspond to the partial data.

[0034] When an object other than a vehicle is recognized, the detection unit 102, for example, transmits type information indicating the recognized type to the determination unit 103, and inputs the processed data (image data) or partial data to the transmission unit 108. After the determination unit 103 determines another IoT device 100 as a destination, the processed data or partial data is transmitted to the determined IoT device 100.

[0035] When it is detected that the type of the processing data is an untarget type, the determining unit 103 determines another IoT device 100 that processes data of the untarget type as a destination device to transmit the processing data. For example, the determining unit 103 determines the destination device by using the type notifying unit 104 to inquire of the server 200 about the IoT device 100 corresponding to the untarget type.

[0036] When it is detected that the type of the processed data is an untargeted type, the type notification unit 104 notifies the server 200 of notification information including the detected untargeted type. The notification information may further include identification information for identifying the own device.

[0037] As described later, the server 200 selects an IoT device 100 that processes processing data of a non-target type included in the notification information, and returns device information indicating the selected device. The device information is received, for example, by the type notification unit 104 or the receiving unit 105, and passed to the determining unit 103. The determining unit 103 determines the IoT device 100 indicated by the received device information as the destination device.

[0038] The receiving unit 105 receives data from external devices such as other IoT devices 100 and the server 200. For example, the receiving unit 105 receives processing data (transfer data) transmitted from another IoT device 100 and identification information of the transmission source IoT device 100. The receiving unit 105 transmits the received processing data and identification information to the selecting unit 106.

[0039] The selection unit 106 selects data to be processed by the execution unit 107. For example, the selection unit 106 selects data to be processed by the execution unit 107 from the processing data (or partial data) input from the detection unit 102 and the transfer data input from the reception unit 105. The selection may be based on any criteria, but for example, a method of selection based on the priority of data or the like may be applied. The priority may indicate, for example, that the data input from the detection unit 102 is given priority over the data received by the reception unit 105, or vice versa.

[0040] The execution unit 107 executes a specific process determined for the device itself. For example, when the detection unit 102 detects that the type of processing data is a target type and the selection unit 106 selects the processing data, the execution unit 107 executes the specific process for the selected processing data. The execution unit 107 transmits information such as a processing result to the server 200 as necessary. For example, when the execution unit 107 executes a vehicle identification process as the specific process and determines that the vehicle whose license plate number has been recognized is not a permitted vehicle, the execution unit 107 notifies the server 200 that a non-permitted vehicle has been detected.

[0041] The transmitting unit 108 transmits the processed data (or partial data) to the IoT device 100 determined as the destination device by the determining unit 103. For example, the transmitting unit 108 stores the data input from the detecting unit 102 as data to be transmitted to another IoT device 100 in the storage unit 121 or the like. When the destination device is determined by the server 200, the transmitting unit 108 transmits the stored data to the destination device.

[0042] The processing unit 104 may be configured to execute further processing (additional processing) on ​​the processing data in response to the processing result by the executing unit 107 or result data which is the processing result on the processing data by the destination device.

[0043] For example, the detection unit 102 further detects whether the type of the result data is an additional processing type indicating a type to be subjected to additional processing. When it is detected that the type of the result data is the additional processing type, the determination unit 103 further determines another IoT device 100 that processes the result data as a destination device. For example, the determination unit 103 determines the destination device by using the type notification unit 104 to inquire of the server 200 about the IoT device 100 corresponding to the specific type.

[0044] For example, it is assumed that the identification process of the IoT device 100b is a person authentication process, and the identification process of the IoT device 100c is a behavior estimation process. It is assumed that the execution unit 107 of the IoT device 100b executes the person authentication process and detects that the person is an unregistered person (such as a suspicious person). For example, the execution unit 107 passes the result data to the detection unit 102. The detection unit 102 detects that the result data indicates a suspicious person type defined as, for example, an additional process type. The determination unit 103 determines the destination device by inquiring of the server 200 about the IoT device 100 corresponding to the suspicious person type using the type notification unit 104. The server 200 determines, as the destination device, the IoT device 100 that executes the behavior estimation process for the data indicating the suspicious person type.

[0045] If the IoT device 100c is determined as the destination device for executing the behavior estimation process, the transmitter 108 of the IoT device 100b transmits the processing data to the IoT device 100c. With such a function, the information processing system 10 can be configured to estimate the possibility of risky behavior of a suspicious individual.

[0046] Each of the above units (reception unit 101, detection unit 102, determination unit 103, type notification unit 104, reception unit 105, selection unit 106, execution unit 107, and transmission unit 108) is realized, for example, by one or more processors. For example, each of the above units may be realized by having a processor such as a CPU execute a program, i.e., by software. Each of the above units may be realized by a processor such as a dedicated IC (Integrated Circuit), i.e., by hardware. Each of the above units may be realized by using a combination of software and hardware. When multiple processors are used, each processor may realize one of the units, or two or more of the units.

[0047] The storage unit 121 stores various data used in various processes of the IoT device 100. For example, the storage unit 121 stores the process data accepted by the acceptance unit 101 and data (process data, partial data) to be transmitted to a transmission destination device.

[0048] The storage unit 121 can be configured from any commonly used storage medium, such as a flash memory, a memory card, a RAM (Random Access Memory), a HDD (Hard Disk Drive), or an optical disk.

[0049] Fig. 3 is a block diagram showing an example of the configuration of the server 200. As shown in Fig. 3, the server 200 includes a receiving unit 201, a selecting unit 202, a device notifying unit 203, and a storage unit 221.

[0050] The storage unit 221 stores various data used in various processes of the server 200. For example, the storage unit 221 stores notification information received from the IoT device 100 and corresponding information for determining a destination device. The storage unit 221 can be configured with any commonly used storage medium such as a flash memory, a memory card, a RAM, a HDD, and an optical disk.

[0051] Fig. 4 is a diagram showing an example of a data structure of the correspondence information. As shown in Fig. 4, the correspondence information is information in a list format in which a type of data is associated with identification information of an IoT device 100 capable of processing the data of the type. A plurality of pieces of identification information may be associated with one type. The correspondence information in Fig. 4 is an example, and may further include other information, or may have a data structure other than the list format.

[0052] The receiving unit 201 receives data from an external device such as the IoT device 100. For example, the receiving unit 201 receives notification information from the IoT device 100. The receiving unit 201 passes the received notification information to the selecting unit 202.

[0053] The selection unit 202 selects, as a destination device, an IoT device 100 corresponding to the type included in the notification information input from the reception unit 201. For example, the selection unit 202 refers to the correspondence information and selects, as a destination device, one of the IoT devices 100 associated with the type included in the notification information. The selection unit 202 transmits, to the device notification unit 203, identification information (destination identification information) of the selected IoT device 100 in addition to identification information (source identification information) of the source IoT device 100.

[0054] When multiple identification information are associated with one type in the correspondence information, the selection unit 202 may select one of the IoT devices 100 (identification information) by taking into consideration at least a part of the processing load, operating status, and network latency (network delay) of each IoT device 100.

[0055] The device notification unit 203 notifies the IoT device 100 that is the sender of the notification information of device information indicating the selected IoT device 100. For example, the device notification unit 203 transmits device information indicating destination identification information to the IoT device 100 corresponding to the sender identification information sent from the selection unit 202.

[0056] Next, a description will be given of information processing by the IoT device 100 according to the first embodiment. Fig. 5 is a flowchart showing an example of information processing in the first embodiment.

[0057] The receiving unit 101 receives input of processing data from the sensor 150 (step S101). The detecting unit 102 detects the type of the processing data (step S102). The detecting unit 102 determines whether the type is a target type targeted by the IoT device 100 (step S103).

[0058] If the type of the processing data is the target type (step S103: Yes), the detection unit 102 passes the processing data to the selection unit 106. Here, it is assumed that the selection unit 106 selects the input processing data (step S104). Note that a process in which the selection unit 106 selects processing data received from another IoT device 100, in other words, a process in which the IoT device 100 operates as a transmission destination device, will be described with reference to FIG. 6.

[0059] The execution unit 107 executes the process for the selected processing data (step S105). The execution unit 107 executes post-processing as necessary (step S106). For example, the execution unit 107 executes post-processing such as notifying the server 200 of the processing result.

[0060] If it is determined in step S103 that the type of the processing data is not a target type (step S103: No), the determination unit 103 executes a process of determining a destination device. For example, the determination unit 103 notifies the server 200 of notification information including the non-target type using the type notification unit 104 (step S107). The type notification unit 104 (or the receiving unit 105) receives the device information transmitted from the server 200 in response to the notification information (step S108). The transmitting unit 108 transmits the processing data to the IoT device 100 corresponding to the destination device indicated by the received device information (step S109).

[0061] FIG. 6 is a flowchart showing an example of information processing by the IoT device 100 when operating as a destination device.

[0062] The receiving unit 105 of the IoT device 100 as a destination device receives data (transfer data) transmitted from another IoT device 100 (step S201). The selecting unit 106 selects the received data (step S202). The executing unit 107 executes processing on the selected processing data (transfer data) (step S203). The executing unit 107 executes post-processing as necessary (step S204).

[0063] Next, a specific example of the information processing system 10 of the present embodiment will be described. Fig. 7 is a diagram showing an example of the configuration of the information processing system 10. As shown in Fig. 7, three IoT devices 100a, 100b, and 100c are configured to execute the following specific processes. IoT device 100a: Vehicle identification processing IoT Device 100b: Person Authentication Processing IoT device 100c: Behavior estimation processing

[0064] For example, the IoT device 100a is a device equipped with a camera specialized for vehicle identification, which is expected to be installed at the vehicle gate of a facility. The IoT device 100a recognizes the license plate number of a vehicle passing through the vehicle gate and compares it with a registered license plate number. If the vehicle is registered, the IoT device 100a notifies the server 200 of information identifying the vehicle (vehicle ID, license plate number, etc.) and the passing time. If an unregistered vehicle is recognized, the IoT device 100a notifies the server 200 of image data and information identifying the vehicle.

[0065] For example, the IoT device 100b is a device equipped with a camera specialized for person authentication, which is expected to be installed at a gate through which people pass, such as an employee service gate. The IoT device 100b performs face detection and face recognition of the person passing through the service gate, and authenticates whether the person is registered as an employee. If the person is registered, the IoT device 100b notifies the server 200 of information identifying the person (such as an employee number) and the time of passage. If an unregistered person is detected, the IoT device 100b closes the gate and notifies the server 200 of the detection result.

[0066] For example, the IoT device 100c is a device equipped with a camera (surveillance camera) that is expected to generate an alarm for suspicious behavior or actions in a warehouse or the like. The IoT device 100c is configured to be able to estimate the behavior of a person captured by the camera. For example, the IoT device 100c is configured to be able to execute advanced AI processing that estimates the likelihood of various behaviors, such as normal walking, walking while checking the surroundings, and behavior such as searching for something.

[0067] Each of the IoT devices 100a, 100b, and 100c is assumed to have an efficient configuration specialized for a specific process determined for each device. For example, the execution unit 107 of the IoT device 100a is configured to have a minimum computing performance for executing the vehicle identification process. Alternatively, the execution unit 107 is configured with a hardware circuit specialized for the vehicle identification process, and is configured to be able to execute the vehicle identification process with low power consumption or at high speed while being unable to execute processes other than the vehicle identification process. That is, each IoT device 100 is configured to have more efficient processing performance and power consumption. Note that each specific process may be realized by any method, but for example, a method using a machine learning model such as a neural network can be applied.

[0068] According to this embodiment, in an environment in which the IoT devices 100 as shown in Fig. 7 are used, distributed processing by a plurality of IoT devices 100 can be executed more efficiently. A specific example of distributed processing will be described with reference to Figs. 8 and 9.

[0069] Fig. 8 is a diagram showing an example of distributed processing by the IoT device 100a and the IoT device 100b. Fig. 8 shows an example of a case where a person who does not usually pass through the vehicle gate where the IoT device 100a is installed is detected.

[0070] The detection unit 102 of the IoT device 100a detects that the processing data is of a non-target type (person) (step S301). The type notification unit 104 of the IoT device 100a notifies the server 200 that processing data of which the type is person has been detected (step S302).

[0071] The selection unit 202 of the server 200 refers to the correspondence information and selects the IoT device 100b as a device (destination device) that performs person authentication processing for the type "person" (step S303). The device notification unit 203 of the server 200 returns device information including identification information of the selected IoT device 100b to the IoT device 100a (step S304).

[0072] The transmitting unit 108 of the IoT device 100a transmits processed data (such as a video stream) about the detected person to the IoT device 100b identified by the identification information included in the device information (step S305).

[0073] The IoT device 100b executes a person authentication process on the received processing data (transfer data) and transmits the processing result to the server 200 (step S306). For example, if the result of the person authentication process is that the person is authenticated as a person registered as an employee, the IoT device 100b notifies the server 200 of the passing time together with the identification information of the IoT device 100a. This makes it possible to record that the authenticated person passed through the vehicle gate where the IoT device 100a is installed and where people normally do not pass, for some reason. As a result, the administrator can take action in accordance with the management policy of the facility, such as checking the reason for passing through.

[0074] If the result of the person authentication process indicates that the person is an unregistered person (such as a person not registered as an employee), the IoT device 100b notifies the server 200 of the authentication result together with the identification information of the IoT device 100a. In this case, too, the administrator can take action in accordance with the management policy of the facility in response to the notification.

[0075] Fig. 9 is a diagram showing an example of distributed processing by the IoT device 100a, the IoT device 100b, and the IoT device 100c. Fig. 9 shows an example of a case where an unregistered person is detected as a result of person authentication processing similar to that in Fig. 8. In this example, the IoT device 100c is configured to estimate the possibility of risky behavior of the unregistered person.

[0076] Steps S401 to S405 are the same as steps S301 to S305 in FIG.

[0077] When an unregistered person is detected, the type notification unit 104 of the IoT device 100b notifies the server 200 that processing data corresponding to the suspicious person type has been detected (step S406).

[0078] The selection unit 202 of the server 200, which has received the notification information indicating the suspicious person type from the IoT device 100b, refers to the correspondence information and selects the IoT device 100c as a device (destination device) that will execute the behavior estimation process for the processing data of the suspicious person type (step S407). The device notification unit 203 of the server 200 returns device information including the identification information of the selected IoT device 100c to the IoT device 100b (step S408).

[0079] The transmitting unit 108 of the IoT device 100b transmits the processed data (such as a video stream) determined to be of the suspicious person type and identification information of the IoT device 100a, which is the device that acquired the processed data, to the IoT device 100c, which is identified by the identification information included in the device information (step S409).

[0080] The IoT device 100c executes a behavior estimation process on the received processing data (transfer data) and transmits the processing result to the server 200 (step S410). For example, the IoT device 100c notifies the server 200 of the possibility of risky behavior, which is the detection result, together with information indicating the type of suspicious person. This enables the administrator to take action in accordance with the management policy of the facility, such as rushing security personnel to the site.

[0081] (Variation 1) As described above, the sensor 150 may be a sensor other than a camera. That is, the information processing system 10 is not limited to a system that distributes processing of image data. For example, each IoT device 100 may include some or all of a temperature sensor, a humidity sensor, a vibration sensor, and an acoustic sensor.

[0082] For example, the IoT device 100a includes at least a temperature sensor and a vibration sensor, and the specific processing is an abnormality detection process using data detected by the temperature sensor (an example of processed data). The IoT device 100b includes at least a vibration sensor and an acoustic sensor, and the specific processing is an abnormality detection process using data detected by the vibration sensor and data detected by the acoustic sensor (acoustic data).

[0083] For example, when the IoT device 100a detects abnormal vibrations from data detected by a vibration sensor, it queries the server 200 about other IoT devices 100 configured to execute the abnormality detection process as a specific process in order to execute more detailed abnormality detection process based on the detected data. When the server 200 selects the IoT device 100b, the IoT device 100a transmits the detected data to the selected IoT device 100b to cause the selected IoT device 100b to execute the abnormality detection process.

[0084] When multiple sensors 150 are provided as in this modified example, the receiving unit 101 receives input of processing data from each of the multiple sensors 150. The detecting unit 102 may detect whether the type of each of the multiple processing data received from each of the multiple sensors 150 is a target type or a non-target type. The transmitting unit 108 may transmit only a part of the multiple processing data input from the multiple sensors 150 (for example, processing data used in a specific process of the destination device) to the destination device.

[0085] (Variation 2) According to the above embodiment, the processing data itself is not transmitted to the server 200, and only the notification information including the information indicating the type (non-target type) is transmitted to the server 200. Therefore, even in a system configuration in which, for example, each IoT device 100 is provided in a private network (for example, a LAN) and the server 200 is provided in an external network of the private network (for example, a public network such as the Internet), it is possible to configure so that the data that should be kept secret is not transmitted to the external network. In other words, it is possible to satisfy the requirements from the viewpoints of security and privacy protection, such as not wanting the data acquired by the sensor 150 to leak to the external network.

[0086] The server 200 may be realized as a server (eg, a gateway server or a local server) provided in the same network as the IoT device 100, rather than in an external network.

[0087] In addition, the IoT device 100 may transmit the processed data acquired by the sensor 150 or a portion of the processed data (partial data) to the server 200 as additional information for the server 200 to select a destination device, and the server 200 may use this data to determine the destination device.

[0088] For example, it is assumed that the detection unit 102 of the IoT device 100 can detect whether the type of the processing data is a target type (e.g., a vehicle) or something else (e.g., not a vehicle), but cannot detect details of the non-target type (e.g., a person or a transport robot). In this case, the type notification unit 104 notifies the server 200 of the processing data together with information indicating that the type is a non-target type. The server 200 uses the processing data to detect details of the type of the processing data, and selects a destination device corresponding to the detected type.

[0089] In this example, not only the notification information but also the processing data is transmitted to the server 200. However, if the server 200 is configured as a local server as described above, it is possible to meet the requirements for security and privacy protection.

[0090] (Variation 3) In the above embodiment, the destination IoT device 100 transmits the processing result of the specific processing performed by the destination IoT device 100 to the server 200. The processing result may be transmitted to the source IoT device 100, and the source IoT device 100 may perform necessary post-processing, such as notifying the server 200 of the processing result.

[0091] As described above, in the first embodiment, when a non-target process other than the specific process executed by an IoT device becomes necessary, another IoT device specialized in the non-target process is determined, and data is sent to the IoT device to perform the non-target process instead. This makes it possible to execute distributed processing by multiple devices more efficiently.

[0092] For example, more advanced processing can be distributed and executed without increasing the power consumption and device size of each IoT device. Since it can be configured not to send processing data to the server, it is possible to reduce power consumption, reduce network traffic, and simplify the server configuration. In addition, it is possible to reduce security and privacy risks by eliminating the need to send low-level abstract data to the server.

[0093] Second embodiment In the first embodiment, the server selected another IoT device (destination device) that executes the non-target process. In the information processing system according to the second embodiment, the source IoT device has a function of selecting a destination device. Note that, although an example in which a server is provided will be described below, the information processing system according to the present embodiment does not necessarily have to have a server.

[0094] The overall configuration of the information processing system of this embodiment is similar to that of the information processing system 10 of the first embodiment shown in Fig. 1, and therefore the description will be omitted. Note that it is not necessary for all IoT devices in the information processing system 10 to have the functions of the second embodiment described below, and for example, some IoT devices may have the functions of the first embodiment.

[0095] Fig. 10 is a block diagram showing an example of the configuration of an IoT device 100-2 according to the second embodiment. As shown in Fig. 10, the IoT device 100-2 includes a reception unit 101, a detection unit 102, a determination unit 103-2, a reception unit 105, a selection unit 106, an execution unit 107, a transmission unit 108, and a storage unit 121-2.

[0096] The second embodiment differs from the first embodiment in that the type notification unit 104 is deleted, and in the functions of the storage unit 121-2 and the determination unit 103-2. The other configurations and functions are similar to those in the block diagram of the IoT device 100 according to the first embodiment in FIG. 2, so the same reference numerals are used and the description thereof will be omitted here.

[0097] Storage unit 121-2 differs from storage unit 121 of the first embodiment in that storage unit 121-2 further stores correspondence information for determining a destination device. The data structure of the correspondence information is similar to that of the first embodiment (FIG. 4).

[0098] The determination unit 103-2 refers to the correspondence information stored in the storage unit 121-2 and determines another IoT device 100-2 corresponding to the type included in the correspondence information as the destination device. When multiple pieces of identification information are associated with one type in the correspondence information, the determination unit 103-2 may determine any one of the IoT devices 100-2 (identification information) as the destination device, in the same manner as the selection unit 202 of the server 200 in the first embodiment, taking into consideration at least a part of the processing load, operating status, and network latency of the other IoT device 100-2.

[0099] In addition, when the function of selecting a destination device by server 200 is also made available, type notification unit 104 may be provided as in the first embodiment. For example, when determination unit 103-2 cannot determine a destination device based on the correspondence information stored in storage unit 121-2, determination unit 103-2 may use type notification unit 104 to inquire of server 200 about the destination device.

[0100] Next, a description will be given of information processing by the IoT device 100-2 according to the second embodiment. Fig. 11 is a flowchart showing an example of information processing in the second embodiment.

[0101] Steps S501 to S506 are similar to steps S101 to S106 in the IoT device 100 according to the first embodiment, and therefore a description thereof will be omitted.

[0102] In this embodiment, if it is determined that the type of the processing data is not the target type (step S503: No), the determining unit 103-2 identifies the IoT device 100-2 corresponding to the type of the processing data (non-target type) from the correspondence information stored in the storage unit 121-2, and determines the identified IoT device 100-2 as the destination device (step S507). The transmitting unit 108 transmits the processing data to the IoT device 100-2 corresponding to the determined destination device (step S508).

[0103] In this manner, in the information processing system according to the second embodiment, it is possible to determine the destination device without making an inquiry to the server.

[0104] As described above, according to the first and second embodiments, distributed processing by a plurality of devices can be executed more efficiently.

[0105] Next, the hardware configuration of each apparatus (IoT device, server) according to the first or second embodiment will be described with reference to Fig. 12. Fig. 12 is an explanatory diagram showing an example of the hardware configuration of the apparatus according to the first or second embodiment.

[0106] The device according to the first or second embodiment includes a control device such as a CPU 51, a storage device such as a ROM (Read Only Memory) 52 or a RAM 53, a communication I / F 54 that connects to a network and communicates, and a bus 61 that connects each part.

[0107] The programs executed by the device according to the first or second embodiment are provided in a state that they are pre-installed in the ROM 52 or the like.

[0108] The program executed by the device according to the first or second embodiment may be configured to be provided as a computer program product by being recorded in an installable or executable format on a computer-readable recording medium such as a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk Recordable), or a DVD (Digital Versatile Disk).

[0109] Furthermore, the program executed by the device according to the first or second embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the program executed by the device according to the first or second embodiment may be provided or distributed via a network such as the Internet.

[0110] The program executed by the device according to the first or second embodiment can cause a computer to function as each part of the device described above. In this computer, the CPU 51 can read the program from a computer-readable storage medium onto a main storage device and execute the program.

[0111] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0112] 10. Information Processing Systems 100a, 100b, 100c, 100-2 IoT devices 101 Reception 102 Detection unit 103, 103-2 Decision Section 104 Type notification section 105 Receiving unit 106 Selection section 107 Executive Department 108 Transmitter 121, 121-2 Storage section 150 Sensors 200 Servers 201 Receiving unit 202 Selection section 203 Device notification section 221 Storage section 300 Network

Claims

1. An information processing system including a plurality of information processing devices and a server device, The information processing devices include A reception unit that receives input of processing data; A detection unit that detects whether a type of the processing data is a target type that is a target type of a predetermined first processing or a non-target type that is not a target type of the first processing; an execution unit that executes the first processing on the processing data when it is detected that the type of the processing data is the target type; a determination unit that, when it is detected that the type of the processing data is the non-target type, determines another information processing device that processes data of the non-target type as a destination device; a transmission unit that transmits at least a portion of the processing data to the determined destination device; a type notification unit that notifies the server device of notification information including the non-target type when it is detected that the type of the processing data is the non-target type; The determination unit determines the destination device based on device information indicating a device that processes the non-target type of data, the device information being transmitted from the server device in response to the notification information; The server device includes: a receiving unit that receives the notification information from the information processing device; a selection unit that selects, based on the non-target type included in the notification information, an apparatus that processes the non-target type of data from among the plurality of information processing apparatuses; a device notification unit that notifies the information processing device that has transmitted the notification information of the device information indicating the selected information processing device; the selection unit selects, from among the plurality of information processing devices, a device that processes the non-target type of data, based on at least a part of a processing load, an operating status, and a network delay of the plurality of information processing devices; Information processing system.

2. An information processing system having a plurality of information processing devices, The information processing devices include A reception unit that receives input of processing data; A detection unit that detects whether a type of the processing data is a target type that is a target type of a predetermined first processing or a non-target type that is not a target type of the first processing; an execution unit that executes the first processing on the processing data when it is detected that the type of the processing data is the target type; a determination unit that, when it is detected that the type of the processing data is the non-target type, determines another information processing device that processes data of the non-target type as a destination device; a transmission unit that transmits at least a part of the processing data to the determined destination device; The detection unit further detects whether a type of result data, which is a processing result of the first processing by the execution unit or a processing result of the processing data executed by the transmission destination device, is a predetermined additional processing type; When it is detected that the type of the result data is the additional processing type, the determination unit further determines, as a destination device, another of the information processing devices that processes the result data. Information processing system.

3. At least some of the plurality of information processing devices are defined as the first processing, the first processing being different from each other.

3. The information processing system according to claim 1 or 2.

4. The detection unit executes a recognition process for recognizing a type of data for the processing data, and detects whether the type of the processing data is the target type or the non-target type based on a result of the recognition process.

3. The information processing system according to claim 1 or 2.

5. the receiving unit receives input of the processing data from each of a plurality of input devices; the detection unit detects, for each of the plurality of processing data received from each of a plurality of input devices, whether the type of the processing data is the target type or the non-target type; 3. The information processing system according to claim 1 or 2.

6. the transmitting unit transmits partial data included in one of the processing data or a part of the processing data among a plurality of input processing data to the destination device; 3. The information processing system according to claim 1 or 2.

Images