Information processing device, inference system, and control method

Abstract

This information processing device (40) comprises: inference processing devices (105_1-105_n) which have different performances or configurations; acquisition units (131, 141) which acquire a trained model, inference data that is to be used for inference processing, and an inference control program for changing a data format of inference data to a data format corresponding to a first inference processing device that executes inference processing among the inference processing devices (105_1-105_n), and for executing the inference processing in the first inference processing device; a data format conversion unit (170) which converts the data format of inference data; and an inference control unit (180) which executes, in the first inference processing device, the inference processing using the trained learning model and the converted inference data of which the data format is converted.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an inference system, and a control method. [Background technology]

[0002] In recent years, the performance of central processing units (CPUs), graphics processing units (GPUs), etc. has improved. CPUs, GPUs, etc. are used in detection processing of suspicious objects or suspicious people, real-time translation, autonomous driving, etc. Also, a number of devices have been proposed, each having an analysis unit that executes analysis processing of the same content (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2013 / 145530 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there may be a case where a plurality of inference processing devices exist. The inference processing devices can perform inference using data. When the performance and configuration of each of the plurality of inference processing devices are the same, there is no need to convert the data format of the data. However, when the performance or configuration of each of the plurality of inference processing devices differs, the data must be converted into a data format that is compatible with the inference processing devices. Therefore, when the performance or configuration of each of the plurality of inference processing devices differs, the user converts the data into a data format that is compatible with the inference processing devices. However, having the user perform the conversion work increases the burden on the user.

[0005] The purpose of this disclosure is to ease the burden. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, there is provided an information processing device. The information processing device includes a plurality of inference processing devices each having a different performance or configuration, and a trained model, inference data used in the inference processing, and a data format corresponding to a first inference processing device that executes the inference processing among the plurality of inference processing devices. Data an acquisition unit that acquires an inference control program for converting the inference process and causing the first inference processing device to execute the inference process; A preprocessing unit that performs preprocessing on the inference data; and a preprocessed data obtained by performing the preprocessing. A data format conversion unit that converts the data format of the inference data, and a conversion unit that converts the data format of the inference data. Pretreated an inference control unit that causes the first inference processing device to execute the inference process using inference data and the trained model; a post-processing unit that performs post-processing on an inference result or data converted from the inference result; and a schedule control unit that calculates a pre-processing time for executing the pre-processing, a post-processing time for executing the post-processing, and an inference processing execution time for executing the inference processing, based on a pre-processing time that is the time when the pre-processing is executed, a post-processing time that is the time when the post-processing is executed, and an inference processing time that is the time when the inference processing is executed, so that a plurality of the pre-processing processes are not executed simultaneously, or a plurality of the post-processing processes are not executed simultaneously, or the pre-processing and the post-processing are not executed simultaneously; has. When the preprocessing unit executes the preprocessing, it executes the preprocessing at the preprocessing time. When the inference control unit causes the first inference processing device to execute the inference processing, it causes the first inference processing device to execute the inference processing at the inference processing execution time. When the post-processing unit executes the post-processing, it executes the post-processing at the post-processing time. Effect of the Invention

[0007] According to the present disclosure, the burden can be reduced. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an inference processing system according to a first embodiment. [Diagram 2] FIG. 1 is a diagram (part 1) showing a practical example of the inference processing system of the first embodiment. [Diagram 3] FIG. 2 is a diagram (part 2) showing a practical example of the inference processing system of embodiment 1. [Figure 4] FIG. 3 is a diagram (part 3) showing a practical example of the inference processing system of embodiment 1. [Diagram 5] FIG. 1 is a diagram showing a case where an information processing device of embodiment 1 has an inference processing system. [Figure 6] 1 is a block diagram showing functions of an information processing device according to a first embodiment; [Figure 7] 4 is a block diagram showing the functions of a schedule control unit according to the first embodiment. FIG. [Figure 8] 1 is a flowchart (part 1) illustrating an example of processing executed by the information processing device according to the first embodiment. [Figure 9] 11 is a flowchart (part 2) illustrating an example of the process executed by the information processing device according to the first embodiment. [Figure 10] FIG. 11 is a diagram illustrating an example of a situation in which the load of the second embodiment increases. [Figure 11] FIG. 11 is a block diagram showing the functions of an information processing device according to a second embodiment. [Figure 12] 13 is a flowchart (part 1) illustrating an example of processing executed by an information processing device according to the second embodiment. [Figure 13] 13 is a flowchart (part 2) illustrating an example of processing executed by the information processing device according to the second embodiment. [Figure 14] FIG. 11 is a diagram illustrating an example of a situation in which the control of the second embodiment is executed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment will be described with reference to the drawings. The following embodiment is merely an example, and various modifications are possible within the scope of the present disclosure.

[0010] Embodiment 1 1 is a diagram showing an inference processing system according to embodiment 1. The inference processing system 100 includes an interface 101, a processor 102, a volatile storage device 103, a non-volatile storage device 104, and inference processing devices 105_1, 105_2, . . . , 105_n (n is a positive integer). The inference processing system 100 may also include a processing circuit.

[0011] The interface 101 transmits various information to the processor 102. Furthermore, when the inference processing system 100 is connected to an external device, the interface 101 communicates with the external device. For example, the processor 102 is a CPU. The processor 102 may be a multiprocessor. For example, the volatile storage device 103 is a RAM (Random Access Memory). For example, the non-volatile storage device 104 is a HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0012] Each of the inference processing devices 105_1,...,105_n has different performance or configuration. Each of the inference processing devices 105_1,...,105_n may be expressed as a device with different performance or type. For example, the inference processing devices 105_1,...,105_n may be a GPU, a Field Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), a Dynamically Reconfigurable Processor (DRP), etc.

[0013] Next, a practical example of the inference processing system 100 will be described. FIG. 2 is a diagram (part 1) showing a practical example of the inference processing system of the first embodiment. FIG. 2 shows a case where a surveillance camera 10 has an inference processing system 100. A recorder 11 is also connected to the surveillance camera 10. A PC (Personal Computer) may be connected to the surveillance camera 10 instead of the recorder 11. For example, the inference processing system 100 performs person inference using an image (i.e., inference data) generated by the surveillance camera 10. The surveillance camera 10 transmits the image and the inference result to the recorder 11.

[0014] FIG. 3 is a diagram (part 2) showing a practical example of the inference processing system of the first embodiment. FIG. 3 shows a case where the recorder 21 has the inference processing system 100. A surveillance camera 20 is connected to the recorder 21. A PC may be connected to the surveillance camera 20 instead of the recorder 21. The surveillance camera 20 transmits an image to the recorder 21. Then, for example, the inference processing system 100 of the recorder 21 performs person inference using the image (i.e., inference data) generated by the surveillance camera 20.

[0015] Fig. 4 is a diagram (part 3) showing a practical example of the inference processing system according to the embodiment 1. Fig. 4 shows a surveillance camera 30 and a recorder 31. Instead of the recorder 31, a PC may be connected to the surveillance camera 30.

[0016] The surveillance camera 30 has an inference processing system 100a. The recorder 31 has an inference processing system 100b. The inference processing systems 100a and 100b achieve the same performance as the inference processing system 100.

[0017] For example, inference processing system 100a includes inference processing device 105_1. Inference processing system 100b includes inference processing devices 105_2, . . . , 105_n. In this manner, inference processing devices may be distributed.

[0018] An apparatus having the inference processing system 100 is called an information processing apparatus. For example, the surveillance camera 10 or the recorder 31 is an information processing apparatus. In the following description, the functions of the inference processing system 100 will be explained using the following information processing device.

[0019] FIG. 5 is a diagram showing a case where the information processing device of the first embodiment has an inference processing system. The information processing device 40 has an inference processing system 100. The information processing device 40 is a device that executes a control method. The inference processing device 105_1 is realized by a GPU. The inference processing device 105_2 is realized by a high-performance GPU. The inference processing device 105_3 is realized by an FPGA. The inference processing device 105_4 is realized by a DSP.

[0020] Next, functions of the information processing device 40 will be described. 6 is a block diagram showing functions of the information processing device of embodiment 1. The information processing device 40 has an interface unit 110, a schedule control unit 120, an inference data storage unit 130, an acquisition unit 131, an inference control program storage unit 140, an acquisition unit 141, a pre-processing unit 151, a post-processing unit 152, a common interface unit 160, a data format conversion unit 170, an inference control unit 180, and an output unit 190.

[0021] The interface unit 110 may be realized by the interface 101 . The inference data storage unit 130 and the inference control program storage unit 140 may be realized as storage areas secured in the volatile storage device 103 or the non-volatile storage device 104 .

[0022] A part or all of the schedule control unit 120, the acquisition unit 131, the acquisition unit 141, the pre-processing unit 151, the post-processing unit 152, the common interface unit 160, and the output unit 190 may be realized by a processing circuit. Also, a part or all of the schedule control unit 120, the acquisition unit 131, the acquisition unit 141, the pre-processing unit 151, the post-processing unit 152, the common interface unit 160, and the output unit 190 may be realized as a module of a program executed by the processor 102.

[0023] The interface unit 110 transmits the inference scenario, information indicating the configuration of the inference processing device, the trained model, and the memory usage used in the inference processing to the schedule control unit 120. The inference scenario includes a list of inference processing, an inference order, a function used in pre-processing or post-processing, and an initial value of a parameter. The trained model may also be called an inference model. The information indicating the configuration of the inference processing device includes the number of inference processing devices and the performance (e.g., memory capacity, number of cores, clock frequency, number of threads, bus speed) of the inference processing devices 105_1 to 105_4.

[0024] Furthermore, the interface unit 110 transmits inference data, which is data used in the inference process, to the inference data storage unit 130. For example, the inference data is an image. Furthermore, the interface unit 110 transmits the inference control program to the inference control program storage unit 140. The inference control program is a program for converting the data format of the inference data into a data format suitable for an inference processing device that executes the inference processing, and for causing the inference processing device to execute the inference processing.

[0025] Next, the schedule control unit 120 will be described. 7 is a block diagram showing functions of the schedule control unit according to embodiment 1. The schedule control unit 120 has an inference scenario storage unit 121, an execution frequency detection unit 122, a configuration storage unit 123, a memory usage storage unit 124, a determination unit 125, and an inference instruction unit 126.

[0026] The inference scenario storage unit 121, the configuration storage unit 123, and the memory usage storage unit 124 may be realized as storage areas secured in the volatile storage unit 103 or the non-volatile storage unit 104. A part or all of the execution frequency detection unit 122, the determination unit 125, and the inference instruction unit 126 may be realized by a processing circuit. Also, a part or all of the execution frequency detection unit 122, the determination unit 125, and the inference instruction unit 126 may be realized as a module of a program executed by the processor 102.

[0027] The inference scenario storage unit 121 stores the inference scenario transmitted by the interface unit 110. The inference scenario storage unit 121 may store a trained model. The execution frequency detection unit 122 detects the execution frequency of each inference process based on the inference scenario. The configuration storage unit 123 stores information transmitted by the interface unit 110 that indicates the configuration of the inference processing device. The memory usage storage unit 124 stores the memory usage transmitted by the interface unit 110 and used in the inference process.

[0028] The determination unit 125 determines an inference processing device to execute an inference process based on the information stored in each of the inference scenario storage unit 121, the configuration storage unit 123, and the memory usage storage unit 124, and the execution frequency of each inference process. The determination unit 125 may also determine to cause the processor 102 to execute an inference process with a low memory usage, and to cause the inference processing devices 105_3 and 105_4 to execute the other inference processes. Furthermore, when the inference scenario indicates that three inference processes are executed consecutively, the determination unit 125 may determine to cause one inference processing device to execute two inference processes with a low execution frequency, and to cause another inference processing device to execute the remaining one inference process. In this way, the information processing device 40 may distribute the inference processes and cause multiple inference processing devices to execute them.

[0029] The inference instruction unit 126 transmits the inference scenario, the inference instruction, and the learned model to the acquisition unit 131. The inference instruction includes device information indicating the inference processing device that is to execute the inference process.

[0030] Next, returning to FIG. 6, the acquisition unit 131 will be described. The acquisition unit 131 acquires the inference scenario, the inference instruction, and the trained model from the inference instruction unit 126. The acquisition unit 131 may acquire the inference scenario, the inference instruction, and the trained model from an external device via the interface unit 110. The external device is a device that can be connected to the information processing device 40. The external device is omitted in the diagram.

[0031] Moreover, the acquiring unit 131 acquires the inference data from the inference data storage unit 130. The acquiring unit 131 may acquire the inference data from an external device via the interface unit 110.

[0032] The preprocessing unit 151 may perform preprocessing on the inference data. For example, the preprocessing is normalization of the inference data, image processing, etc. For example, the preprocessing unit 151 performs the preprocessing using a function used for the preprocessing and an initial value of a parameter included in the inference scenario.

[0033] The acquiring unit 141 acquires the inference control program from the inference control program storage unit 140. For example, when the inference instruction includes device information indicating the inference processing device 105_1 that is to execute the inference processing among the inference processing devices 105_1 to 105_4, the acquiring unit 141 acquires, based on the device information, an inference control program for converting the data format of the inference data into a data format suitable for the inference processing device 105_1 and for causing the inference processing device 105_1 to execute the inference processing, from the inference control program storage unit 140. The acquiring unit 141 may acquire the inference control program from an external device via the interface unit 110.

[0034] The common interface unit 160 transmits the inference data, the inference control program, the inference scenario, the trained model, and the inference instruction to the data format conversion unit 170. In addition, when the preprocessing unit 151 executes preprocessing on the inference data, the common interface unit 160 transmits to the data format conversion unit 170 the preprocessed inference data obtained by executing the preprocessing, the inference control program, the inference scenario, the trained model, and the inference instruction.

[0035] Here, the data format conversion unit 170 and the inference control unit 180 may be realized by a processing circuit that uses an inference control program. Also, the data format conversion unit 170 and the inference control unit 180 may be realized as modules of the inference control program executed by the processor 102. In addition, when preprocessing is executed, the inference control program is a program for converting the data format of the preprocessed inference data into a data format that corresponds to the inference processing device that executes the inference processing, and is a program for causing the inference processing device to execute the inference processing.

[0036] The data format conversion unit 170 converts the data format of the inference data. The data format conversion unit 170 transmits the converted inference data, the inference scenario, the trained model, and the inference instruction to the inference control unit 180. In addition, when preprocessing is performed, the data format conversion unit 170 converts the data format of the preprocessed inference data. The data format conversion unit 170 transmits the converted preprocessed inference data, the inference scenario, the trained model, and the inference instruction to the inference control unit 180.

[0037] The inference control unit 180 causes the inference processing device to execute an inference process using the converted inference data and the learned model. In particular, when the device information indicates the inference processing device 105_1, the inference control unit 180 transmits the converted inference data and the learned model to the inference processing device 105_1. The inference control unit 180 transmits an inference execution instruction to the inference processing device 105_1. As a result, the inference processing device 105_1 executes the inference process using the converted inference data and the learned model. Furthermore, when preprocessing has been performed, the inference control unit 180 causes the inference processing device to execute inference processing using the converted preprocessed inference data and the trained model.

[0038] After the inference process is executed, the inference control unit 180 acquires the inference result from the inference processing device that executed the inference process. The inference control unit 180 transmits the inference result to the data format conversion unit 170. As will be described later, if the inference result is not converted into common data, the inference control unit 180 transmits the inference result to the post-processing unit 152.

[0039] The data format conversion unit 170 converts the inference result into common data. The data format conversion unit 170 transmits the converted data to the post-processing unit 152.

[0040] The post-processing unit 152 performs post-processing on the received data (i.e., the inference result or the converted data). For example, the post-processing includes feature analysis of the inference result, imaging of sequence data, and the like. For example, the post-processing unit 152 performs post-processing using a function used for post-processing and initial parameter values ​​included in the inference scenario. The post-processing unit 152 transmits the data obtained by the post-processing to the output unit 190.

[0041] The output unit 190 outputs the received data to the interface unit 110. In addition, the output unit 190 outputs an inference processing completion notification to the inference instruction unit 126.

[0042] Here, the acquiring unit 131 and the acquiring unit 141 may be realized by a single acquiring unit. Furthermore, the number of acquisition units, pre-processing units, post-processing units, common interface units, data format conversion units, and inference control units may be the same as the number of inference processing devices.

[0043] Next, the process executed by the information processing device 40 will be described with reference to a flowchart. FIG. 8 is a flowchart (part 1) illustrating an example of processing executed by the information processing device according to the first embodiment. (Step S11) The acquisition unit 131 acquires an inference scenario, a trained model, and an inference instruction from the schedule control unit 120. (Step S12) The acquisition unit 131 acquires inference data from the inference data storage unit . (Step S13) The preprocessing unit 151 performs preprocessing on the inference data. (Step S14) The acquisition unit 141 acquires the inference control program from the inference control program storage unit 140.

[0044] (Step S15) The common interface unit 160 transmits the preprocessed inference data, the inference control program, the inference scenario, the trained model, and the inference instruction to the data format conversion unit 170. (Step S16) The data format conversion unit 170 converts the data format of the preprocessed inference data. (Step S17) The inference control unit 180 causes the inference processing device to execute an inference process using the pre-conversion processed inference data and the trained model. For example, the inference control unit 180 transmits an inference execution instruction to the inference processing device. (Step S18) The inference processing device executes the inference processing.

[0045] FIG. 9 is a flowchart (part 2) illustrating an example of the process executed by the information processing device according to the first embodiment. (Step S21) The inference control unit 180 transmits the inference result to the data format conversion unit 170. (Step S22) The data format conversion unit 170 converts the inference result into common data. (Step S23) The post-processing unit 152 performs post-processing on the received data. (Step S24) The output unit 190 outputs the received data to the interface unit 110. (Step S25) The output unit 190 outputs an inference processing completion notification to the schedule control unit 120.

[0046] According to the first embodiment, the information processing device 40 has inference processing devices 105_1 to 105_4. The inference processing devices 105_1 to 105_4 have different performance or configuration. The information processing device 40 automatically converts inference data into a data format corresponding to the inference processing device that executes the inference processing. Therefore, the user does not need to perform the conversion work. Therefore, the information processing device 40 can reduce the burden.

[0047] The acquired inference control program is a program suited to the inference processing device, and therefore the information processing device 40 can perform control suited to the inference processing device by using the inference control program.

[0048] When the performance or configuration of each of the inference processing devices 105_1 to 105_4 is changed, the schedule control unit 120 automatically performs a process of switching the inference control program according to the inference processing device, thereby reducing the burden on the user of setting the inference processing to be executed by the inference processing devices 105_1 to 105_4. Furthermore, when a new inference processing device is added to the inference processing system 100, an inference processing program for the added inference processing device is created and the inference processing program is input to the inference control program storage unit 140, whereby the schedule control unit 120 determines the inference processing device to execute the inference processing. Then, the acquisition unit 141 acquires the inference control program from the inference control program storage unit 140. This reduces the burden on the user of setting the inference processing to be executed by the inference processing device.

[0049] Embodiment 2 Next, a description will be given of embodiment 2. In embodiment 2, differences from embodiment 1 will be mainly described. In embodiment 2, descriptions of matters common to embodiment 1 will be omitted. As described above, the pre-processing unit 151 and the post-processing unit 152 may be realized by a processing circuit. Also, the pre-processing unit 151 and the post-processing unit 152 may be realized as a program module executed by the processor 102. When the pre-processing unit 151 and the post-processing unit 152 are executed simultaneously, a high load is imposed on the processing circuit or the processor 102. A situation in which the load is high will be specifically described.

[0050] 10 is a diagram showing an example of a situation in which the load of the second embodiment becomes high. The preprocessing unit 151 executes preprocessing before causing the inference processing device 105_1 to execute the inference processing. The preprocessing unit 151 also executes preprocessing before causing the inference processing device 105_2 to execute the inference processing. In this way, when two preprocessing processes are executed simultaneously, the load of the processing circuit or the processor 102 becomes high.

[0051] 10 shows a case where a post-processing after the inference processing of the inference processing device 105_1 is executed and a post-processing after the inference processing of the inference processing device 105_2 is executed simultaneously. In this way, when two post-processings are executed simultaneously, the load on the processing circuit or the processor 102 becomes high. 10 shows a case where pre-processing before the inference processing device 105_1 executes the inference processing and post-processing after the inference processing device 105_2 executes the inference processing are executed simultaneously. In this way, when the pre-processing and the post-processing are executed simultaneously, the load on the processing circuit or the processor 102 becomes high. Therefore, the information processing device 40 performs control so that the processes of the pre-processing unit 151 and the post-processing unit 152 are not executed simultaneously.

[0052] 11 is a block diagram showing functions of the information processing device of the embodiment 2. The information processing device 40 further includes a time transmission unit 153. The time transmission unit 153 may be realized by a processing circuit. In addition, the time transmission unit 153 may be realized as a module of a program executed by the processor 102.

[0053] The time transmitting unit 153 receives a preprocessing time from the preprocessing unit 151. The preprocessing time is the time during which the preprocessing is performed. In addition, the time transmitting unit 153 receives a post-processing time from the post-processing unit 152. The post-processing time is the time during which the post-processing is performed. The time transmitting unit 153 transmits the preprocessing time and the post-processing time to the inference instruction unit 126.

[0054] Furthermore, the inference control unit 180 acquires the inference processing time from the inference processing device. For example, the inference control unit 180 acquires the inference processing time from the inference processing device 105_1. The inference processing time is the time during which the inference processing is executed. The inference control unit 180 transmits the inference processing time to the time transmitting unit 153 via the data format conversion unit 170, the common interface unit 160, and the post-processing unit 152. The time transmitting unit 153 transmits the inference processing time to the inference instruction unit 126.

[0055] The inference instruction unit 126 calculates a preprocessing time at which preprocessing is to be executed, a postprocessing time at which postprocessing is to be executed, and an inference processing execution time at which inference processing is to be executed, based on the preprocessing time, the postprocessing time, and the inference processing time. In this way, the inference instruction unit 126 calculates a time at which multiple preprocessing processes are not executed simultaneously. The inference instruction unit 126 also calculates a time at which multiple postprocessing processes are not executed simultaneously. Furthermore, the inference instruction unit 126 calculates a time at which preprocessing and postprocessing are not executed simultaneously. The inference instruction unit 126 transmits the inference scenario, the inference instruction, the preprocessing time, the postprocessing time, the trained model, and the inference processing execution time to the acquisition unit 131.

[0056] The acquisition unit 131 acquires the inference scenario, the inference instruction, the pre-processing time, the post-processing time, the trained model, and the inference processing execution time from the inference instruction unit 126. The pre-processing unit 151 executes pre-processing at a pre-processing time. The inference control unit 180 causes the inference processing device to execute the inference processing at an inference processing execution time. The post-processing unit 152 executes post-processing at a post-processing time.

[0057] Next, the process executed by the information processing device 40 will be described with reference to a flowchart. Fig. 12 is a flowchart (part 1) showing an example of processing executed by the information processing device of the second embodiment. The processing in Fig. 12 differs from the processing in Fig. 8 in that steps S11a, 13a, 17a, and 18a are executed. Therefore, in Fig. 12, steps S11a, 13a, 17a, and 18a will be described. Descriptions of the processing other than steps S11a, 13a, 17a, and 18a will be omitted.

[0058] (Step S11a) The acquisition unit 131 acquires an inference scenario, an inference instruction, a pre-processing time, a post-processing time, a trained model, and an inference processing execution time from the schedule control unit 120. (Step S13a) The preprocessing unit 151 executes preprocessing on the inference data at the preprocessing time. (Step S17a) The inference control unit 180 causes the inference processing device to execute the inference processing using the pre-conversion processed inference data and the trained model at the inference processing execution time. For example, the inference control unit 180 transmits an inference execution instruction to the inference processing device. (Step S18a) The inference processing device executes the inference processing at the inference processing execution time.

[0059] Fig. 13 is a flowchart (part 2) showing an example of processing executed by the information processing device of the second embodiment. The processing in Fig. 13 differs from the processing in Fig. 9 in that step S23a is executed. Therefore, step S23a will be described in Fig. 13. Descriptions of the processing other than step S23a will be omitted. (Step S23a) The post-processing unit 152 performs post-processing on the received data at the post-processing time.

[0060] A case will be illustrated in which the information processing device 40 performs control so that the processes of the pre-processing unit 151 and the post-processing unit 152 are not executed simultaneously. Fig. 14 is a diagram showing an example of a situation in which the control of the second embodiment is executed. As shown in Fig. 14, the processes of the pre-processing unit 151 and the post-processing unit 152 are not executed simultaneously. Therefore, the load on the processing circuit or the processor 102 is reduced.

[0061] According to the second embodiment, the information processing device 40 can reduce the load on the processing circuit or the processor 102.

[0062] Furthermore, the pre-processing time, the post-processing time, and the inference processing time may also be transmitted to the determination unit 125. When determining an inference processing device to execute an inference processing, the determination unit 125 may determine the inference processing device using the pre-processing time, the post-processing time, and the inference processing time.

[0063] As shown in FIG. 4, the inference processing device may be distributed. For example, the surveillance camera 30 may be referred to as a first device. For example, the recorder 31 may be referred to as a second device. A system including the surveillance camera 30 and the recorder 31 may be referred to as an inference system. The surveillance camera 30 has one or more inference processing devices. The recorder 31 has one or more inference processing devices. The surveillance camera 30 or the recorder 31 has the functions shown in FIG. 6. For example, the surveillance camera 30 or the recorder 31 has an acquisition unit that converts the inference data into a data format corresponding to the first inference processing device that executes the inference processing among the one or more inference processing devices of the surveillance camera 30 and the one or more inference processing devices of the recorder 31, and acquires an inference control program for causing the first inference processing device to execute the inference processing, a data format conversion unit that converts the data format of the inference data, and an inference control unit that causes the first inference processing device to execute the inference processing using the converted inference data and the learned model.

[0064] The features of each of the embodiments described above can be combined with each other as appropriate. [Explanation of symbols]

[0065] 10 surveillance camera, 11 recorder, 20 surveillance camera, 21 recorder, 30 surveillance camera, 31 recorder, 40 information processing device, 100 inference processing system, 100a inference processing system, 100b inference processing system, 101 interface, 102 processor, 103 volatile storage device, 104 non-volatile storage device, 105_1, 105_2, 105_3, 105_4,..., 105_n inference processing device, 110 interface unit, 120 schedule control unit, 121 inference scenario storage unit, 122 execution frequency detection unit, 123 configuration storage unit, 124 memory usage storage unit, 125 determination unit, 126 inference instruction unit, 130 inference data storage unit, 131 acquisition unit, 140 Inference control program memory unit, 141 acquisition unit, 151 preprocessing unit, 152 postprocessing unit, 153 time transmission unit, 160 common interface unit, 170 data format conversion unit, 180 inference control unit, 190 output unit.

Claims

1. A plurality of inference processing devices each having different performance or configuration; an acquisition unit that acquires a learned model, inference data that is data used in an inference process, and an inference control program that converts the data into a data format corresponding to a first inference processing device among the plurality of inference processing devices that is to execute the inference process, and causes the first inference processing device to execute the inference process; A pre-processing unit that performs pre-processing on the inference data; a data format conversion unit that converts a data format of preprocessed inference data obtained by executing the preprocessing; an inference control unit that causes the first inference processing device to execute the inference processing using pre-processed inference data whose data format has been converted and the trained model; a post-processing unit that performs post-processing on the inference result or data converted from the inference result; a schedule control unit that calculates a preprocessing time for executing the preprocessing, a postprocessing time for executing the postprocessing, and an inference processing execution time for executing the inference processing, based on a preprocessing time that is a time for executing the preprocessing, a postprocessing time that is a time for executing the postprocessing, and an inference processing time that is a time for executing the inference processing, so that a plurality of the preprocessing processes are not executed simultaneously, or a plurality of the postprocessing processes are not executed simultaneously, or the preprocessing and the postprocessing are not executed simultaneously; having When the preprocessing unit executes the preprocessing, the preprocessing unit executes the preprocessing at the preprocessing time; when the inference control unit causes the first inference processing device to execute the inference processing, the inference control unit causes the first inference processing device to execute the inference processing at the inference processing execution time; When the post-processing is to be executed, the post-processing unit executes the post-processing at the post-processing time. Information processing device.

2. a first device having one or more inference processing devices; a second device having one or more inference processing devices; Including, the one or more inference processing devices of the first device and the one or more inference processing devices of the second device have different performance or configurations, The first device or the second device is an acquisition unit that acquires a learned model, inference data that is data used in an inference process, and an inference control program that converts data into a data format corresponding to a first inference processing device that executes the inference process, out of the one or more inference processing devices of the first device and the one or more inference processing devices of the second device, and causes the first inference processing device to execute the inference process; A pre-processing unit that performs pre-processing on the inference data; a data format conversion unit that converts a data format of preprocessed inference data obtained by executing the preprocessing; an inference control unit that causes the first inference processing device to execute the inference processing using pre-processed inference data whose data format has been converted and the trained model; a post-processing unit that performs post-processing on the inference result or data converted from the inference result; a schedule control unit that calculates a preprocessing time for executing the preprocessing, a postprocessing time for executing the postprocessing, and an inference processing execution time for executing the inference processing, based on a preprocessing time that is a time for executing the preprocessing, a postprocessing time that is a time for executing the postprocessing, and an inference processing time that is a time for executing the inference processing, so that a plurality of the preprocessing processes are not executed simultaneously, or a plurality of the postprocessing processes are not executed simultaneously, or the preprocessing and the postprocessing are not executed simultaneously; having When the preprocessing unit executes the preprocessing, the preprocessing unit executes the preprocessing at the preprocessing time; when the inference control unit causes the first inference processing device to execute the inference processing, the inference control unit causes the first inference processing device to execute the inference processing at the inference processing execution time; When the post-processing is to be executed, the post-processing unit executes the post-processing at the post-processing time. Inference system.

3. An information processing device having a plurality of inference processing devices each having different performance or configuration, acquiring a learned model, inference data that is data used in an inference process, and an inference control program for converting data into a data format corresponding to a first inference processing device among the plurality of inference processing devices that is to execute the inference process, and for causing the first inference processing device to execute the inference process; performing pre-processing on the inference data; Converting a data format of the preprocessed inference data obtained by performing the preprocessing; causing the first inference processing device to execute the inference processing using the pre-processed inference data whose data format has been converted and the trained model; performing post-processing on the inference result or on the data into which the inference result has been transformed; calculating a preprocessing time at which the preprocessing is to be executed, a postprocessing time at which the postprocessing is to be executed, and an inference processing execution time at which the inference processing is to be executed, based on a preprocessing time which is the time at which the preprocessing is executed, a postprocessing time which is the time at which the postprocessing is executed, and an inference processing time which is the time at which the inference processing is executed, so that a plurality of the preprocessing processes are not executed simultaneously, or a plurality of the postprocessing processes are not executed simultaneously, or so that the preprocessing processes and the postprocessing are not executed simultaneously, When the pre-processing is to be performed, the pre-processing is performed at the pre-processing time; when the inference processing is to be executed by the first inference processing device, the inference processing is executed by the first inference processing device at the inference processing execution time; When the post-processing is to be executed, the post-processing is executed at the post-processing time. Control methods.

Images