Design execution device, design execution system, and design execution method

Abstract

To provide a design execution device, a design execution system, and a design execution method for improving convenience of design and execution.SOLUTION: A design execution device includes: a design unit that includes, for each function, a node containing outline information, without a substance of processing that implements a function, that defines information identifying the processing, can connect a plurality of nodes by an external operation, and generates execution order data that defines an execution order of a specific function group by connecting specific node groups; a generation unit that, according to the execution order data generated by the design unit, by inheriting the outline information of a preceding node among the specific node groups to the subsequent node connected to a succeeding stage of the preceding node, generates configuration information defining the outline information used in each of the specific function groups according to the execution order; and an execution unit that materializes processing for realizing the specific function group using the configuration information generated by the generation unit, and executes the materialized processing in the execution order.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a design execution device, a design execution system, and a design execution method for designing and executing a target system. [Background technology]

[0002] As background art in this technical field, Patent Document 1 discloses a program development support device that generates a program for executing data processing described in a graph format on a target device. This program development support device includes a GUI unit, a program generation unit, a processing execution function database, and a data transfer function database. In this program development support device, if a certain process included in the data processing can be executed by different types of computing devices installed on the target device, the processing execution function database stores processing execution functions for executing the process on each computing device, and the data transfer function database stores data transfer functions for the process. The program development support device allows the GUI unit to select which computing device will execute the process, and the program generation unit loads the processing execution function and data transfer function corresponding to the selected computing device and generates a program for executing the desired data processing on the target device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-062554 Summary of the Invention [Problem to be solved by the invention]

[0004] In order to generate a program for executing the desired data processing on a target device, the technology described in Patent Document 1 requires that the data processing method be read from a graph format. However, the details of this reading method are unclear.

[0005] Furthermore, Patent Document 1 does not describe in detail how to execute a program even if it generates it, and in particular, if the execution platform for program generation and program execution is different, it is necessary to prepare a separate process from the generated program to its execution. Therefore, it is not possible to verify the behavior of the generated program.

[0006] The present invention aims to improve the convenience of design and implementation. [Means for solving the problem]

[0007] A system design and execution device according to one aspect of the invention disclosed in the present application is characterized by having a design unit that has, for each function, a node that does not have the substance of the processing that realizes the function but includes summary information that specifies information that identifies the processing, and that allows connection of a plurality of the nodes through external operations, and that generates execution order data that specifies the execution order of a specific group of functions by connecting specific groups of nodes; a generation unit that generates configuration information that specifies the summary information used in each of the specific groups of functions in accordance with the execution order by inheriting the summary information of a previous node among the specific group of nodes to a subsequent node that is connected subsequent to the previous node in accordance with the execution order data generated by the design unit; and an execution unit that instantiates the processing that realizes the specific group of functions using the configuration information generated by the generation unit, and executes the instantiated processing in the execution order. [Effects of the Invention]

[0008] According to the exemplary embodiments of the present invention, it is possible to improve the convenience of design and implementation. Problems, configurations and effects other than those mentioned above will become clear from the following description of the examples. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram of an example of a system configuration of a design execution system according to a first embodiment. [Figure 2]FIG. 2 is a block diagram showing an example of the hardware configuration of a computer. [Figure 3] FIG. 3 is a block diagram of an example of a functional configuration of the design execution system according to the first embodiment. [Figure 4] FIG. 4 is an explanatory diagram of a detailed system design method performed by the design unit according to the first embodiment. [Figure 5] FIG. 5 is a diagram illustrating a detailed configuration information generating method performed by the generating unit according to the first embodiment. [Figure 6] FIG. 6 is an explanatory diagram of detailed processing related to system execution by the function generating unit, generating unit, holding unit, executing unit, control unit, and processing device according to the first embodiment. [Figure 7] FIG. 7 is a flowchart illustrating an example of a detailed processing procedure from system design to execution by the design execution system according to the first embodiment. [Figure 8] FIG. 8 is a flowchart of a detailed procedure of ID assignment by the design execution system according to the second embodiment. [Figure 9] FIG. 9 is a diagram illustrating a detailed configuration information generating method performed by the generating unit according to the second embodiment. [Figure 10] FIG. 10 is an explanatory diagram of a detailed example of a timing chart when the execution unit according to the second embodiment executes the system. [Figure 11] FIG. 11 is a block diagram of an example of a functional configuration of a design execution system according to the third embodiment. [Figure 12] FIG. 12 is an explanatory diagram of a detailed processing example of generating configuration information in the generating unit of the design execution system according to the fourth embodiment. [Figure 13] FIG. 13 is an explanatory diagram showing a specific example of the overlap integration and node connection adjustment process. [Figure 14] FIG. 14 is a flowchart of a detailed generation process performed by the generation unit of the design execution system according to the fourth embodiment. [Figure 15] FIG. 15 is an explanatory diagram of a detailed processing example of generating configuration information in the generating unit of the design execution system according to the fifth embodiment. [Figure 16] 13 is a flowchart showing a detailed procedure of a generation process performed by a generation unit of the design execution system according to the fifth embodiment. [Figure 17] FIG. 17 is an explanatory diagram of a detailed processing example of configuration information generation in the generation unit of the design execution system according to the sixth embodiment. [Figure 18] 13 is a flowchart showing a detailed procedure of a generation process performed by a generation unit of the design execution system according to the sixth embodiment. [Figure 19] FIG. 19 is an explanatory diagram of a specific example 2 of the process of integrating overlaps and organizing node connections according to the sixth embodiment. [Figure 20] FIG. 20 is a flowchart of a detailed generation process performed by the generation unit of the design execution system according to the sixth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The examples shown below are illustrative of the present invention, and some omissions and simplifications have been made as appropriate for clarity of explanation. The present invention can also be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0011] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0012] Examples of various types of information may be described using expressions such as "table," "list," and "queue," but the various types of information may also be expressed using data structures other than these. For example, various types of information such as "XX table," "XX list," and "XX queue" may also be expressed as "XX information." When describing identification information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these are interchangeable.

[0013] When there are multiple components with the same or similar functions, they may be described using the same reference numeral with different subscripts. When there is no need to distinguish between these multiple components, the subscripts may be omitted.

[0014] In the embodiments, processing performed by executing a program may be described. Here, a computer executes the program using a processor (e.g., a CPU or a GPU) and performs processing defined by the program using storage resources (e.g., a memory) and interface devices (e.g., a communication port). Therefore, the entity performing the processing by executing the program may be the processor. Similarly, the entity performing the processing by executing the program may be a controller, device, system, computer, or node having a processor. The entity performing the processing by executing the program may be any computing unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit may be, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0015] A program may be installed on a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in an embodiment, two or more programs may be realized as one program, or one program may be realized as two or more programs. [Example]

[0016] <Design Execution System> 1, a design execution system 100 includes a server 111 and one or more clients 110. The server and clients are communicatively connected via a network 101 such as the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network). The server 111 is a computer that manages the clients 110.

[0017] The design execution system 100 has a function group creation function, a configuration information generation function, and a system execution function. The function group creation function creates functions required for system design for each processing unit through operations by the user of the design execution system 100. The configuration information generation function designs the system that each user wants to realize by combining functions selected from the function group obtained by the function group creation function, and generates information about the designed system as configuration information. The system execution function executes the designed system from the configuration information.

[0018] The function group creation function, configuration information generation function, and system execution function may be implemented in either the server 111 or the client 110, as long as they are implemented in the design execution system 100. For example, the server 111 may implement the function group creation function, and the client 110 may implement the configuration information generation function and the system execution function. Alternatively, the server 111 may implement the function group creation function and the system design function, and the client 110 may have the system execution function and simply receive the system designed by the server 111 and execute the received system.

[0019] A computer that implements the function group creation function is referred to as a function group creation device, and a computer that implements at least the system execution function of the configuration information generation function and the system execution function is referred to as a design execution device. Also, while a client-server type design execution system 100 is exemplified in FIG. 1 , a standalone type system design execution device may also be used. For ease of explanation, the first embodiment will be described taking as an example a design execution system 100 in which the server 111 is a function group creation device that implements the function group creation function, and the client 110 is a design execution device that implements the system execution function. The client 110 may transmit information to a browser of an external device (not shown, which may be the server 111) that can communicate with the network 101, and be remotely operated on the browser of the external device.

[0020] <Example of computer hardware configuration> 2 is a block diagram showing an example of the hardware configuration of a computer (server 111, client 110). The computer 200 has a processor 201, a storage device 202, a processing device 203, an input device 204, an output device 205, and a communication interface (communication IF) 206. The processor 201, the storage device 202, the processing device 203, the input device 204, the output device 205, and the communication IF 206 are connected by a bus 207. The processor 201 controls the computer 200. The storage device 202 serves as a working area for the processor 201. The storage device 202 is a non-transitory or temporary recording medium that stores various programs and data. Examples of the storage device 202 include a read-only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), and a flash memory.

[0021] The processing device 203 is an accelerator that speeds up a specific function, and specifically, for example, a GPU (Graphics Processing Unit), an AI (Artificial Intelligence) chip, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array).

[0022] The input device 204 inputs data. Examples of the input device 204 include various sensors such as a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, and a camera. The output device 205 outputs data. Examples of the output device 205 include a display, a printer, and a speaker. The communication IF 206 connects to the network 101 and transmits and receives data. Note that the processor 201 may be composed of multiple processors.

[0023] <Example of functional configuration of design execution system 100> 3 is a block diagram illustrating an example of a functional configuration of the design execution system 100 according to the first embodiment. The server 111 includes a function group creation unit 130. The client 110 includes a design unit 120, a generation unit 121, a storage unit 122, an execution unit 123, a control unit 124, and a processing device 203.

[0024] Specifically, these are realized by, for example, having the processor 201 execute a program stored in the storage device 202 shown in Fig. 2, or by the storage device 202. First, an example of the functional configuration on the server 111 side will be described.

[0025] The function group creation unit 130, as a function group creation function, creates a function group 420 required for system design by a user who uses the design execution system 100, and outputs the created function group 420 to the storage unit 122. Each function of the function group 420 may be created in a program format such as a class or a function, or may be created as a library to black-box the contents of the processing.

[0026] Here, for some functions of the function group 420, it is possible to select a processing device 203, and the processing speed increases depending on the selected processing device 203. Hereinafter, this function will be referred to as a "specific speed-up function."

[0027] Next, an example of a functional configuration on the client 110 side will be described. As one of its configuration information generation functions, the design unit 120 uses a graphical user interface to connect nodes having function summary information to design a design object (a system to be executed by the client 110), and outputs system information that specifies the execution order of processes realized by functions corresponding to the nodes adopted in the design as execution order data to the generation unit 121. The function summary information does not have the substance of the process that realizes the function, but is information for identifying what process is to be performed. As long as it can identify what process is to be performed, for example, numerical values ​​defined in advance within the design execution system 100 may be used as the function summary information.

[0028] The generation unit 121 generates configuration information 320 that defines function summary information according to the execution order from the system information, and outputs the information to the storage unit 122. Details of the processing by the design unit 120 will be described later with reference to FIG. 4, and details of the generation of configuration information by the generation unit 121 will be described later with reference to FIG.

[0029] The retaining unit 122 saves the function group 420 acquired from the function group creation unit 130 and the configuration information 320 acquired from the generation unit 121, and outputs them to the execution unit 123. Here, if there are no changes to the function group 420 or the configuration information 320, it is possible to omit the generation of the function group 420 or the configuration information 320. In this case, the retaining unit 122 outputs the retained function group 420 and the configuration information 320 to the execution unit 123. This eliminates the need to regenerate the configuration information 320 or the function group 420 that has already been generated, thereby shortening processing time and simplifying system startup after design.

[0030] The execution unit 123 instantiates the system information described in the configuration information 320 acquired from the storage unit 122 for the corresponding functions in the function group 420, and executes the system. If the instantiated system includes a specific speed-up function, the execution unit 123 selects a device to perform processing in accordance with the instantiated specific speed-up function, and executes the processing in the processing device 203 via the control unit 124.

[0031] When a specific speed-up function is included in the system materialized by the execution unit 123, the control unit 124 controls the processing device 203 to execute the specific speed-up function.

[0032] The processing device 203 executes the processing of the specific speed-up function under the control of the control unit 124. Details of the execution unit 123, the control unit 124, and the processing device 203 will be described later with reference to FIG.

[0033] <Design Department 120> 4 is an explanatory diagram showing a detailed system design method by the design unit 120. The design unit 120 can be accessed by inputting a specified address from a web browser 400, and provides an operable interface. By enabling access and operation via the web browser 400 in this way, access is possible from clients 110 other than the client 110 equipped with the design unit 120, and remote design is also possible. However, the interface of the design unit 120 is not limited to this, and it may be distributed as an installable tool, for example, and system design may be performed through a dedicated interface screen.

[0034] The design unit 120 has a palette 20 that holds a group of nodes used in system design, and a workspace 240 that is a workspace for designing the system, and displays these on the web browser of the client 110.

[0035] Palette 20 is categorized by the function summary of the node. Specifically, for example, the classification of input function 210 holds input-related nodes called camera input 211 and video input 212. Camera input 211 is a node that holds function summary information related to capturing images from a camera that can be connected to client 110. Video input 212 is a node that holds function summary information related to playing back images of captured video continuously frame by frame.

[0036] Process 220 has nodes related to processes called person detection 221 and skeleton detection 222. Person detection 221 is a node that has function overview information for acquiring the position and size of a person from an acquired image and outputting coordinate information for enclosing the detected person in a rectangle. Skeleton detection 222 is a node that has function overview information for detecting the position of a person's skeleton from an acquired image as coordinate information within the image.

[0037] The output function 230 has output-related nodes called a result drawing 231 and an execution node 232. The result drawing 231 is a node that has function overview information for drawing an image acquired by the input function 210 with the result of execution by the process 220 added thereto. The execution node 232 is the node that is connected last in the system designed by the design unit 120 of the first embodiment, and the details of the operation of the execution node 232 will be described later with reference to FIG.

[0038] The nodes held in the palette 20 described above, the format in which nodes are held for each functional overview, and the functional overviews to be classified are not limited to these, and may be freely defined by the vendor providing the design unit 120, and may be configured to allow additions and deletions.

[0039] A user designing a system operates input device 204 to place required nodes from palette 20 in workspace 240 and connect the nodes according to the processing flow desired by the user. For example, when designing a system that detects the position of a person from an image captured by a camera and displays the detected person's position in a rectangle together with the image, camera input 241, person detection 242, result drawing 243, and execution node 244 are placed in workspace 240, and the nodes (camera input 241, person detection 242, result drawing 243, execution node 244) are connected by vectors 251 to 253 according to the processing order.

[0040] Here, detailed settings for each node can be made for each node using detailed setting screens 246 to 248. The detailed setting screens 246 to 248 are displayed by specifying them with the input device 204, for example. The detailed setting screen 246 is a screen for setting detailed settings for the function of camera input 241, and sets the resolution and frame rate of the camera used for capturing images. The detailed setting screen 247 is a screen for setting detailed settings for the function of person detection 242, and sets the DNN (Deep Neural Network) used for person detection 242 and the device that performs DNN calculations. The detailed setting screen 248 is a screen for setting detailed settings for the function of result rendering 243, and sets a function for enlarging the window size for performing result rendering 243 and a function for displaying the processing speed of the designed system. The setting values ​​in the detailed setting screens 246 to 248 can be changed by user operation.

[0041] As described above, by connecting nodes on the workspace 240, the design unit 120 generates system information, and the generation unit 121 generates configuration information from this system information. Note that in Fig. 4, for example, different numbers are assigned to nodes held in the palette 20 and nodes placed in the workspace 240, such as camera inputs 211 and 241, even if the node names are the same. This is because, compared to the nodes held in the palette 20, the nodes placed in the workspace 240 contain detailed setting values ​​for use in the system, and the different numbers are assigned to distinguish between the nodes and the differences in the functional summary information held by the nodes.

[0042] <Example of generating configuration information> 5 is an explanatory diagram showing an example of generation of configuration information from system information by the generation unit 121. Each node (camera input 241, person detection 242, result rendering 243) that constitutes the system information has function overview information 300, 301, 302 that is not the actual function but identifies what processing the node performs. Note that the function overview information 300, 301, 302 also includes information set on the detail setting screens 246, 247, 248.

[0043] The generation unit 121 outputs the function summary information 300, 301, and 302 of each node to the subsequent node. For example, the camera input 241 outputs its function summary information 300 to the person detection 242.

[0044] At this time, the subsequent node outputs integrated function summary information, which includes the function summary information acquired from the previous node and the function summary information of the node, to the subsequent node. For example, the person detection 242 outputs integrated function summary information 310, which includes the function summary information 300 acquired from the camera input 241 and the function summary information 301 of the person detection 242, to the result rendering 243. Furthermore, the result rendering 243 outputs integrated function summary information 320, which includes the integrated function summary information 310 (function summary information 300, function summary information 301) acquired from the person detection 242 and the function summary information 302 of the result rendering 243, to the execution node 244.

[0045] Each node also assigns an ID to its function summary information. For example, the ID of the function summary information 300 of a starting node (camera input 241) that has no previous nodes is set to "0," and subsequent nodes read the maximum ID value listed in the function summary information or integrated function summary information received from the previous node and assign a value incremented from the read ID value as the ID of that node. Therefore, the ID of the function summary information 301 of the person detection 242 is "1," and the ID of the function summary information 302 of the result drawing 243 is "2." However, the ID assignment method may be such that no duplicate values ​​are assigned among the nodes that make up the designed system, and the ID value of the node is not limited to a value incremented from the value of the ID of the previous node.

[0046] The execution node 244 is the last node connected to the system designed by the design unit 120 in the first embodiment, and the integrated function summary information 320 acquired by the execution node 244 includes the function summary information 300 to 301 of all nodes designed by the design unit 120. The integrated function summary information 320 acquired by the execution node 244 becomes the configuration information generated by the generation unit 121. Hereinafter, this will be referred to as configuration information 320, and the function summary information 300, 301, and 302 will be referred to as configurations 300, 301, and 302.

[0047] <System execution example> FIG. 6 is an explanatory diagram showing detailed processing related to system execution by the function group creation unit 130, the generation unit 121, the storage unit 122, the execution unit 123, the control unit 124, and the processing device 203.

[0048] The storage unit 122 stores the configuration information 302 generated by the generation unit 121 and the function group 420 generated by the function group creation unit 130. The function group 420 stores the entities of the functions (for example, classes 421 to 423) corresponding to all the nodes stored in the palette 20 by the design unit 120.

[0049] The execution unit 123 selects and instantiates the configurations 300, 301, and 302 described in the configuration information 320 from the function group 420 acquired from the storage unit 122. The execution unit 123 lists the instantiated functions (camera input 241, person detection 242, result rendering 243) in the order of processing execution, for example, as in list 431, and executes the listed processing on the execution platform 432. If a specific speed-up function is included in the instantiated functions (camera input 241, person detection 242, result rendering 243), the execution unit 123 selects a device to perform processing in accordance with the instantiated specific speed-up function, and executes the processing on the processing device 203 via the control unit 124.

[0050] For example, the configuration 301 of the person detection 242 in the configuration information 320 contains the description "Device: GPU." In accordance with this description, the execution unit 123 selects the GPU as the processing device 203 only for the processing of the person detection 242, and when processing the person detection 242, outputs the processing content and information about the processing device 203 to the control unit 124. Based on the acquired information, the control unit 124 executes the specified processing on the processing device 203 and outputs the processing result to the execution unit 123.

[0051] <Example of processing procedure of design execution system 100> 7 is a flowchart illustrating an example of a processing procedure of the design execution system 100 according to the embodiment 1. In the function group creation, the server 111 distributes the function group created by the function group creating unit 130 to the client 110 (step S700).

[0052] In generating the configuration information, the client 110 designs the system using a graphical user interface provided by the design unit 120 (step S720).

[0053] Next, the client 110 generates configuration information from the designed system using the generation unit 121 (step S721). The function group 420 generated by the server 111 and the configuration information 320 generated by the client 110 are stored in the client 110 by the storage unit 122 (step S722). Here, the storage of the function group 420 and the storage of the configuration information 320 do not have to be performed simultaneously. For example, when the server 111 generates the function group 420, only the function group 420 may be stored in the client 110, and when the client 110 generates the configuration information 320, the configuration information 320 may be stored in the client 110; the order in which the storage unit 122 stores the function group 420 and the configuration information 320 is not limited.

[0054] In system execution, the client 110 reads the function group 420 and configuration information 320 stored in the storage unit 122 (step S740), materializes the system information described in the configuration information 320 acquired by the execution unit 123 from the function group 420 (step S742), and executes the system using the execution unit 123, the control unit 124, and the processing device 203 (step S743).

[0055] To execute the system, configuration information generation and function creation are executed, and the function group 420 and configuration information 320 must be stored in the storage unit 122. However, as long as the function group 420 and configuration information 320 are stored in the storage unit 122, it is not necessary to execute configuration information generation and function creation every time the system is executed. In other words, as long as the function group 420 and configuration information 320 are stored in the storage unit 122, it is possible to execute only the system execution alone.

[0056] For example, when execution is performed by the execution unit 123, there is no need to launch Nod-RED, which constitutes the design unit 120, and press a start button. Then, for example, execution by the execution unit 123 can be started simply by turning on the power of the design execution device.

[0057] Furthermore, the configuration information generation and the function generation may be executed only when there is a change in the configuration information 320 and the function group 420, respectively, stored in the storage unit 122. Furthermore, by executing the configuration information generation and system execution processes in a single flow, the user can obtain an environment in which the system designed on the graphical interface can be immediately executed.

[0058] Thus, according to the first embodiment, a system is designed using a graphical user interface, configuration information 320 is generated from the designed system, and the system is executed according to the generated configuration information 320, thereby enabling the system to be executed immediately from the system design platform provided by the graphical user interface.

[0059] In particular, by dividing the process into stages into the process of generating the function group 420, the process of designing the system and generating the configuration information 320, and the process of executing the system from the generated configuration information 320 and the function group 420, it is possible to execute system design and system execution in a single flow, or to execute the system execution separately, thereby obtaining a design and execution system that simultaneously achieves ease of system design and embedded implementation. [Example]

[0060] The second embodiment will be described focusing on the differences from the first embodiment. The same reference numerals are used to designate the same parts as the first embodiment, and the description thereof will be omitted. In the second embodiment, a detailed processing method will be described when the configuration information 320 to be generated includes parallel processing or multi-process processing.

[0061] 8 is a flowchart showing a detailed procedure of ID assignment in the generation unit 121 of the design execution system 100 according to the second embodiment. The generation unit 121 determines whether the node to be processed is a start node (step S800). If there is no previous node, it is determined to be a start node. If it is a start node (step S800: Yes), the generation unit 121 assigns "0" to the ID of the node (step S801), and proceeds to the process of step S806.

[0062] If it is determined in step S800 that the node is not the start node (step S800: No), the generating unit 121 acquires function summary information or integrated function summary information from the previous node (step S802).

[0063] If the designed system information includes parallel processing or branching processing, the same processing in the same node may be assigned a different ID in the integrated function summary information obtained from the previous node, resulting in duplication.

[0064] For this reason, the generation unit 121 eliminates the same process of the same node (step S803). Specifically, for example, when the same function overview information is included in a plurality of pieces of input function overview information or integrated function overview information, the generation unit 121, for example, leaves only one piece of the same function overview information and deletes the rest.

[0065] Furthermore, if the designed system information includes parallel processing or branching processing, the same ID may be assigned to a node that parallelizes the processing of another node in the acquired integrated function overview information.

[0066] Therefore, the generation unit 121 assigns a sub-ID for node identification to another process of another node to which the same ID is assigned (step S804). Specifically, for example, the generation unit 121 assigns a unique branch number to the end of the ID value for that other node. The branch numbers are assigned in ascending order among the same IDs.

[0067] For the integrated function summary information after performing the duplicate elimination in step S803 and the allocation of sub-IDs to the same ID in step S804, the generation unit 121 reads the maximum value of the ID and assigns a value incremented from the maximum value of the read ID as the ID of the node (step S805), and proceeds to processing in step S806.

[0068] In step S806, the generation unit 121 checks whether the node to be processed is an end node (step S806). Specifically, for example, if the node is an execution node, it is an end node. If it is an end node (step S806: Yes), the process proceeds to step S808. If it is not an end node (step S806: No), the process proceeds to step S807.

[0069] In step S807, the generation unit 121 outputs the function summary information processed in the node to the subsequent node, and the process returns to step S802.

[0070] In step S808, the generation unit 121 outputs the processed function summary information to the storage unit 122 as the configuration information 320, and the ID management of the node is completed.

[0071] <Example of generating configuration information for a system configuration including parallel processing> FIG. 9 is an explanatory diagram illustrating a detailed processing example of the configuration information generation in the generating unit 121 of the design execution system 100 according to the second embodiment.

[0072] Since camera input 241 is the starting node, generation unit 121 generates function summary information 930 with an ID value of "0", and outputs function summary information 930 to the subsequent node, person detection 242, via vector 251, and to skeleton detection 910 via vector 920.

[0073] In person detection 242, generation unit 121 reads "0", the maximum ID value, from acquired function summary information 930, and sets the incremented value, "1", as the ID value of the node. Generation unit 121 adds function summary information 931 of the node and the generated ID value "1" to acquired function summary information 930 to generate integrated function summary information 941, and outputs function summary information 931 to result rendering 243, which is the subsequent node, via vector 252.

[0074] In skeleton detection 910, the generation unit 121 reads "0", which is the maximum ID value, from the acquired function summary information 930, and sets the incremented value, "1", as the ID value of the node. The generation unit 121 adds function summary information 932 of the node and the generated ID value "1" to the acquired function summary information 930 to generate integrated function summary information 942, and outputs the integrated function summary information 942 to the subsequent node, result rendering 243, via vector 921, and to image distribution 911 via vector 922.

[0075] When the generation of the integrated function summary information 941 in the person detection 242 and the generation of the integrated function summary information 942 in the skeleton detection 910 are completed, the person detection 242 and the skeleton detection 910 have both performed a process of setting the ID value to "1".

[0076] Furthermore, since the result drawing 243 with two inputs receives the integrated function summary information 941 and the integrated function summary information 942, the function summary information 930 output by the camera input 241 overlaps in the result drawing 243.

[0077] In the result drawing 243, the generation unit 121 eliminates the overlapping function summary information 930 of the overlapping camera input 241 using the acquired integrated function summary information 941 and integrated function summary information 942 (step S803).

[0078] In the result drawing 243, the generation unit 121 further assigns sub-IDs for identification to the person detection 242 and skeleton detection 910, whose ID value is "1" (step S804). The sub-ID is assigned by adding an underscore to the existing ID. With this method, the ID value of the person detection 242 is set to "1_1", and the ID value of the skeleton detection 910 is set to "1_2". However, as another method of assigning sub-IDs, a key for the sub-ID may be generated and an identifiable ID may be assigned to the generated sub-ID, and the method of assigning sub-IDs is not limited thereto.

[0079] In the result drawing 243, the generation unit 121 further reads the IDs excluding the sub-IDs of the function summary information for which processing has been performed, and increments the value from the maximum value of "1" to "2", which is the ID of the function summary information 933 of the node. The integrated function summary information 951 thus generated is output in a vector 253 to the execution node 244, which is the subsequent node.

[0080] In image distribution 911, the generation unit 121 reads "1", which is the maximum ID value, from the acquired integrated function summary information 942, and sets the incremented value, "2", as the ID of the function summary information 934 of the node. The generation unit 121 adds the function summary information 934 of the node and the generated ID value "2" to the acquired integrated function summary information 942 to generate integrated function summary information 952, and outputs the integrated function summary information 952 to the execution node 244, which is the subsequent node, via vector 923.

[0081] Here, "type":"multi-process" included in the function summary information 934 of the node is information that commands that the processing of the image distribution 911 is assigned as a separate process and is performed independently from other processing. A detailed processing method for multi-processing will be described later.

[0082] When the generation of the integrated function summary information 951 in the result drawing 243 and the generation of the integrated function summary information 952 in the image distribution 911 are completed, the process of setting the ID value to "2" exists in the result drawing 243 and the image distribution 911.

[0083] In addition, since the execution node 244, which has two inputs, receives the integrated function summary information 951 and the integrated function summary information 952, the execution node 244 has overlapping function summary information 930 output by the camera input 241 and function summary information 932 output by the skeleton detection 910.

[0084] In the execution node 244, the generation unit 121 eliminates the duplicated function summary information 930 of the duplicated camera input 241 using the acquired integrated function summary information 951 and integrated function summary information 952. There is also a duplicate for the skeleton detection 910 whose ID value excluding the sub-ID is "1", so the generation unit 121 eliminates the duplicate. Here, the function summary information 932 of the skeleton detection 910 with "ID:1" to which no sub-ID is assigned is deleted (step S803).

[0085] Furthermore, the generation unit 121 assigns sub-IDs for identification to the function summary information 933 of the resultant drawing 243 and the function summary information 934 of the image distribution 911, both of which have an ID value of "2." Sub-IDs are assigned by adding an underscore to the existing ID followed by the sub-ID. With this method, the ID of the function summary information 933 of the resultant drawing 243 is assigned as "2_1," and the ID of the function summary information 934 of the image distribution 911 is assigned as "2_2."

[0086] Since the execution node 244 is an end node, the generation unit 121 generates the function summary information 960 that has been processed as configuration information (hereinafter referred to as configuration information 960) and outputs it to the storage unit 122.

[0087] <Example of system execution of configuration information including parallel processing> FIG. 10 is an explanatory diagram illustrating a detailed example of a timing chart when the execution unit 123 of the design execution system 100 according to the second embodiment executes the system.

[0088] Upon receiving the completion of the processing of the camera input 1001, the execution unit 123 starts the processing of the person detection 1002 and the skeleton detection 1003 in parallel. When the parallel processing that starts simultaneously is completed, the next processing starts upon completion of all the parallel processing.

[0089] The execution unit 123 starts result drawing 1004 upon completion of the processing of person detection 1002 and skeleton detection 1003, and when result drawing 1004 ends, starts camera input 1011 for the next frame. Here, for example, if there is no next image to process, the processing of the execution unit 123 ends.

[0090] The execution unit 123 starts image distribution 1005 upon completion of the preceding process of skeleton detection 1003. Since image distribution 1005 is specified as a multi-process process, it starts in synchronization with the completion of skeleton detection 1003, but after it starts, it is executed asynchronously with other processes.

[0091] As described above, according to the second embodiment, by branching the nodes and writing them in parallel at the time of system design, it is possible to parallelize the processing and speed up the processing of the entire system. Also, by specifying multi-processing for the nodes at the time of system design, it is possible to obtain a design execution system 100 that can execute processing asynchronously with other processing. [Example]

[0092] The third embodiment will be described focusing on the differences from the first and second embodiments. The same reference numerals are used to denote the same components as the first and second embodiments, and the description thereof will be omitted. In the third embodiment, the function group creation unit 130 in the server 111 containerizes the function group 420 together with the execution platform 432 and the program execution platform 1101 using a container platform such as Docker.

[0093] Furthermore, the execution unit 123 holds a container execution platform 1102 in order to execute processing that realizes the containerized functions. This enables processing of the function group 420 even if the server 111 and the client 110 have different processing environments, such as an operating system (OS) or a processor 201.

[0094] Furthermore, even if multiple clients 110 have different operating systems, processors 201, etc., by containerizing each function of the function group 420 provided by the server 111, it becomes possible to perform processing using a single function group 420 without having to customize the function group 420 to suit each client 110.

[0095] FIG. 11 is a block diagram illustrating an example of a functional configuration of a design execution system 100 according to the third embodiment.

[0096] The function group creation unit 130 generates a function group 420 required for system design by a user of the design execution system 100 for each processing unit, and containerizes the generated function group 420 together with the execution platform 432 and the program execution platform 1101 using a container platform such as Docker, and outputs the containerized function group 1120 to the storage unit 122.

[0097] The generating unit 121 generates configuration information 1103 from the designed system information and outputs it to the holding unit 122 .

[0098] The storage unit 122 outputs the acquired containerized functions 1120 and configuration information 1103 to the execution unit 123 .

[0099] The execution unit 123 has a containerization function group 1120, a container execution platform 1102, and configuration information 1103. The execution unit 123 places the containerization function group 1120 acquired from the storage unit 122 in an upper layer of the container execution platform 1102, and places the configuration information 1103 in a lower layer of the container execution platform 1102.

[0100] The containerized function group 1120 includes a function group 420, an execution platform 432, and a program execution platform 1101. The program execution platform 1101 for executing the function group 420 and the execution platform 432 is, for example, a platform for executing functions written in a programming language such as Python or C++, or a guest OS platform required when it is desired to execute functions on an OS different from the OS implemented on the client 110.

[0101] This makes it possible to execute the containerized function group 1120 without being affected by the hardware or software environment of the client 110. When the containerized function group 1120 is changed, the function group creation unit 130 reflects the changes and deploys a new containerized function group 1120.

[0102] The container execution platform 1102 absorbs the difference between the environment of the client 110 and the environment of the containerized function group 1120, and has a function to execute the containerized function group 1120 regardless of the environment of the client 110. The configuration information 1103 is generated in the same manner as in the first and second embodiments, and is arranged in the hierarchy of the container execution platform 1102.

[0103] The execution unit 123 selects the configuration described in the configuration information 1103 from the function group 420 in the containerized function group 1120 acquired from the storage unit 122, and instantiates each function. Then, the execution unit 123 executes processing using the program execution platform 1101 and the container execution platform 1102 via the execution platform 432. At this time, the execution unit 123 selects the processing device 203 to perform processing according to the acceleration function, and executes the processing on the processing device 203 via the control unit 124. This speeds up the processing.

[0104] When the server 111 outputs the containerized function set 1120 to multiple clients 110, it may be necessary to perform processing appropriate for each client 110, such as connecting the containerized function set 1120 to a port specific to each client 110 via the container execution platform 1102. In such cases, the storage unit 122 may automatically assign an appropriate process when connecting the containerized function set 1120, or the design unit 120 may set a value specific to the client 110.

[0105] As described above, according to the third embodiment, the function group creation unit 130 in the server 111 generates the function group 420 by containerizing it together with the execution platform 432 and the program execution platform using a container platform such as Docker. In addition, the execution unit 123 holds the container execution platform 1102 to execute the containerized function group 420.

[0106] This makes it possible to process the function group 420 even if the server 111 and the client 110 have different processing environments, such as an operating system (OS) or processor 201. Furthermore, by containerizing the function group 420 provided by the server 111, even if multiple clients 110 have different OSs or processors 201, it becomes possible to perform processing with a single function group 420 without customizing the function group 420 for each client 110. [Example]

[0107] The fourth embodiment will be described focusing on the differences from the first, second, and third embodiments. The same reference numerals are used to designate the same parts as the first, second, and third embodiments, and the description thereof will be omitted. The fourth embodiment will describe a detailed processing method when parallel processing or multi-process processing is included in the configuration information 320 to be generated.

[0108] However, unlike Example 2, each node cannot synchronize with the node even when it receives multiple inputs from the previous node, and a system is assumed in which, upon receiving any input from the previous node, the node acquires the function summary information of the previous node, combines it with the function summary information of the node in question, and starts processing to output it to the subsequent node. Hereinafter, this system will be referred to as an asynchronous system.

[0109] In the first to third embodiments, each node cannot synchronize with the previous node even if it receives multiple inputs. Therefore, if the system being designed includes parallel processing or multi-process processing, multiple execution paths are generated, and an execution node is called for each execution path. For example, in the example shown in Figure 9, three execution paths are generated, so the execution node 244 is called three times.

[0110] To address this issue, a synchronization node that aggregates multiple execution paths can be inserted before the execution node to consolidate the execution node's invocations from multiple execution paths into a single call. However, the connection settings for each execution path to the synchronization node must be manually performed by the user. Therefore, if a user makes a manual connection setting error, only the processing of some of the execution paths will reach the synchronization node, and the execution node will not be invoked. This requires the user to go through the trouble of checking whether the number of execution paths to be synchronized is correct in the connection setting properties screen.

[0111] Therefore, in the fourth embodiment, a system is provided that enables synchronous processing without a synchronous node for parallel processing or multi-process processing in such an asynchronous system.

[0112] <Example of generating configuration information for a system configuration including parallel processing> 12 is an explanatory diagram illustrating a detailed processing example of configuration information generation in the generation unit 121 of the design execution system 100 according to the fourth embodiment. In FIG. 12, a camera input 1200, an inference A 1201, an inference B 1202, an inference C 1204, a result drawing 1205, and an execution node 1206 exist as a node group.

[0113] The output of Camera Input 1200 is connected to the input of Inference A 1201 by vector 1210, and to the input of Inference B 1202 by vector 1211. The output of Inference A 1201 is connected to the input of Result Drawing 1205 by vector 1212.

[0114] The output of Inference B 1202 is connected to the input of Inference A 1203 by vector 1213 and to the input of Inference C 1204 by vector 1214. The output of Inference A 1203 is connected to the input of Draw Result 1205 by vector 1215. The output of Inference C 1204 is connected to the input of Draw Result 1205 by vector 1216. The output of Draw Result 1205 is connected to the input of Execute Node 1206 by vector 1217.

[0115] The camera input 1200 is the starting node. The generation unit 121 sets the ID value to "AAA" and acquires "2" as the number of outputs, which is the number of nodes connected in the subsequent stage. The ID "AAA" set by the generation unit 121 here is an ID different from the IDs given to any other nodes arranged in the system information (information specifying the execution order of processes realized by functions corresponding to nodes adopted in the design of the system executed by the client 110).

[0116] This means that not only are different IDs assigned to nodes with different functions, but also that different IDs are assigned when the same function is placed as a different node. For example, inference A1201 and inference A1203 have the same function, but are placed as different nodes, so different IDs are assigned to each.

[0117] The generation unit 121 sets the set ID "AAA" for the camera input 1200, the number of outputs "2", and "camera input" indicating the function of the node in question as function summary information 1220. The generation unit 121 outputs the function summary information 1220 from the camera input 1200 to the subsequent inference A 1201. The generation unit 121 also outputs the function summary information 1221 and the function summary information 1222 from the camera input 1200 to inference B 1202. In this case, the output of the camera input 1200 is branched to inference A 1201 as well as inference B 1202, so the number of outputs of the function summary information 1220 to 1222 is "2".

[0118] The function summary information 1220 to 1222 is the same information as the function summary information. The reason for describing them separately is to explain that the function summary information is output in parts according to the number of branches in the asynchronous system. Details will be described later in the operation of each node.

[0119] Here, we will explain the behavior of each of the subsequent nodes 1202 to 1206 that are not the starting node and are other than the camera input 1200. Each of the nodes 1202 to 1206 sets a unique ID for that node, and outputs integrated function summary information that includes the function summary information acquired from the previous node and the function summary information of that node to the subsequent node.

[0120] Here, the function summary information includes the number of outputs, but if the output of the subsequent node does not branch, the number of outputs obtained from the previous node is inherited and output to the subsequent node. For example, if the number of outputs from a certain node's previous node is "2", but the output of the subsequent node does not branch, that is, there is only one subsequent node. In this case, the number of outputs "2" is inherited by the subsequent node. This is because the number of outputs "2" already includes "1" for the output to the subsequent node (the path from the start node to the end node via that).

[0121] On the other hand, if the output of the subsequent node is branched, the number of outputs is updated according to the number of acquired outputs and the number of branches of the subsequent node, and output to the subsequent node. For example, if the number of outputs acquired from the previous node is "2" and the output of the subsequent node is branched into two, that is, if there are two subsequent nodes, the number of outputs "2" is added by "1" for the output of the newly branched path, resulting in the number of outputs "3". This number of outputs "3" is output to each of the two subsequent nodes.

[0122] Each node cannot synchronize with itself even when it receives multiple inputs from the previous node, and when it receives any input from the previous node, it acquires the function summary information of the previous node and starts the process of outputting it to the subsequent node together with the function summary information of the node in question.

[0123] The generation unit 121 sets the ID value to "BBB" in inference A1201 and acquires function summary information 1220 from camera input 1200, which is the previous node. Furthermore, since the only node connected downstream of inference A1201 is result drawing 1205 and the output of inference A1201 does not branch, the number of outputs acquired from camera input 1200, "2", is inherited, and the number of outputs of inference A1201 becomes "2". In other words, the number of outputs "2" is because "1" in the vector 1212 indicating the output to result drawing 1205, which is the downstream node of inference A1201 (the path from camera input 1200 to the end node via camera input 1200) has already been counted in camera input 1200.

[0124] In inference A1201, the ID "BBB", "inference A", and the number of outputs "2" are set as function summary information 1223. The generation unit 121 outputs integrated summary information including the function summary information 1223 of inference A1201 and the function summary information 1220 acquired from the camera input 1200 to the result drawing 1205 at the subsequent stage.

[0125] The generation unit 121 sets the ID value to "CCC" in inference B 1202 and acquires function summary information 1221 from the preceding node, camera input 1200. Because the node following inference B 1202 branches into inference A 1203 and inference C 1204, the number of outputs acquired from camera input 1200, "2," plus the branching, becomes "3."

[0126] In other words, the number of outputs "2" already counts "1" in the vector 1213 indicating the output to inference A1203, which is one of the subsequent nodes of inference B1202 (the path from camera input 1200 to the end node via inference B1202), but does not count "1" in the vector 1214 indicating the output to inference C1204, which is the other subsequent node of inference B1202 (the path from camera input 1200 to the end node via inference B1202).

[0127] In inference B1202, the ID "CCC", "inference B", and the number of outputs "3" are set as function summary information 1224. The generation unit 121 outputs integrated summary information including the function summary information 1224 of inference B1202 and the function summary information 1221 acquired from the camera input 1200 to the subsequent inferences A1203 and A1204, respectively.

[0128] The generation unit 121 sets the ID value to "BBB'" in inference A 1203, and acquires function summary information 1221, 1224 from inference B 1202, which is the previous node. Since the only node connected to the next node of inference A 1203 is result drawing 1205, and the output of inference A 1203 does not branch, the number of outputs "3" acquired from inference B 1202 is inherited, and the number of outputs of inference A 1203 becomes "3".

[0129] In inference A1203, the ID "BBB'", "inference A", and the number of outputs "3" are set as function summary information 1229. The generation unit 121 outputs integrated summary information including the function summary information 1229 of inference A1203 and the function summary information 1221, 1224 acquired from inference B1202 to the result drawing 1205 at the subsequent stage.

[0130] The generation unit 121 sets the ID value of inference C 1204 to "EEE" and acquires function summary information from the previous node, inference B 1202. Since the only node connected to the next node of inference C 1204 is result drawing 1205 and the output of inference C 1204 does not branch, the number of outputs acquired from inference B 1202, "3", is inherited, and the number of outputs of inference C 1204 becomes "3".

[0131] In inference C 1204, the ID “EEE”, “inference C”, and the number of outputs “3” are set as function summary information 1230. The generation unit 121 outputs integrated summary information including the function summary information 1230 of inference C 1204 and the function summary information 1221, 1224 acquired from inference B 1202 to the result drawing 1205 at the subsequent stage.

[0132] The result drawing 1205 has three preceding nodes: inference A 1201, inference A 1203, and inference C 1204. In the fourth embodiment, an asynchronous system is assumed, and therefore processing of the result drawing 1205 begins when the output of any one of the preceding nodes 1201, 1203, and 1204 is input to the result drawing 1205. In other words, the result drawing 1205 receives the outputs from the three preceding nodes 1201, 1203, and 1204 and operates three times.

[0133] The operation of result drawing 1205 when it receives output from inference A 1201 will be described. In result drawing 1205, the generation unit 121 sets the ID value to "DDD" and acquires function summary information 1220, 1223 from inference A 1201, which is the previous node. Since the only node connected to the subsequent stage of result drawing 1205 is execution node 1206 and the output of result drawing 1205 does not branch, the number of outputs acquired from inference A 1201, "2", is inherited, and the number of outputs of result drawing 1205 becomes "2".

[0134] In the result drawing 1205, the ID “DDD”, “result drawing”, and the number of outputs “2” are set as function summary information 1226. The generation unit 121 outputs integrated summary information 1231 including the function summary information 1226 of the result drawing 1205 and the function summary information 1220, 1223 acquired from the inference A 1201 to the subsequent execution node 1206.

[0135] The operation of result drawing 1205 when it receives output from inference A 1203 will be described. The generation unit 121 sets the ID value to "DDD" in result drawing 1205, and acquires function summary information 1221, 1224, and 1229 from inference A 1203, which is the node preceding it. Since the only node connected to the subsequent stage of result drawing 1205 is execution node 1206 and the output of result drawing 1205 does not branch, the number of outputs "3" acquired from inference A 1203 is inherited, and the number of outputs of result drawing 1205 becomes "3".

[0136] In the result drawing 1205, the ID “DDD”, “result drawing”, and the number of outputs “3” are set as the function summary information 1227. The generation unit 121 outputs the integrated summary information 1232 including the function summary information 1227 and the function summary information 1221, 1224, and 1229 acquired from the inference A 1203 to the subsequent execution node 1206.

[0137] The operation of result drawing 1205 when it receives output from inference C 1204 will be described. The generation unit 121 sets the ID value to "DDD" in result drawing 1205, and acquires function summary information 1222, 1225, and 1230 from inference C 1204, which is the previous node. Since the only node connected to the next stage of result drawing 1205 is execution node 1206 and the output of result drawing 1205 does not branch, the number of outputs acquired from inference C 1204, "3", is inherited, and the number of outputs of result drawing 1205 becomes "3".

[0138] In the result drawing 1205, the ID “DDD”, “result drawing”, and the number of outputs “3” are set as the function summary information 1228. The generation unit 121 outputs the integrated summary information 1233 including the function summary information 1228 and the function summary information 1222, 1225, and 1230 acquired from the inference C 1204 to the subsequent execution node 1206.

[0139] In this asynchronous system, there is a possibility that some of the functional information summary information may be input in duplicate at the node to be aggregated. Note that the order of output to the result drawing 1205 of the three nodes, inference A 1201, inference A 1203, and inference C 1204, is not limited, and the order may be reversed.

[0140] The previous node of execution node 1206 is result drawing 1205, but since the number of inputs to result drawing 1205 is "3", result drawing 1205 is executed three times. Since result drawing 1205 outputs each execution result to execution node 1206, execution node 1206 is also executed three times.

[0141] The execution node 1206 asynchronously acquires the integrated summary information 1231, the integrated summary information 1232, and the integrated summary information 1233 in no particular order. The three pieces of integrated summary information 1231 to 1233 contain duplicated pieces of the same functional summary information, such as the functional summary information 1220 to 1222 relating to the camera input 1200. The following describes in detail the processing of the execution node 1206 for performing synchronization processing after eliminating duplicates of the same information for the integrated summary information that is output in multiple parts by the asynchronous system.

[0142] The execution node 1206 includes a function summary information acquisition 1240 , an output branch number determination 1241 , a duplication integration 1242 , a node connection arrangement 1243 , and a configuration information generation 1244 .

[0143] The function summary information acquisition 1240 acquires, in no particular order, the integrated summary information 1231 , the integrated summary information 1232 , and the integrated summary information 1233 output from the previous result drawing 1205 , and outputs them to the output branch number decision 1241 .

[0144] The output branch number decision 1241 counts the number of acquired pieces of integrated summary information for each of the integrated summary information 1231 to 1233 acquired in no particular order, and compares the counted number of pieces of integrated summary information with the number of outputs held in each of the integrated summary information 1231 to 1233.

[0145] The output branch number decision 1241 accumulates the acquired integrated summary information until the counted number of integrated summary information matches the output number. When the counted number of integrated summary information matches the output number, the output branch number decision 1241 outputs the accumulated integrated summary information to the overlap integration 1242.

[0146] The operation of the output branch number decision 1241 will be explained using a specific example. First, when the output branch number decision 1241 acquires the integrated summary information 1231, the output branch number decision 1241 counts the number of acquired integrated summary information (initial value is "0") to "1", confirms that the number of outputs of the integrated summary information 1231 is "2", and stores the integrated summary information 1231.

[0147] Next, when the output branch number determination 1241 acquires the integrated summary information 1232, the output branch number determination 1241 counts the number of acquired integrated summary information pieces to "2", confirms that the output number of the integrated summary information 1232 is "3", and stores the integrated summary information 1232.

[0148] Finally, when the output branch number decision 1241 acquires the integrated summary information 1233, the output branch number decision 1241 counts the number of acquired integrated summary information pieces to "3", confirms that the number of outputs of the integrated summary information 1233 is "3", and stores the integrated summary information 1233. Since the counted number of acquisitions "3" matches the number of outputs "3", the output branch number decision 1241 outputs the integrated summary information 1231, integrated summary information 1232, and integrated summary information 1233 that have been stored up to that point to the overlap integration 1242.

[0149] The overlap integration 1242 integrates overlapping node functions using IDs included in all of the acquired integration summary information, and outputs the integrated integration summary information to the node connection arrangement 1243 as overlap integration summary information.

[0150] The node connection sorting 1243 checks the connection relationship of each node for the overlap integration summary information obtained from the overlap integration 1242, and for nodes with multiple previous nodes, adds synchronization processing information such that processing will begin once output to the node from all previous nodes is complete, and outputs this as synchronized overlap integration summary information to the configuration information generation 1244. The node connection relationship refers to the relationship in which nodes are connected to each other by vectors. Here, the processing of the overlap integration 1242 and the node connection sorting 1243 will be specifically explained using figures.

[0151] 13 is an explanatory diagram illustrating a specific example of the processing of the overlap integration 1242 and the node connection arrangement 1243 according to the fourth embodiment. Fig. 13 illustrates changes in the integration summary information when the overlap integration 1242 and the node connection arrangement 1243 in the execution node 1206 generate a synchronized overlap integration summary information group 1302 from the acquired integration summary information group 1300.

[0152] 12. Note that there is a connection relationship between adjacent blocks on the left and right in each of the integrated summary information 1231 to 1233. Taking the integrated summary information 1231 as an example, there is a node connection relationship between the functional summary information 1220 and 1223, and there is a node connection relationship between the functional summary information 1223 and 1226.

[0153] The overlap integration 1242 determines that function summary information having the same ID is the same node and integrates it. For example, the same ID "AAA" is assigned to the function summary information 1220 to 1222 of the camera input 1200, so the overlap integration 1242 determines that they are the same node, integrates them, and outputs them as overlap integration summary information 1311.

[0154] On the other hand, although the function summary information 1223 of inference A1201 and the function summary information 1229 of inference A1203 are the same function (inference A), the IDs are different ("BBB" and "BBB'"), so the overlap integration 1242 determines them as different nodes and does not integrate them.

[0155] In addition, since the function summary information 1224 of inference B 1202 and the function summary information 1225 of inference B 1202 are assigned the same ID "CCC," the overlap integration 1242 determines that they are the same node and integrates them, outputting them as overlap integrated summary information 1312. Furthermore, since the function summary information 1226 to 1228 of resultant drawing 1205 are assigned the same ID "DDD," the overlap integration 1242 determines that they are the same node and integrates them, outputting them as overlap integrated summary information 1313. Once integration of all overlapping function summary information has been completed for the acquired integrated summary information, the overlap integrated summary information 1311 to 1313 and the function summary information 1223, 1229, and 1230 are output to the node connection sorting 1243 as an overlap compound summary information group 1301.

[0156] The node connection sorter 1243 checks the connection relationships of each node for the overlap integration summary information group 1301 acquired from the overlap integration 1242, and assigns synchronization processing information to nodes with multiple previous nodes so that processing will begin once output from all previous nodes to the node is complete, and outputs this as synchronized overlap integration summary information to the configuration information generator 1244.

[0157] Specifically, for example, in overlapping compound summary information 1311 to 1313 and function summary information 1223, 1229, and 1230 in overlapping compound summary information group 1301, the connection relationships between nodes in horizontally adjacent blocks are inherited from integrated summary information group 1300.

[0158] For example, in the case of overlapping compound summary information 1311, the node connection relationship with function summary information 1223 inherits the node connection relationship between function summary information 1220 and 1223. Furthermore, the node connection relationship with overlapping compound summary information 1312 inherits the node connection relationship between function summary information 1221 and 1224 and the node connection relationship between function summary information 1222 and 1225.

[0159] Meanwhile, the node connection arrangement 1243 checks whether or not there is a node connection relationship between the function summary information 1223 and the overlapping compound summary information 1313. The function summary information 1223 has a node connection relationship with the function summary information 1226. The function summary information 1226 is integrated into the overlapping integrated summary information 1313. Therefore, the node connection arrangement 1243 determines that there is a node connection relationship between the function summary information 1223 and the overlapping compound summary information 1313. Therefore, the node connection arrangement 1243 connects the function summary information 1223 and the overlapping compound summary information 1313. In this way, the node connection arrangement 1243 checks the node connection relationship in the overlapping integrated summary information group 1301.

[0160] Furthermore, the previous node of function summary information 1226 of result drawing 1205 is function summary information 1223 of inference A 1201. The previous node of function summary information 1227 of result drawing 1205 is function summary information 1229 of inference A 1203. The previous node of function summary information 1228 of result drawing 1205 is function summary information 1230 of inference C 1204. The same ID "DDD" is assigned to function summary information 1226 to 1228 of result drawing 1205, and therefore they are the same node. Therefore, it is determined that result drawing 1205 has three previous nodes (function summary information 1223 of inference A 1201, function summary information 1229 of inference A 1203, and function summary information 1230 of inference C 1204).

[0161] After obtaining the output of the three previous nodes (functional overview information 1223 of inference A 1201, functional overview information 1229 of inference A 1203, and functional overview information 1230 of inference C 1204) for the result drawing 1205, the node connection sorter 1243 assigns synchronization processing information to the duplicate composite overview information group 1301 to create a synchronized duplicate elimination integrated overview information group 1302 and outputs it to the configuration information generator 1244 in order to start processing.

[0162] The synchronization processing information is information that starts processing once output to all of the multiple preceding nodes has been completed. In this example, there are three nodes (function summary information 1223 of inference A 1201, function summary information 1229 of inference A 1203, and function summary information 1230 of inference C 1204) preceding the result drawing 1205, so synchronization processing information 1320 is added to the overlap integration summary information 1313 that indicates the result drawing 1205. This synchronization processing information 1320 has the function of causing the result drawing 1205 to start processing once it has received output from all of the function summary information 1223, 1229, and 1230.

[0163] As a result, the result drawing 1826 can start processing and synchronize the processing after waiting for the outputs of inference A 1823, inference A 1829, and inference C 1830. Note that although the start node does not have a previous node, for example, since the overlapping compound summary information 1311 is camera input 1200, it is possible to add synchronization processing information such as starting camera input 1200 when an interrupt occurs in all of the multiple nodes other than camera input 1200.

[0164] In this way, the node connection arrangement 1243 completes the node connection confirmation and generates a synchronous overlap integrated summary information group 1302 to which the synchronous processing information 1320 has been added, and outputs it to the configuration information generation 1244 .

[0165] The configuration information generation unit 1244 generates the synchronous overlap integration summary information group 1302 acquired from the node connection arrangement unit 1243 as the configuration information 320 .

[0166] <Example of processing procedure of generation unit 121> 14 is a flowchart showing a detailed generation process procedure by the generation unit 121 of the design execution system 100 according to the fourth embodiment. The generation unit 121 determines whether the node to be processed is a start node (step S1400). If there is no previous node, it is determined to be a start node. If it is not a start node (step S1400: No), the generation unit 121 acquires function summary information from the previous node (step S1401) and proceeds to the process of step S1402. If it is determined in step S1400 that the node is a start node (step S1400: Yes), the process proceeds to the process of step S1402.

[0167] In step S1402, the generation unit 121 sets a node-specific ID for the node, and the process proceeds to step S1403.

[0168] In step S1402, the generation unit 121 obtains the number of outputs from the number of nodes connected to the subsequent stage, and proceeds to the processing of step S1404.

[0169] In step S1404, the generation unit 121 outputs to the subsequent node integrated function summary information that includes the function summary information acquired from the previous node, a unique ID, the number of outputs, and the function summary information of the node, and proceeds to processing in step S1405.

[0170] In step S1405, the generation unit 121 checks whether the node to be processed is an end node (step S1405). Specifically, for example, if the node is an execution node, it is an end node. If it is an end node (step S1405: Yes), the process proceeds to step S1406. If it is not an end node (step S1405: No), the process proceeds to step S1400.

[0171] In step S1406, the generation unit 121 acquires the integrated summary information in the function summary information acquisition 1240, and the process proceeds to step S1407.

[0172] In step S1407, the generation unit 121 counts the number of pieces of integrated summary information obtained in the output branch number judgment 1241 and compares it with the number of outputs held in each piece of integrated summary information.If they match (step S1407: Yes), the generation unit 121 outputs the accumulated integrated summary information to the duplicate integration 1242 and proceeds to step S1408; if they do not match (step S1407: No), the generation unit 121 accumulates the acquired integrated summary information and proceeds to step S1400.

[0173] In step S1408, the generation unit 121 integrates the overlapping functions of the nodes using the IDs contained in all the acquired integrated summary information in the overlap integration 1242, generates the integrated integrated summary information as overlapping integrated summary information, and proceeds to step S1409.

[0174] In step S1409, the generation unit 121 checks the connection relationship of each node from the acquired overlap integration summary information in the node connection sorting 1243, and assigns synchronization processing information to nodes with multiple previous nodes so that processing will begin once output from all previous nodes to the node is complete, thereby generating a synchronized overlap integration summary information group 1301, and proceeds to step S1410.

[0175] In step S1410, the generating unit 121 generates the acquired synchronized overlap integrated summary information group 1301 as the configuration information 320 in the configuration information generation 1244, and the processing of the generating unit 121 ends.

[0176] As described above, according to the fourth embodiment, in the asynchronous system, each node in the design unit 120 cannot synchronize with the node even if it receives multiple inputs from the previous node. Furthermore, when the asynchronous system receives any input from the previous node, it acquires the function summary information of the previous node, combines the function summary information of the current node, and starts a process of outputting the combined information to the subsequent node. Even in such an asynchronous system, it is possible to execute synchronous processing as a parallel system designed by branching. [Example]

[0177] The fifth embodiment will be described, focusing on the differences from the first to fourth embodiments. The same reference numerals are used to designate the same parts as the first to fourth embodiments, and the description thereof will be omitted. The fifth embodiment will describe a detailed processing method in an asynchronous system when the generated configuration information 320 includes parallel processing or multi-process processing.

[0178] However, unlike the fourth embodiment, a system is assumed in which nodes branched for parallelization are once consolidated into one node, and the parallelization is consolidated, and then parallelization is performed again. The fifth embodiment provides a system that enables synchronous processing for parallel processing or multi-process processing in such a system.

[0179] <Example of generating configuration information for a system configuration including parallel processing> 15 is an explanatory diagram illustrating a detailed processing example of configuration information generation in the generation unit 121 of the design execution system 100 according to the fifth embodiment. In FIG. 15, a camera input 1500, an inference A 1501, an inference B 1502, an inference C 1504, a result drawing 1505, an image distribution 1506, and an execution node 1507 exist as a node group.

[0180] The output of camera input 1500 is connected to the input of inference A 1501 by vector 1510 and to the input of inference B 1502 by vector 1511. The output of inference A 1501 is connected to the input of inference C 1504 by vector 1512.

[0181] The output of Inference B 1502 is connected to the input of Inference C 1504 by vector 1513. The output of Inference C 1504 is connected to the input of Render Results 1505 by vector 1510, and the output of Inference C 1504 is connected to the input of Image Delivery 1506 by vector 1511.

[0182] The output of the Render Results 1505 is connected to the input of the Execute Node 1507 by a vector 1517. The output of the Image Delivery 1506 is connected to the input of the Execute Node 1507 by a vector 1513.

[0183] The fifth embodiment differs from the fourth embodiment in that a flag signal is added to the branched node after parallelization. The flag signal is a signal for distinguishing the branch.

[0184] The generation unit 121 sets the ID value to "AAA" in the camera input 1500, and sets the set ID "AAA" as the function summary information 1520. The generation unit 121 outputs the function summary information 1520 from the camera input 1500 to the subsequent inference A 1501 and inference B 1502. In this case, the output of the camera input 1500 is branched to inference A 1501 as well as inference B 1502, so the number of outputs is "2".

[0185] Furthermore, in the fifth embodiment, the generation unit 121 outputs a flag signal for a node that has multiple nodes in the subsequent stage and branches. Specifically, the generation unit 121 outputs a flag signal “1” and a flag signal “2” together with function summary information 1520 to inference A 1501 and inference B 1502 in the subsequent stage, respectively, for the camera input 1500.

[0186] In addition, when there are multiple subsequent nodes and function summary information is output to the subsequent nodes, the generating unit 121 may assign individual numbers to the subsequent nodes even if the information is the same, as in Example 4. However, in Example 5, in order to emphasize the difference from Example 4, the description will be given assuming that the same numbers are assigned to some branches and function summary information is output to the subsequent nodes.

[0187] For this reason, the functional summary information of camera input 1500 may be assigned different numbers as in Example 4 and output to inference A 1501 and inference B 1502, but here the generation unit 121 outputs it to inference A 1501 and inference B 1502 as functional summary information 1520 with the same number.

[0188] The generation unit 121 sets the ID value to "BBB" in inference A1501, and acquires function summary information 1520 and a flag signal "1" from the preceding node, camera input 1500. Since the only node connected to the following stage of inference A1501 is inference C1504 and the output of inference A1501 does not branch, the number of outputs "2" acquired from camera input 1500 is inherited, and the number of outputs of inference A1501 becomes "2".

[0189] Inference A 1501, the ID “BBB”, “Inference A”, the number of outputs “2”, and the flag signal “1” are set as function summary information 1521. The generation unit 121 outputs integrated summary information including the function summary information 1521 and function summary information 1520 acquired from the camera input 1500 to the subsequent inference C 1504.

[0190] The generation unit 121 sets the ID value of inference B1502 to "CCC" and acquires function summary information 1520 from the preceding node, camera input 1500. Since the only node connected to the following stage of inference B1502 is inference C1504 and the output of inference B1502 does not branch, the number of outputs acquired from camera input 1500, which is "2", is inherited, and the number of outputs of inference B1502 becomes "2".

[0191] Inference B 1502, the ID “CCC”, “inference B”, the number of outputs “2”, and the flag signal “2” are set as function summary information 1522. The generation unit 121 outputs integrated summary information including the function summary information 1522 and the function summary information 1520 acquired from the camera input 1500 to the subsequent inference C 1504.

[0192] The generation unit 121 sets the ID value to "DDD" in inference C1504, and acquires function summary information from the preceding nodes, inference A1501 and inference A1502, in no particular order. As the operation of inference C1504, first, the operation when function summary information 1521 is received from inference A1501 will be described.

[0193] In inference C 1504, the generation unit 121 sets the ID value to "DDD" and acquires function summary information 1520, 1521 from inference A 1501, which is the previous node. From inference C 1504, the generation unit 121 acquires the number of outputs "2" from the number of nodes connected in the subsequent stage (result rendering 1505, image distribution 1506).

[0194] However, the generation unit 121 detects the presence of the flag signal "1" through inference C1504, and detects that there is already a branch at the previous node, including the previous node.The generation unit 121 then sets the number of outputs to "4" in order to consider all combinations of branches that take into account the branches at the previous and previous nodes and the branches at the subsequent nodes.That is, the number of outputs is set to "4" by multiplying the number of flag signals "2" by the number of outputs "2."

[0195] Furthermore, since the node subsequent to inference C 1504 branches, the generation unit 121 prepares flag signals "3" and "4." Then, in inference C 1504, the IDs "DDD" and "inference C" are set as function summary information 1523. The generation unit 121 combines the function summary information 1523 with the function summary information 1520 and 1521 acquired from the previous node to output the integrated summary information to the subsequent result drawing 1505 and image distribution 1506. Furthermore, the generation unit 121 outputs the flag signals "3" and "4" prepared by inference C 1504 to the result drawing 1505 and image distribution 1506, respectively.

[0196] Secondly, the operation of inference C 1504 when function summary information 1522 is received from inference B 1502 will be described.

[0197] In inference C 1504, the generation unit 121 sets the ID value to "DDD" and acquires function summary information 1520, 1522 from inference B 1502, which is the previous node. From inference C 1504, the generation unit 121 acquires the number of outputs "2" from the number of nodes (result rendering 1505, image distribution 1506) connected to the subsequent node.

[0198] However, the generation unit 121 detects the presence of the flag signal "1" by inference C1504, and detects that there is already a branch at the previous node, including the previous node. Then, the generation unit 121 sets the number of outputs to "4" in order to consider all combinations of branches that take into account the branches at the previous and previous nodes and the branches at the subsequent nodes.

[0199] Furthermore, since the node subsequent to inference C 1504 branches, the generation unit 121 prepares flag signals "3" and "4." Then, in inference C 1504, the IDs "DDD" and "inference C" are set as function summary information 1524. The generation unit 121 combines the function summary information 1524 with the function summary information 1520 and 1522 acquired from the previous node to output the integrated summary information to the subsequent result drawing 1505 and image distribution 1506. Furthermore, the generation unit 121 outputs the flag signals "3" and "4" prepared by inference C 1504 to the result drawing 1505 and image distribution 1506, respectively.

[0200] Next, we will explain the processing of result drawing 1505 and image distribution 1506. Inference C 1504 is located before result drawing 1505 and image distribution 1506. The processing of result drawing 1505 and image distribution 1506 will be explained by first focusing only on the processing of inference C 1504, result drawing 1505, and image distribution 1506, which are the previous nodes.

[0201] The generation unit 121 sets the ID value to "EEE" in the result drawing 1505, and acquires the function summary information 1520, 1521, and 1523 and the flag signal "3" from the previous node, inference C 1504. Since the only node connected to the next stage of the result drawing 1505 is the execution node 1507 and the output of the result drawing 1505 does not branch, the number of outputs "4" acquired from inference C 1504 is inherited, and the number of outputs of the result drawing 1505 becomes "4".

[0202] In result drawing 1505, the ID “EEE”, “result drawing”, the number of outputs “4”, and the flag signal “3” are set as function summary information 1525. The generation unit 121 outputs integrated summary information including the function summary information 1525 and the function summary information 1520, 1521, and 1523 acquired from inference C 1504 to the subsequent execution node 1507.

[0203] The generation unit 121 sets the ID value to "FFF" in image distribution 1506, and acquires function summary information 1520, 1521, 1523 and a flag signal "4" from inference C 1504, which is the previous node. Since the only node connected to the next stage of image distribution 1506 is execution node 1507 and the output of image distribution 1506 does not branch, the number of outputs "4" acquired from inference C 1504 is inherited, and the number of outputs of image distribution 1506 becomes "4".

[0204] In image distribution 1506, the ID "FFF", "image distribution", the number of outputs "4", and the flag signal "4" are set as function summary information 1526. The generation unit 121 outputs integrated summary information including the function summary information 1526 and the function summary information 1520, 1521, and 1523 acquired from inference C 1504 to the subsequent execution node 1507.

[0205] Regarding the processing of result drawing 1505 and image distribution 1506, secondly, the processing including branching of nodes before inference C 1504, which is the previous node, will be explained.

[0206] If we focus only on the processing of inference C 1504, which is the previous node of result drawing 1505, result drawing 1505, and image distribution 1506, as described above, result drawing 1505 and image distribution 1506 each obtain the output from inference C 1504 and perform processing.

[0207] However, at a node before the previous node (inference C1504), there is a branch in the path passing through vector 1510 and vector 1511, so in inference C1504, two processes are initiated in response to the outputs from inference A1501 and inference B1502. In other words, inference C1504 outputs function summary information 1523 that follows the path of inference A1501 and function summary information 1524 that follows the path of inference B1502 to result drawing 1505 and image distribution 1506, respectively.

[0208] Therefore, the execution node 1507 acquires integrated summary information including the function summary information 1520, 1521, 1523, and 1525, integrated summary information including the function summary information 1520, 1521, 1523, and 1526, integrated summary information including the function summary information 1520, 1522, 1524, and 1527, and integrated summary information including the function summary information 1520, 1522, 1524, and 1528. In other words, the execution node 1507 is started four times.

[0209] The detailed processing of the execution node 1507 will be explained below, focusing only on the differences from the fourth embodiment. The execution node 1507 differs from the execution node 1206 of the fourth embodiment in that a flag signal check 1540 is added.

[0210] If there is a branch before the previous node, it is necessary to generate configuration information taking all branches into consideration. For this reason, a flag signal is added to the node after the branch to check whether function summary information including all nodes is included in the execution node 1507. The flag signal check 1540 checks whether all added flag signals are included in the acquired integrated summary information, and accumulates the acquired integrated summary information until all flag signals have been confirmed. When all added flag signals have been confirmed, the flag signal check 1540 outputs the accumulated integrated summary information to the overlap integration 1242.

[0211] <Example of processing procedure of generation unit 121> Fig. 16 is a flowchart illustrating a detailed procedure of a generation process by the generation unit 121 of the design execution system 100 according to the fifth embodiment. Fig. 16 illustrates an example of generating configuration information using ID assignment, output number setting, duplication integration, flag confirmation, and the like in the generation unit 121 of the design execution system 100 according to the fifth embodiment.

[0212] Regarding the detailed processing procedure for generating configuration information in the generation unit 121 of the design execution system 100, only the differences from the fourth embodiment will be described. The detailed processing procedure for generating configuration information differs from the fourth embodiment in that steps S1600, S1601, and S1602 are added to the flowchart in FIG.

[0213] In step S1600, the generation unit 121 determines whether the output of the node to be processed is branched (step S1600). If the output of the node is branched (step S1600: Yes), the process proceeds to step S1601, and if the output of the node is not branched (step S1600: No), the process proceeds to step S1405.

[0214] In step S1601, the generating unit 121 outputs a flag signal to the subsequent node, and the subsequent node sets the acquired flag signal in the function summary information, and the process proceeds to step S1405.

[0215] In step S1602, the generation unit 121 checks whether all added flag signals are included in the acquired integrated summary information in flag signal check 1440. If all added flag signals have been checked (step S1602: Yes), the generation unit 121 outputs the accumulated integrated summary information to overlap integration 1242 and proceeds to step S1408. If all added flag signals cannot be checked (step S1602: No), the generation unit 121 accumulates the acquired integrated summary information and proceeds to step S1400.

[0216] In this way, according to the fifth embodiment, it is possible to obtain a design execution system 100 that can perform synchronous processing for parallel processing or multi-process processing in an asynchronous system in which nodes that branch out for parallelization are once consolidated into one node, and the parallelization is consolidated and then parallelized again. [Example]

[0217] The sixth embodiment will be described focusing on the differences from the first to fifth embodiments. The same reference numerals are used to designate the same parts as the first to fifth embodiments, and the description thereof will be omitted. The sixth embodiment provides a system that enables the execution of prioritized process processing in a system in which parallel processing and multi-process processing are included in the generated configuration information 320.

[0218] <Example of generating configuration information for a system configuration including parallel processing> 17 is an explanatory diagram showing a detailed processing example of configuration information generation in the generation unit 121 of the design execution system 100 according to Example 6. Example 6 differs from Examples 4 and 5 in that the function summary information includes two-dimensional coordinate information 1730 to 1740 in the workspace 240 where each node is arranged, and a priority confirmation 1750 is added to the execution node 1706.

[0219] Two-dimensional coordinate information 1730 to 1740 is the two-dimensional coordinate position in workspace 240 where the node is placed when assigning an ID to each node, and is added to the function summary information.

[0220] The priority confirmation 1750 checks the position of each node in the acquired synchronized overlap integration summary information group 1301, adds a processing priority so that processing of paths that are higher on the vertical axis of the two-dimensional coordinate system in the workspace 240 is executed with priority, and outputs the prioritized synchronized overlap integration summary information to the configuration information generation 1244.

[0221] Specifically, for example, in the branch following camera input 1200, the node for inference A 1201 is located at the top in the layout, so the process with higher priority is executed. This priority is also inherited by subsequent nodes. In other words, the processes for camera input 1200, inference A 1201, and result drawing 1205 are processed preferentially via other paths.

[0222] 18 is an explanatory diagram illustrating a specific example 1 of the processing of the overlap integration 1242 and the node connection arrangement 1243 according to the embodiment 6. FIG. 18 illustrates a change in the integrated summary information from the integrated summary information group 1800 acquired by the overlap integration 1242 and the node connection arrangement 1243 in the execution node 1206 to the overlap combined summary information group 1801.

[0223] For example, the function summary information 1220 to 1222 of the camera input 1200 are assigned the same ID "AAA" and the same two-dimensional coordinate information 1730 to 1732, so the overlap integration 1242 determines that they are the same node, integrates them, and outputs them as overlap integration summary information 1811.

[0224] Furthermore, since the function summary information 1224 of inference B1202 and the function summary information 1225 of inference B1202 are assigned the same ID "CCC" and the same two-dimensional coordinate information 1734, 1735, the overlap integration 1242 determines that they are the same node, integrates them, and outputs them as overlap integration summary information 1812.

[0225] Furthermore, since the function summary information 1227 and 1228 of the resultant drawing 1205 are assigned the same ID "EEE" and the same two-dimensional coordinate information 1739 and 1740, the overlap integration 1242 determines that they are the same node, integrates them, and outputs them as overlap integration summary information 1313. The function summary information 1230 is also assigned the ID "EEE", but since the two-dimensional coordinate information 1738 is different from the two-dimensional coordinate information 1739 and 1740, the function summary information 1230 is not integrated into the overlap integration summary information 1313.

[0226] 19 is an explanatory diagram illustrating a specific example 2 of the process of the overlap integration 1242 and the node connection rearrangement 1243 according to the embodiment 6. In FIG. 19, a change in the integrated summary information from the overlap combined summary information group 1801 to the synchronized overlap integrated summary information group 1802 is shown.

[0227] The node connection arrangement 1243 assigns node connection confirmation and synchronization processing information 1920, and the priority confirmation 1750 assigns a priority 1930 to the combination of the overlapping integrated summary information 1811 (camera input 1200), the function summary information 1223 (inference A 1201), and the function summary information 1226 (result drawing 1205), i.e., the integrated summary information 1231. The integrated summary information 1231 to which the priority 1930 is set is executed with priority over the integrated summary information 1232 and 1233. In this way, a prioritized synchronized overlapping integrated summary information group 1902 is generated. This makes it possible, for example, to reduce the processing time for high-priority processing.

[0228] <Example of processing procedure of generation unit 121> Fig. 20 is a flowchart illustrating a detailed procedure of a generation process by the generation unit 121 of the design execution system 100 according to the sixth embodiment. Fig. 20 illustrates an example of generating configuration information using ID assignment, output number setting, duplication integration, flag checking, and the like in the generation unit 121 of the design execution system 100 according to the sixth embodiment.

[0229] Regarding the detailed processing procedure for generating configuration information in the generation unit 121 of the design execution system 100, only the differences from the fourth and fifth embodiments will be described. The detailed processing procedure for generating configuration information differs from the fourth embodiment in that step S2000 and step S2001 are added.

[0230] In step S2000, the generation unit 121 acquires the two-dimensional coordinates of the position where the node is to be placed in the workspace 240, and outputs them to step S1404.

[0231] In step S2001, the generation unit 121 checks the position of each node in priority confirmation 1650, adds a processing priority so that processing of paths that are higher on the vertical axis of the two-dimensional coordinate system is executed with priority, generates prioritized synchronized overlap integrated summary information, and proceeds to processing in step S1410.

[0232] In this way, according to the sixth embodiment, by assigning processing priorities to nodes branched for parallelization based on the node placement position, it is possible to obtain a design execution system 100 that can reduce the processing time for specific processes even in a processor with limited calculation speed and processing resources.

[0233] As explained above, according to the design execution device of the above-mentioned Examples 1 to 6, the system to be designed can be designed using a graphical user interface, and the system can be executed from the design platform, and after the system is designed, the system can be executed using the execution platform alone.

[0234] The design execution apparatus according to the first to sixth embodiments can also be configured as follows (1) to (11).

[0235] (1) The design support device (client 110) includes a design unit 120, a generation unit 121, and an execution unit 123. The design unit 120 has a node for each function, which does not have the actual process of implementing the function but includes summary information that specifies information identifying the process. The design unit 120 can connect multiple nodes via external operations. The design unit 120 generates execution order data (system information) that specifies the execution order of specific functions related to the design object by connecting specific node groups related to the design object. The generation unit 121 generates configuration information 320 for the design object, which specifies the summary information used in each of the specific function groups according to the execution order, by inheriting the summary information of a previous node in the specific node group to a subsequent node connected subsequent to the previous node, in accordance with the execution order data generated by the design unit 120. The execution unit 123 instantiates processes that implement the specific function groups using the configuration information 320 generated by the generation unit 121 and executes the instantiated processes in the execution order.

[0236] This improves the convenience of designing and executing the design object. Specifically, for example, since the design unit 120 and the generation unit 121 handle a group of nodes that do not have the actual processes that implement the functions, only the inheritance of summary information is performed between the nodes. In other words, no inter-node communication occurs regarding the results of processing between the nodes. Furthermore, there is no need to pass data of the processing results between the design unit 120, the generation unit 121, the configuration information 320, and the storage unit 122, which execute the processes that implement the functions for each node, and the execution platform 432. This improves the efficiency of design and improves the convenience for designers. Furthermore, the execution unit 123 instantiates (instantiates) only the necessary functions using the configuration information 320, thereby improving the efficiency of execution of the design object and improving the convenience for designers.

[0237] (2) In the design execution device of (1) above, the design unit 120 displays the nodes by their functions, and allows a plurality of the nodes to be connected by external operation (for example, by the designer operating the input device 204), and generates the execution sequence data by connecting the specific group of nodes.

[0238] This allows the design object to be designed using a graphical user interface, improving convenience for designers.

[0239] (3) In the design execution apparatus of (2) above, the design unit 120 displays detailed setting screens 246 to 248 as the summary information of the node in response to the external operation.

[0240] This allows designers to visually see which functions have which items and what their setting values ​​are, improving convenience for designers. Also, by allowing the setting values ​​to be changed by user operation, convenience for designers is further improved.

[0241] (4) In the design execution device of (2) above, the design unit 120 displays the nodes for each function on a browser 400 of an external device that can communicate with the design execution device, and enables connection of a plurality of the nodes by the external operation from the external device, and generates the execution sequence data by connecting the specific group of nodes.

[0242] This allows the design and execution of a design object to be performed remotely from an external device, thereby improving convenience for designers.

[0243] (5) In the design execution device of (1) above, the generation unit 121 refers to the execution order data, and for each node of the specific node group, if there is a previous node, generates integrated summary information by integrating the summary information of the previous node and the summary information of the node, and if the integrated summary information of the previous node has been generated at the previous node, generates integrated summary information by integrating the integrated summary information of the previous node and the summary information of the node, and outputs the integrated summary information generated at the final node as the configuration information 320.

[0244] This allows the designer to simply connect nodes, and at each node, all summary information from the first node to the node in question is integrated and passed to the subsequent node. Therefore, simply by the designer connecting nodes, the processes that realize specific functions are executed in the appropriate execution order. This improves convenience for the designer.

[0245] (6) In the design execution apparatus of (5) above, the generation unit 121 assigns a first identification information (ID) to each of the plurality of summary information in the integrated summary information if the first identification information (ID) that uniquely identifies the summary information within the integrated summary information has not been assigned (step S801).

[0246] This allows the designer to simply connect the nodes, and the first identification information is set in the summary information from the first node to the node in question in accordance with the execution order, so that the process groups that realize the specific functions are executed in the order of the first identification information (ID).This improves convenience for the designer.

[0247] (7) In the design execution device of (6) above, if there are multiple preceding nodes for each node in the specific node group and different summary information to which the same first identification information is assigned exists for the multiple preceding nodes, the generation unit 121 assigns second identification information (sub-ID) that distinguishes between the different summary information to the first identification information (step S804).

[0248] This eliminates duplication of first identification information by simply connecting nodes without the designer having to be aware of the duplication of first identification information, making it possible to realize parallel processing and multi-processing. Also, malfunctions in the execution unit 123 can be avoided in advance. This improves convenience for designers.

[0249] (8) In the design execution apparatus of (1) above, if there are multiple preceding nodes for each node in the specific node group and the same summary information exists in the multiple preceding nodes, the generation unit 121 eliminates duplication of the same summary information and inherits it to the succeeding node (step S803).

[0250] This allows the designer to simply connect nodes, without having to be aware of duplication of summary information, and to prevent duplicate execution of the same process in the execution unit 123 in advance. This therefore improves the efficiency of design and improves convenience for the designer.

[0251] (9) The design execution device of (1) above has a storage unit 122 that stores a function group 420, which is a set of functions, and the configuration information 320. When the function group 420 and the configuration information 320 are stored in the storage unit 122, the execution unit 123 selects the specific function group from the function group 420 stored in the storage unit 122, instantiates a process that realizes the specific function group using the configuration information 320, and executes the instantiated process in the execution order.

[0252] As a result, as long as the function group 420 and the configuration information 320 are stored, the system can be executed by the execution unit 123 alone at the time of execution after system design, without executing the design unit 120 and the generation unit 121. This improves the efficiency of design and convenience for designers. For example, when executing by the execution unit 123, operations such as starting up Nod-RED, which constitutes the design unit 120, and pressing a start button are not required. Then, execution by the execution unit 123 begins simply by turning on the power of the design execution device. Alternatively, the execution unit 123 may be started by being started from a specific program or in response to a start signal.

[0253] (10) In the design execution device of (9), the storage unit 122 stores a containerized function set 1120 obtained by containerizing the function set 420, and the execution unit 123 stores the containerized function set 1120 stored in the storage unit 122. Specific containerization featuresThe specific containerized function group is instantiated using the configuration information, and the instantiated processes are executed in the execution order.

[0254] This eliminates the need for version management of the function set 420, and makes it possible to avoid malfunctions during execution of the execution unit 123. Containerization in this way facilitates maintenance of the function set 420, improving convenience for designers.

[0255] (11) The design execution apparatus of (1) above includes a processing device 203 that executes a specific process, and a control unit 124 that controls the processing device 203 based on a process that realizes a function of controlling the processing device 203 that is materialized by the execution unit 123.

[0256] This allows the functions described in the configuration information 320 to be materialized and the designed system to be executed while cooperating with the processing device 203 .

[0257] (12) In the design execution device of (5) above, the generation unit 121 acquires identification information (ID) that uniquely identifies the node and the number of outputs that indicates the number of execution paths of the execution order data that are identified by the position of the node based on the number of the connected subsequent nodes, stores them in the summary information, and generates the configuration information 320 by adding synchronization processing information that starts processing when output from the previous node to the node is completed.

[0258] This allows synchronous processing (calling of execution nodes) to be performed in an asynchronous system that performs parallel processing or multi-process processing.

[0259] (13) In the design execution device of (12) above, the generation unit 121 determines whether or not the integrated summary information for the number of outputs has been acquired at the final-stage node based on the number of outputs, and if so, generates the configuration information 320 by adding the synchronization processing information.

[0260] This eliminates the need for a synchronization node to consolidate multiple execution path invocations of an execution node into a single call, and prevents human errors such as incorrect connection settings made by the user manually.

[0261] (14) In the design execution device of (12) above, the generation unit 121 generates integrated summary information by integrating multiple pieces of summary information having the same identification information from the integrated summary information of the number of outputs, and generates the configuration information 320 by adding the synchronization processing information to the integrated summary information.

[0262] This makes it possible to reduce the amount of data in the configuration information 320 and simplify the execution order data.

[0263] (15) In the design execution device of (13) above, when the node branches to a plurality of subsequent nodes, the generation unit 121 acquires a flag signal relating to the number of branches for each subsequent node, stores the flag signal in the summary information of the subsequent node, and determines whether or not the final node has acquired the integrated summary information for the number of outputs based on the flag signal and the number of outputs. If so, the generation unit 121 generates the configuration information 320 by adding the synchronization processing information.

[0264] This allows synchronous processing (calling of execution nodes) to be performed even in asynchronous systems where nodes that branch out for parallelization are once consolidated into one node and parallelization is consolidated, and then parallelization is performed again.

[0265] (16) In the design execution device of (12) above, the generation unit 121 acquires coordinate information indicating the coordinate position of each node of the specific node group placed in the workspace 240 by the external operation, and stores the coordinate information in the summary information of the node.

[0266] This allows the user to confirm which node is placed at which position on the workspace 240.

[0267] (17) In the design execution device of (16) above, the generation unit 121 generates the configuration information 320 by setting priorities of multiple execution paths included in the execution order data based on the coordinate information at the final-stage node.

[0268] This allows a priority to be set for each execution path, and processing related to the execution path can be executed according to the priority.

[0269] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added to, deleted from, or replaced with other configurations.

[0270] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example by designing them as integrated circuits, or may be realized in software by having processor 201 interpret and execute programs that realize the respective functions.

[0271] Information such as programs, tables, and files that realize each function can be stored in storage devices such as memory, hard disks, and SSDs (Solid State Drives), or on recording media such as IC (Integrated Circuit) cards, SD cards, and DVDs (Digital Versatile Discs).

[0272] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines that are necessary for implementation. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0273] 100 Design Execution System 101 Network 110 clients 111 Server 201 processor 202 Storage Devices 203 Processing Device 204 Input Devices 205 Output Devices 206 Communication Interface 207 Bus 120 Design Department 121 Generation part 122 Holding part 123 Executive Department 124 Control Unit 130 Function Group Creation Department 1240 Get Function Overview 1241 Output Number Judgment 1242 Duplicate Integration 1243 Node Connection Arrangement 1244 Configuration information generation

Claims

1. a design unit that has a node for each function that does not have the substance of a process that realizes the function but includes summary information that specifies information that identifies the process, and that can connect multiple nodes by external operations, and that generates execution order data that specifies the execution order of specific function groups by connecting specific node groups; a generation unit that generates configuration information that defines the summary information used in each of the specific function groups in accordance with the execution order by inheriting summary information of a previous node among the specific node group to a subsequent node connected subsequent to the previous node in accordance with the execution order data generated by the design unit; an execution unit that instantiates processes for realizing the specific group of functions using the configuration information generated by the generation unit, and executes the instantiated processes in the execution order; A design execution device comprising:

2. 2. The design execution apparatus according to claim 1, the design unit displays the nodes for each function, allows a plurality of the nodes to be connected by an external operation, and generates the execution sequence data by connecting the specific group of nodes; A design execution device characterized by:

3. 3. The design execution apparatus according to claim 2, the design unit displays the summary information of the node in response to the external operation. A design execution device characterized by:

4. 3. The design execution apparatus according to claim 2, the design unit displays the nodes for each of the functions on an external device that can communicate with the design execution device, and is capable of connecting a plurality of the nodes by the external operation from the external device, and generates the execution sequence data by connecting the specific group of nodes; A design execution device characterized by:

5. 2. The design execution apparatus according to claim 1, the generation unit refers to the execution order data, and for each node of the specific node group, if there is a previous node, generates integrated summary information by integrating summary information of the previous node and summary information of the node, and if integrated summary information of the previous node has been generated at the previous node, generates integrated summary information by integrating integrated summary information of the previous node and summary information of the node, and outputs the integrated summary information generated at the final node as the configuration information. A design execution device characterized by:

6. 6. The design execution apparatus according to claim 5, the generation unit assigns, to each of the plurality of pieces of summary information within the integrated summary information, first identification information that uniquely identifies the summary information within the integrated summary information, if the first identification information is not assigned to the summary information. A design execution device characterized by:

7. 7. The design execution apparatus according to claim 6, the generation unit, for each node in the specific node group, if there are a plurality of the preceding nodes and different pieces of summary information to which the same first identification information is assigned exist for the plurality of preceding nodes, assigns second identification information to the first identification information, which distinguishes between the different pieces of summary information; A design execution device characterized by:

8. 2. The design execution apparatus according to claim 1, the generation unit, for each node in the specific node group, if there are a plurality of preceding nodes and the same summary information exists in the plurality of preceding nodes, eliminates duplication of the same summary information and inherits it to the succeeding node; A design execution device characterized by:

9. 2. The design execution apparatus according to claim 1, a storage unit that stores a function group that is a set of the functions and the configuration information; When the function group and the configuration information are stored in the storage unit, the execution unit selects the specific function group from the function groups stored in the storage unit, instantiates processes that realize the specific function group using the configuration information, and executes the instantiated processes in the execution order. A design execution device characterized by:

10. 10. The design execution apparatus according to claim 9, the storage unit stores a containerized function set obtained by containerizing the function set; the execution unit selects a specific containerized function group from the containerized function group stored in the storage unit, instantiates the specific containerized function group using the configuration information, and executes the instantiated processes in the execution order. A design execution device characterized by:

11. 2. The design execution apparatus according to claim 1, a processing device for performing a specific process; a control unit that controls the processing device based on a process that realizes a function that controls the processing device, the process being realized by the execution unit; A design execution device comprising:

12. 6. The design execution apparatus according to claim 5, the generation unit acquires identification information that uniquely identifies the node and an output number that indicates the number of execution paths of the execution order data that are identified by the position of the node based on the number of the connected subsequent nodes, stores these in the summary information, and generates the configuration information by adding synchronization processing information that starts processing when output from the previous node to the node is completed. A design execution device characterized by:

13. 13. The design execution apparatus according to claim 12, the generation unit determines whether or not the integrated summary information for the number of outputs has been acquired at the final-stage node based on the number of outputs, and if so, generates the configuration information by adding the synchronization processing information. A design execution device characterized by:

14. 13. The design execution apparatus according to claim 12, the generation unit generates integrated summary information by integrating a plurality of pieces of summary information having the same identification information among the integrated summary information for the number of outputs, and generates the configuration information by adding the synchronization processing information to the integrated summary information. A design execution device characterized by:

15. 14. The design execution apparatus according to claim 13, the generation unit, when branching from the node to a plurality of the subsequent nodes, acquires a flag signal relating to the number of branches for each of the subsequent nodes, and stores the acquired flag signal in the summary information of the subsequent node; and, at the final node, determines whether or not the integrated summary information for the number of outputs has been acquired based on the flag signal and the number of outputs; and, if acquired, generates the configuration information by adding the synchronization processing information. A design execution device characterized by:

16. 13. The design execution apparatus according to claim 12, the generation unit acquires coordinate information indicating a coordinate position where each node of the specific node group is placed in a working area by the external operation, and stores the coordinate information in the summary information of the node. A design execution device characterized by:

17. 17. The design execution apparatus according to claim 16, the generation unit generates the configuration information by setting priorities of a plurality of execution paths included in the execution order data based on the coordinate information in the final-stage node. A design execution device characterized by:

18. A design execution system having a server that holds a group of functions and a client that can communicate with the server, a design unit that has a node for each function that does not have the substance of a process that realizes the function but includes summary information that specifies information that identifies the process, and that can connect multiple nodes by external operations, and that generates execution order data that specifies the execution order of specific function groups by connecting specific node groups; a generation unit that generates configuration information that defines the summary information used in each of the specific function groups in accordance with the execution order by inheriting summary information of a previous node among the specific node group to a subsequent node connected subsequent to the previous node in accordance with the execution order data generated by the design unit; an execution unit that instantiates processes that realize the specific group of functions using the configuration information generated by the generation unit, and executes the instantiated processes in the execution order; A design execution system comprising:

19. 20. The design and execution system of claim 18, The client: The design unit, the generation unit, and the execution unit, A design execution system comprising:

20. 20. The design and execution system of claim 18, The server The design unit and the generation unit are included, The client: The execution unit is provided. A design execution system characterized by:

21. A computer having a processor that executes a program and a storage device that stores the program, a design process in which a node is provided for each function, which node does not have the substance of a process that realizes the function but includes summary information that specifies information that identifies the process, and a plurality of the nodes can be connected by external operations, and which generates execution sequence data that specifies the execution sequence of a specific group of functions by connecting specific groups of nodes; a generation process for generating configuration information that defines the summary information used in each of the specific function groups in accordance with the execution order by inheriting summary information of a previous node among the specific node group to a subsequent node connected subsequent to the previous node, according to the execution order data generated by the design process; an execution process that instantiates processes that realize the specific group of functions using the configuration information generated by the generation process, and executes the instantiated processes in the execution order; A design execution method comprising:

Images