Information processing device, control device, control method, and program

Abstract

To achieve a technique for visualizing a program execution state in a control target and supporting program creation of a user.SOLUTION: An information processing device for creating a program to be executed by a control device includes: transmission means configured to transmit the program and an execution method of the program to the control device; and control means. The control means transmits no acquisition request of an execution state of the program to the control device when the execution method of the program is a first execution method, and transmits an acquisition request of the execution state of the program to the control device and displays the execution state of the program received from the control device when the execution method of the program is a second execution method different from the first execution method.SELECTED DRAWING: Figure 4B

Description

[Technical field]

[0001] The present invention relates to a technique for supporting program development. [Background technology]

[0002] The most common way to program is to write text by typing it in, which is called "coding" or "text programming." In contrast to this, there is "visual programming," which is a method of creating programs using visual objects instead of text.

[0003] There are various types of "visual programming," such as combining objects called blocks on a screen, or connecting rectangular or circular objects on a screen with arrows or lines. Systems that use "visual programming" to control real-world objects such as robots are also known.

[0004] Patent Document 1 describes a technique that enables a user to program a controlled object to perform a desired operation. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6664641 Summary of the Invention [Problem to be solved by the invention]

[0006] However, Patent Document 1 does not mention displaying the program execution status to check (debug) whether the current program is the expected program while the user is creating the program. In other words, to check whether the program is the expected program, it is necessary to visually check the operation of the controlled object from the outside, but Patent Document 1 does not allow visual check of the program execution status of the controlled object, making it difficult to determine whether the program is the expected program.

[0007] The present invention has been made in view of the above problems, and has as its object to realize a technology for visualizing the program execution status of a control target and assisting users in creating programs. [Means for solving the problem]

[0008] In order to solve the above problems and achieve the object, the present invention provides an information processing device that creates a program to be executed by a control device, comprising: a transmission means that transmits the program and an execution method of the program to the control device; and a control means that, if the execution method of the program is a first execution method, does not transmit a request to obtain the execution status of the program to the control device, and, if the execution method of the program is a second execution method different from the first execution method, transmits a request to obtain the execution status of the program to the control device and displays the execution status of the program received from the control device. Effect of the Invention

[0009] According to the present invention, in program development, the program execution status of a control target can be visualized to assist a user in creating a program. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram illustrating a configuration of a development support device and a control target according to an embodiment of the present invention. [Figure 2A] FIG. 2 is a diagram illustrating a configuration of a program including programmed blocks according to the embodiment; [Figure 2B] FIG. 4 is a diagram illustrating a UI screen provided by the programming tool of the first embodiment. [Figure 2C] FIG. 4 is a diagram illustrating a UI screen provided by the programming tool of the first embodiment. [Figure 2D] FIG. 4 is a diagram illustrating a UI screen provided by the programming tool of the first embodiment. [Figure 2E] FIG. 4 is a diagram illustrating a UI screen provided by the programming tool of the first embodiment. [Diagram 3] FIG. 2 is a diagram for explaining a scenario including a program and an execution method according to the first embodiment. [Figure 4A] FIG. 4 is a sequence diagram illustrating a process of a first execution method in the first embodiment. [Figure 4B] FIG. 4 is a sequence diagram illustrating a process of a second execution method in the first embodiment. [Diagram 5] 4 is a flowchart illustrating a process of a control target according to the first embodiment. [Figure 6] 6 is a flowchart illustrating a process of executing a program in the second execution mode in step S508 of FIG. 5. [Figure 7] 6 is a flowchart illustrating a process of executing a program in the first execution mode in step S506 of FIG. 5. [Figure 8] 10 is a flowchart illustrating a process of executing a program in the second execution mode in step S508 of FIG. 5 according to the second embodiment. [Figure 9] FIG. 11 is a diagram illustrating an example of a UI screen provided by a programming tool according to the second embodiment. [Figure 10] FIG. 13 is a diagram illustrating an example of a UI screen provided by a programming tool according to the third embodiment. [Figure 11] 11 is a flowchart illustrating a process of executing a program in the second execution mode in step S508 of FIG. 5 according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0012] [Embodiment 1] The system of this embodiment includes a control target that operates according to a control program, and a development support device that supports development of the control program.

[0013] The development support device of this embodiment is, for example, an information processing device such as a personal computer (PC), but may also be a smartphone, a tablet device, or the like.

[0014] The development support device of this embodiment provides a visual programming tool (hereinafter, referred to as the programming tool) that can create a control program by combining blocks. The programming tool of this embodiment is executed as an application that runs on a PC serving as the development support device. A block is a visual object that represents a processing unit of a program. There are various types of blocks, and a program can be created by combining them. A program created with the programming tool is composed of at least one block.

[0015] The processes of the development support device of this embodiment may be implemented by a single PC, or may be implemented by distributing the functions among multiple PCs as necessary. The multiple PCs are connected to each other so that they can communicate with each other.

[0016] The controlled object in this embodiment is a control device such as an automatic camera equipped with a PTZ function. The PTZ function is a function that can control the panning, tilting, and zooming of the controlled object. PTZ is an abbreviation of the initials of Pan, Tilt, and Zoom. Pan is the horizontal oscillation. Tilt is the vertical oscillation. Zoom is zooming up (telephoto) and zooming out (wide angle).

[0017] <Configuration of development support device> First, the configuration of the development support device of this embodiment will be described with reference to FIG.

[0018] FIG. 1A is a block diagram illustrating the configuration of a development support device 100 according to this embodiment.

[0019] The development support device 100 includes a control unit 101 , a non-volatile memory 102 , a work memory 103 , a storage device 104 , an input interface 105 , an output interface 106 , a network interface 107 , a communication interface 108 , and a system bus 109 .

[0020] The control unit 101 includes an arithmetic processor such as a CPU or MPU that controls the entire development support device 100. The non-volatile memory 102 is a ROM that stores programs and parameters executed by the processor of the control unit 101. Here, the programs refer to programs for executing the processes of the first and second embodiments described below. The work memory 103 is a RAM that temporarily stores programs and data supplied from an external device or the like. The storage device 104 is an internal device such as a hard disk or a memory card built into the development support device 100, or an external device such as a hard disk or a memory card detachably connected to the development support device 100. The storage device 104 includes a memory card or a hard disk constituted by a semiconductor memory, a magnetic disk, or the like. The storage device 104 also includes a storage medium constituted by a disk drive that reads / writes data from / to optical disks such as DVDs and Blu-ray (registered trademark) Discs.

[0021] The input interface 105 accepts user operations on an input device 110 such as a mouse, keyboard, or touch panel, and outputs operation instructions to the control unit 101. The output interface 106 displays data held by the development support device 100 or data supplied from an external device on an output device 111 such as a display or monitor composed of an LCD or an organic EL. The network interface 107 connects to a network 112 such as the Internet or a LAN (Local Area Network). The communication interface 108 connects to an external device 113 such as a control target 150 via a communication line such as 4G or 5G so as to be able to communicate with each other. The system bus 109 connects the components 101 to 107 of the development support device 100 so as to be able to exchange data.

[0022] The processing of the development support device 100 of this embodiment is realized by reading software provided by an application. Note that the application has software for utilizing basic functions of the OS installed in the development support device 100. Note that the OS of the development support device 100 may have software for realizing the processing of this embodiment.

[0023] <Control target configuration> Next, the configuration of the controlled object of this embodiment will be described with reference to FIG.

[0024] FIG. 1B is a block diagram illustrating the configuration of a controlled object 150 according to this embodiment.

[0025] The controlled object 150 includes a lens barrel 151 and a control box 154 .

[0026] The lens barrel 151 includes a lens unit 155 and an imaging section 156 , and is rotationally driven in the tilt direction by a tilt driving section 152 , and is rotationally driven in the pan direction by a pan driving section 153 .

[0027] The lens unit 155 includes a zoom lens and a focus lens, and the zoom position and focus position are driven and controlled by a lens driving unit 157 .

[0028] The imaging unit 156 is equipped with an image sensor such as a CMOS that converts the subject image light formed by the lens unit 155 into an electrical signal, and converts the captured analog image signal into a digital signal and outputs it to the image processing unit 159 and the communication unit 164 as image data.

[0029] Tilt drive unit 152 includes a drive mechanism such as a motor and a driver for driving and rotating barrel 151 in the pitch direction. Pan drive unit 153 includes a drive mechanism such as a motor and a driver for driving and rotating barrel 151 in the yaw direction. Tilt drive unit 152 and pan drive unit 153 drive and rotate barrel 151 in the tilt direction and pan direction in response to a drive signal from barrel drive unit 158.

[0030] The control box 154 includes a lens driving section 157 , a lens barrel driving section 158 , an image processing section 159 , a subject information recording section 160 , a storage section 161 , a control section 162 , a subject detection section 163 , and a communication section 164 .

[0031] Lens driving unit 157 includes a driving mechanism such as a motor and a driver for driving the zoom lens and focus lens included in lens unit 155. Lens driving unit 157 drives lens unit 155 to a zoom position and a focus position in response to a control command (target position and driving speed) from control unit 162.

[0032] The lens barrel driver 158 outputs drive signals to the tilt driver 152 and the pan driver 153 in response to a control command from the control unit 162, and drives the lens barrel 151 in the tilt direction and the pan direction.

[0033] Note that the control commands from control unit 162 to lens driving unit 157 and lens barrel driving unit 158 ​​are not limited to target positions and driving speeds, and may be acceleration, time-division data to the target position, or the like.

[0034] The image processing unit 159 performs image processing such as distortion correction, white balance adjustment, and color interpolation processing on the image data captured by the imaging unit 156, and outputs the image data after the image processing to the subject detection unit 163. In addition, the image processing unit 159 converts the image data after the image processing into a recording format such as JPEG or MPEG, and outputs it to the storage unit 161.

[0035] The storage unit 161 includes a ROM, which is a non-volatile memory that stores programs and parameters executed by the processor of the control unit 162, and a RAM, which is a volatile memory that temporarily stores programs and data supplied from the development support device 100. The storage unit 161 also stores, as shooting situation information of the control target 200 to be transmitted to the development support device 100, image data (captured image) captured by the imaging unit 156, information indicating whether or not a subject has been detected from the captured image, subject information detected from the captured image, information indicating whether or not the captured image has been captured, and the like.

[0036] The control unit 162 includes an arithmetic processor such as a CPU or MPU that controls the entire control target 150. The control unit 162 executes a program stored in the storage unit 161 and performs automatic shooting by controlling each component of the control target 150. To perform automatic shooting without user operation, the control unit 162 executes a search sequence for detecting a subject to be shot (target subject), a tracking sequence for tracking the subject detected by the search sequence, and a shooting sequence including zoom control, focus control, and exposure control for the subject.

[0037] The search sequence detects a subject within the angle of view while changing the pan / tilt position and zoom position. The tracking sequence determines a target subject based on the subject detection result by the subject detection unit 163. The tracking sequence then determines target positions for the zoom position of the lens unit 155 and the tilt and pan directions of the lens barrel 151 to obtain an appropriate composition for shooting such that the target subject fits within the angle of view, and controls the lens driver 157 and the lens barrel driver 158 based on the determined target positions. The shooting sequence starts and ends shooting at appropriate timing, and controls the recording of image data.

[0038] Subject detection unit 163 includes processes for determining the orientation of a subject such as a person's face, detecting the position of the subject, detecting the size of the subject, registering the subject, and uniquely identifying the registered subject. Subject detection unit 163 extracts subject information (e.g., face size, face position, face orientation, face identification result, etc.) from the subject area of ​​the image data output from image processing unit 159, and outputs it to control unit 162 and subject information recording unit 160.

[0039] The subject information recording section 160 records the subject information based on the subject detection result input from the subject detection section 163 .

[0040] The communication unit 164 is communicably connected to an external device such as the development support device 100 via a communication line such as 4G or 5G. The communication unit 164 also transmits the shooting situation information stored in the storage unit 161, the subject information recorded in the subject information recording unit 160, the control command (pan / tilt position and zoom position) output from the control unit 162, and the image data captured by the imaging unit 156 to the external device.

[0041] <Programming tool explanation> Next, a programming tool that operates in the development support device 100 of this embodiment will be described with reference to FIG.

[0042] Fig. 2A is a diagram illustrating a configuration of a program including programmed blocks, Fig. 2B to Fig. 2E are diagrams illustrating UI screens provided by the programming tool of this embodiment.

[0043] First, a method for creating the program 200 using the development support device 100 of this embodiment will be described.

[0044] Program 200 is created by dragging and dropping any of a plurality of blocks displayed in block list 210 shown in FIG. 2B onto operation area 212 and combining the blocks in operation area 212. In the example of FIG. 2A, program 200 has blocks 201 to 203 selected from block list 210.

[0045] Depending on the type of block, the user may be able to arbitrarily set parameters for the block, and these can be input by clicking inside the block frame 211. Parameters that can be input are, for example, the name of the subject registered in the control target 150, and the pan / tilt position of the control target 150. The name of the subject is set when the development support device 100 acquires from the control target 150 image data associated with the subject information registered in the control target 150, displays it, and the user inputs the name in the name registration screen 215 in FIG. 2E. The set name of the subject is also associated with the "faceid" of the scenario, which will be described later in FIG. 3, and transmitted to the control target 150.

[0046] The name of the subject is input by selecting it from a list of names for each subject, and the pan / tilt position is input by input device 110 at any angle between 0 degrees and 360 degrees.

[0047] Although the subject's name and pan / tilt position are given as examples of block parameters that can be input, the parameters that can be input are not limited to these and may be, for example, a date and time or the number of seconds to specify the execution timing of the block.

[0048] Next, display of the program execution status by the programming tool of this embodiment will be described.

[0049] 2B, an arrow 204 is aligned with a block being executed, and the block being executed is surrounded by a thick frame 205, thereby emphasizing the block being executed. This clearly indicates the block currently being executed. Also, the name of the subject in the image captured by the control target 150, the shooting conditions of the image, the pan / tilt position, and the like are displayed in an information display area 206.

[0050] The name of the subject is obtained from the control object 150 and displayed when the subject in the image matches the registered subject information. The shooting conditions of the image are obtained from the control object 150 and displayed. The pan / tilt position is obtained from the control object 150 and displayed. Furthermore, image data (live view images) 207-209 (FIGS. 2B-4D) captured by the control object 150 are obtained in real time from the control object 150 and displayed.

[0051] In this way, by displaying the execution status of the program by the control target 150 in the development support device 100, the user can visually check the program execution status and can confirm whether or not the expected program has been created.

[0052] The system of this embodiment can execute a program in either a first execution method or a second execution method, which will be described later.

[0053] The first execution method executes a program immediately when the power supply of the control target 150 is turned on. The first execution method can be specified by the user operating the button 213 of the programming tool.

[0054] In the second execution method, the user issues an instruction using a programming tool, which causes the control target 150 to execute a program, and as described above, the execution status can be visually confirmed using the development support device 100. The second execution method can be specified by the user operating the button 214 on the programming tool.

[0055] <Scenario description> Next, a scenario 300 of this embodiment will be described with reference to FIG.

[0056] Scenario 300 includes program 302 and execution method 301 for program 302. Program 302 is created in development support device 100 in the same format as program 200 in Fig. 2A. Program 302 is converted from block-format program 200 to JSON (JavaScript Object Notation) format program 302 in development support device 100. When block-format program 200 is converted to text format program 302, one block is converted to one line of text. Block 201 in Fig. 2A corresponds to program 303 in Fig. 3, blocks 202 and 304, and blocks 203 and 305, respectively.

[0057] A program 303 in which the "Wait until [Mom] is found" block 201 is converted to JSON format represents a block that waits for a subject whose "faceid" is registered as "1" in the control target 150. "Faceid" is subject information registered by the control target 150, and is stored in the control target 150 and the development support device 100 in association with the name of the subject entered by the user in the development support device 100.

[0058] A program 304 in which the “take still image” block 202 has been converted into JSON format represents a block for taking a still image. A program 305 in which the “turn right

[30] degrees” block 203 has been converted into JSON format represents a block for panning to a position 30 degrees to the right from the current pan / tilt position.

[0059] The execution method is specified by the user operating the programming tool. When the execution method is specified, the specified execution method is converted into JSON data such as program 301. When the first execution method is specified, it is converted into "executeType":"1", and when the second execution method is specified, it is converted into "executeType":"2".

[0060] The scenario 300, which is made up of the programs 302 to 303 and the execution method 301 converted into the JSON format, is transmitted from the development support device 100 to the control target 150.

[0061] In addition to the blocks in the block list in Fig. 2A that make up program 200, such as "Wait until [Mom] is found" (201), "Take a still image" (202), and "Turn right

[30] degrees" (203), there are other blocks such as "Turn left / up / down

[30] degrees" and "Take a video", as well as blocks with conditions such as "After [5] seconds have passed" and "When the power button of the controlled object is pressed". The [Mom] and

[30] parts in [] in each block indicate values ​​that can be input by the user. In other words, a person's name other than [Mom] and parameters other than

[30] can be set as desired.

[0062] Next, with reference to FIG. 4, a processing sequence according to the first execution method and the second execution method in the system of this embodiment will be described.

[0063] First, a processing sequence 400 of the first execution method will be described with reference to FIG. 4A.

[0064] In S401 and S402, a program specifying the first execution method is created in the development support device 100 and transmitted to the control target 150. As described above, the user can specify the first execution method by operating the button 213 of the programming tool shown in FIG.

[0065] In S403 and S404, the control target 150 analyzes the program received from the development support device 100 and stores the analysis results.

[0066] In S405, the control target 150 immediately executes the program. Also, in S406 to S408, even if the power supply of the control target 150 is turned off, the program can be read out from the storage unit 161 and executed immediately when the power supply of the control target 150 is turned on again.

[0067] Next, a processing sequence 410 of the second execution method will be described with reference to FIG. 4B.

[0068] In S411 and S412, a program specifying the second execution method is created in the development support device 100 and transmitted to the control target 150. As described above, the user can specify the second execution method by operating the button 214 of the programming tool shown in FIG.

[0069] In steps S413 and S414, the control target 150 analyzes the program received from the development support device 100 and temporarily stores the analysis results. That is, when the power supply to the control target 150 is turned off in S419, the program is erased, and even if the power supply to the control target 150 is turned on in S420, the program is not executed.

[0070] In S415, the control target 150 executes the program in the second execution method, and at the same time, in S416, the development support device 100 requests the control target 150 to acquire the program execution status, and in S417, the control target 150 transmits the program execution status to the development support device 100. Then, in S418, the development support device 100 displays the program execution status. The program execution status displayed on the development support device 100 is as shown in Figs. 2B to 2E.

[0071] Next, the processing of the control target 150 in this embodiment will be described with reference to FIG.

[0072] 5 is realized by the control unit 162 of the control target 150 controlling each component by executing a program stored in the storage unit 161. In the following, it is assumed that the development support device 100 and the control target 150 are connected so as to be able to communicate with each other.

[0073] In step S 501 , the control unit 162 receives a scenario from the development support device 100 via the communication unit 164 .

[0074] In step S502, the control unit 162 analyzes the program received in step S501, and obtains the execution method, the number of blocks, the configuration number of each block, the command value, and the condition value. The configuration number is a number indicating the ordinal number of each block. The command value and the condition value will be described in detail later.

[0075] In step S503, the control unit 162 determines the execution method based on the analysis result of step S502. The execution method is either the first execution method or the second execution method, and is transmitted together with the program. The execution method is determined to be the first execution method if the value of "executeType" in the execution method 301 is "1", and the second execution method if the value is "2". The scenario 300 in FIG. 3 indicates that the program 302 is executed by the second execution method. Furthermore, the control unit 162 stores the program received in step S501 and the analysis result of step S502 in the storage unit 161. In this case, the program is stored in a different area (ROM or RAM) of the storage unit 161 depending on the execution method.

[0076] If the control unit 162 determines that the execution method is the first execution method, the process proceeds to step S505. If the control unit 162 determines that the execution method is the second execution method, the process proceeds to step S507. In step S505, the control unit 162 stores the program in a non-volatile memory (ROM).

[0077] In step S506, the control unit 162 executes the program in the first execution mode.

[0078] In step S507, the control unit 162 temporarily stores the program in a volatile memory (RAM).

[0079] In step S508, the control unit 162 executes the program in the second execution mode.

[0080] Next, the process of executing the program in the second execution mode in step S508 of FIG. 5 will be described with reference to FIG.

[0081] The process of FIG. 6 is executed by designating the second execution method in the program that the development support device 100 transmits to the control target 150.

[0082] An example of processing for executing the program 302 in FIG. 3 using the second execution method in step S508 in FIG. 5 will be described below.

[0083] In step S601, control unit 162 executes the 0th block based on the block configuration number obtained by analyzing the program in step S502 in Fig. 5. In the case of program 302 of this embodiment, the configuration numbers obtained in step S502 are 0th block 303, 1st block 304, and 2nd block 305. In other words, 303, which is the 0th block, is executed.

[0084] In step S602, the control unit 162 determines the processing content of the block based on the command value and condition value of the block obtained in step S502. A command value is a value that each block necessarily has, and in the example of program 302 in FIG. 3, it corresponds to the value of "action" that indicates the processing content of the block. A condition value is a value that the user can set arbitrarily, and may or may not have it depending on the processing content of the block, and in the example of program 302 in FIG. 3, it corresponds to the values ​​of "faceid" and "angle". In the case of block 303, which has the 0th block configuration number in FIG. 3, the command value is "wait" and the condition value is "1", so it is determined that the processing content of the block is "wait until [mother] is found".

[0085] In step S603, the control unit 162 commands a block process of "wait until [mother] is found." That is, the control target 150 of this embodiment outputs a command to wait until it detects a person registered as "faceid": "1", that is, the mother.

[0086] In step S604, the control unit 162 determines whether or not a request to obtain the program execution status has been received from the development support device 100. If the control unit 162 determines that a request to obtain the program execution status has been received from the development support device 100, the control unit 162 proceeds to step S605. If the control unit 162 determines that a request to obtain the program execution status has not been received from the development support device 100, the control unit 162 proceeds to step S606.

[0087] In step S605, the control unit 162 transmits the program execution status to the development support device 100 via the communication unit 164. The control unit 162 transmits shooting status information including "executing block configuration number: 0, detected subject: none, captured image: 207" to the development support device 100. Based on this shooting status information, the development support device 100 displays the UI screen of FIG. 2B.

[0088] In step S606, the control unit 162 determines whether or not the condition for completing the block process, that is, the condition for "finding mother" is satisfied, and repeats the processes of steps S604 and S605 until the control target 150 detects the mother.

[0089] If the control unit 162 detects the mother, the process proceeds to step S607, and the block configuration number is incremented by 1. Since the block configuration number is now 1, the process ends execution of the 0th block in step S608, and the process proceeds to the first block 304. Returning to step S601, the control unit 162 executes the first block 304. In step S602, since the value of "action" in block 304 is "shoot", it is determined to be a "take a still image" block. Since this block does not have any parameters that can be arbitrarily set by the user, it does not have a condition value.

[0090] In step S603, the control unit 162 commands the block process of "take a still image."

[0091] In step S604, if the control unit 162 determines that an execution status acquisition request has been received from the development support device 100, in step S605, the communication unit 164 transmits shooting status information including "executing block configuration number: 1, detected subject: mother, shooting status: still image shot, captured image: 208" to the development support device 100. Based on this shooting status information, the development support device 100 displays the UI screen of Fig. 2C.

[0092] Since block 304 does not have a condition value, control unit 162 determines whether or not a still image has been taken in step S606. If control unit 162 determines that a still image has been taken, it adds 1 to the block configuration number in step S607. In other words, the block configuration number becomes 2 from 1, so execution of the first block ends in step S608 and the process proceeds to the second block 305.

[0093] Returning to step S 601 , the control unit 162 executes the second block 305 .

[0094] In step S602, the control unit 162 determines that the block 305 is a "turn right

[30] degrees" block since the "action" value of the block 305 is "right" and the condition value is "30".

[0095] In step S603, the control unit 162 commands the block process of "turn right by

[30] degrees." The control unit 162 outputs a control command to the lens barrel driving unit 158 ​​so that the control target 150 pans 30 degrees to the right from the current pan / tilt position.

[0096] In step S604, if the control unit 162 determines that a program execution status acquisition request has been received from the development support device 100, in step S605, the communication unit 164 transmits shooting status information including "executing block configuration number: 2, detected subject: none, PTZ position: 25 degrees right, captured image: 209" to the development support device 100. Based on this shooting status information, the development support device 100 displays the screen in Fig. 2D.

[0097] In step 606, the control unit 162 determines whether or not the condition for completing the block processing, that is, the condition of "panning to a position 30 degrees to the right" is satisfied, and this is repeated until the position 30 degrees to the right is reached.

[0098] If the control unit 162 determines that the position 30 degrees to the right has been reached, it adds 1 to the block configuration number in step S607. That is, the block configuration number becomes 3 from 2, and execution of the second block ends in step S608, but the loop condition in step S601 is less than the number of blocks, that is, less than 3. Therefore, the control unit 162 determines that this block is the last block, exits the loop from steps S601 to S608, and ends the process.

[0099] In addition, if it is determined in step S604 of FIG. 6 that a request to obtain execution status has not been received, a completion condition determination for the block processing is performed in step S606, but the block processing of steps S603 and S606 and the processing of steps S604 and S605 may be executed in parallel.

[0100] In the second execution method, as described in step S414 of FIG. 4B, the program is stored temporarily, and when the power supply of the control target 150 is turned off, the program is erased.

[0101] Next, the process of executing the program in the first execution mode in step S506 of FIG. 5 will be described with reference to FIG.

[0102] The process of FIG. 7 is executed by designating the first execution method in the program that the development support device 100 transmits to the control target 150.

[0103] An example of processing for executing the program 302 in FIG. 3 using the first execution method in step S506 in FIG. 5 will be described below.

[0104] Steps S601 to S603 and S606 to S608 performed in the first execution method are the same as those in FIG.

[0105] In the first execution method, the control unit 162 and the communication unit 164 do not execute the reception judgment of the execution status (S604) and the transmission of the execution status (S605) that were executed in the second execution method. In other words, in the first execution method, the development support device 100 does not display the execution status.

[0106] In the first execution method, unlike the second execution method, even if the power of the control target 150 is turned off, the program is instantly read out from the storage unit 161 and executed when the power of the control target 150 is turned on again.

[0107] As described above, according to this embodiment, in control program development, it is possible to support the user in creating a program by visualizing the program execution status of the control target 150. In detail, it is possible to visually confirm the execution status that cannot be determined only from the external operation of the control target 150, such as which block the control target 150 is executing, the subject detected by the control target 150, the pan / tilt position, and the captured image, so that it becomes possible to confirm whether the program being created is the expected program or not.

[0108] In this embodiment, the program is stored in a different area (ROM or RAM) depending on the specified execution method, but it may be stored in the same area. In that case, the execution method is stored together with the program.

[0109] Furthermore, in the present embodiment, the process of analyzing the program and then storing the analysis results has been described, but the program may be stored first and then analyzed later.

[0110] Furthermore, when an execution status acquisition request is received from the development support device 100 while the program is being executed in the first execution method, the control target 150 does not have to transmit any information, or may transmit error information.

[0111] In the first execution method, an example has been described in which the program is executed immediately when the power supply of the control target 150 is turned on, but the program may be executed by another trigger. For example, the program may be executed by the operation of a button provided on the control target 150.

[0112] [Embodiment 2] In the second embodiment, an example in which the second execution method of the first embodiment is improved will be described.

[0113] In the program 200 illustrated in Fig. 2A, a block such as "Take a still image" (202) takes less time to process than a block that performs condition determination based on a condition value such as "Wait until [Mom] is found" (201). Because of this, block processing progresses relatively quickly, making it difficult for the user to reliably visually check the execution status of each block being executed.

[0114] Therefore, in the second embodiment, a method for enabling a user to visually check the execution status of blocks one by one without fail will be described.

[0115] The configurations of the development support device 100 and the control target 150 in the second embodiment are the same as those in the first embodiment.

[0116] The control unit 162 determines whether or not a processing progress request has been received from the development support device 100, and controls the progress of processing of the block according to the result of the determination.

[0117] The process from when the control target 150 receives the program to when it determines the execution method is the same as that in FIG. 5, so the process after it is determined that the execution method is the second execution method will be described.

[0118] 8 and 9, a process for enabling a user to reliably visually check the execution status of blocks one by one will be described.

[0119] Fig. 8 is a flowchart illustrating a process of executing a program in the second execution mode in step S508 of Fig. 5 according to the second embodiment. Fig. 9 is a diagram illustrating a UI screen provided by the programming tool of the second embodiment.

[0120] The processes in steps S601 to S608 in FIG. 8 are similar to those in FIG.

[0121] The programming tool of this embodiment includes a button 900 for the user to request the progress of block processing, as shown in Fig. 9. When the user operates the button 900, the development support device 100 transmits a processing progress request to the control target 150.

[0122] In step S801, the control unit 162 determines whether or not a processing progress request has been received from the development support device 100. Even if it is determined in step S606 that the completion condition for the block processing has been satisfied, the control unit 162 will not proceed to the processing of the next block until it receives a processing progress request from the development support device 100.

[0123] In FIG. 8, after the completion condition determination of the block process in step S606, the reception determination of the process progress request in step S801 is performed, but these may be performed in parallel.

[0124] As described above, according to this embodiment, in control program development, a user can operate the buttons 900 of the development support device 100 to reliably visually check the program execution status of the control target 150 one by one.

[0125] [Embodiment 3] In the third embodiment, an example in which the processing in the second embodiment is further improved will be described.

[0126] In the second embodiment, a method for stopping all blocks while progressing through the processing of the blocks one by one has been described. In this case, in the case of a program consisting of many blocks, stopping each block one by one takes time, so it is assumed that you want to stop at a block several blocks ahead.

[0127] Therefore, in the third embodiment, a method of stopping at a block several blocks ahead specified by the user will be described.

[0128] The configurations of the development support device 100 and the control target 150 in the third embodiment are the same as those in the first embodiment.

[0129] The control unit 162 determines whether or not to stop processing of the block according to the stop position specified by the development support device 100, and controls the progress of processing of the block according to the result of the determination.

[0130] The process from when the control target 150 receives the program to when it determines the execution method is the same as that in FIG. 5, so the process after it is determined that the execution method is the second execution method will be described.

[0131] Fig. 10 is a diagram illustrating a UI screen provided by a programming tool according to embodiment 3. Fig. 11 is a flowchart illustrating a process of executing a program in the second execution mode in step S508 of Fig. 5 according to embodiment 3.

[0132] In the development support device 100 of this embodiment, the user can specify a block at which the user wishes to stop. More specifically, the user can specify a block at which the user wishes to stop by clicking on any block of the program in the operation area 212 of FIG. 10(a). In the example of FIG. 10(a), it is assumed that while block 201 is being executed, the user clicks on block 203 to specify it as the stop position. In this case, the control unit 162 executes blocks 201 and 202, and then the development support device 100 highlights block 203 but does not execute it, and stops in the state of FIG. 10(b). Thereafter, when the user operates the button 1000, the control unit 162 starts executing the processing of block 203.

[0133] Next, the processing of the control target 150 in this embodiment will be described with reference to FIG.

[0134] The processes in steps S601 to S608 in FIG. 11 are similar to those in FIG.

[0135] In step S1101, the control unit 162 determines whether or not to stop immediately before executing the next block based on the block configuration number of the stop position specified in the development support device 100, and if it determines that the block to stop at is the next block, it stops immediately before the next block in step S1102.

[0136] In step S1103, the control unit 162 repeats the processes of steps S1101 and S1102 and continues to suspend the blocks until a process progress request is received by the user operating the button 1000.

[0137] As described above, according to this embodiment, in control program development, by operating the button 1000 of the development support device 100, the user can not only reliably check the program execution status of the control target 150 one by one, but also specify multiple future blocks that the user wishes to stop.

[0138] In the present embodiment, an example has been described in which the blocks on the way to the block designated as the stop position are executed, but it is also possible to stop at the designated block without executing the blocks on the way.

[0139] [Other embodiments] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0140] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention.

[0141] The disclosure of this specification includes the following information processing device, control device, control method, and program. [Configuration 1] An information processing device that creates a program to be executed by a control device, a transmission means for transmitting the program and an execution method of the program to the control device; When the execution method of the program is the first execution method, the request for acquiring the execution status of the program is not transmitted to the control device; and a control means for, when the execution method of the program is a second execution method different from the first execution method, sending a request to obtain the execution status of the program to the control device and displaying the execution status of the program received from the control device. [Configuration 2] a programming tool for visually creating the program and specifying an execution method for the program; the first execution method is an execution method that enables the control device to execute the program, 2. The information processing device according to configuration 1, wherein the second execution method is an execution method capable of displaying an execution status of a program being executed in the control device. [Configuration 3] The program includes at least one processing unit, 3. The information processing device according to configuration 1 or 2, wherein the processing unit includes the content of the processing to be executed. [Configuration 4] 4. The information processing device according to configuration 3, wherein the processing unit includes a condition for completing the processing. [Configuration 5] 5. The information processing device according to configuration 3 or 4, wherein the control means displays the execution status of the program for each processing unit received from the control device. [Configuration 6] 5. The information processing device according to configuration 3 or 4, wherein the control means transmits a request to the control device to execute a next processing unit when the processing unit being executed in the control device is completed. [Configuration 7] 5. The information processing device according to configuration 3 or 4, wherein the control means designates a processing unit whose execution is to be stopped among a plurality of processing units and transmits the designated processing unit to the control device. [Configuration 8] A control device capable of communicating with an information processing device, a receiving means for receiving a program and an execution method of the program from the information processing device; a control means for executing the program based on the execution method; The control device, characterized in that the control means transmits the execution status of the program to the information processing device in response to receiving a request for acquiring the execution status of the program from the information processing device. [Configuration 9] The control device according to configuration 8, wherein the program execution method includes a first execution method capable of executing the program and a second execution method capable of transmitting the execution status of the program to the information processing device. [Configuration 10] A storage means for storing the program, The control means In the case of the first execution method, the program is stored so as not to be erased when the power is turned off; 10. The control device according to configuration 9, wherein in the second execution method, the program is temporarily stored so as to be erased when the power is turned off. [Configuration 11] The program includes at least one processing unit, 11. The control device according to any one of configurations 8 to 10, wherein the processing unit includes the content of the processing to be executed. [Configuration 12] 12. The control device according to claim 11, wherein the processing unit includes a condition for completing the processing. [Configuration 13] The control device according to configuration 11 or 12, characterized in that the control means transmits the execution status of the program for each processing unit to the information processing device in response to receiving a request to obtain the execution status of the program from the information processing device. [Configuration 14] an analysis means for analyzing the program; The analysis means acquires the execution method of the program, the number of processing units constituting the program, configuration numbers, processing contents, and processing completion conditions; 13. The control device according to claim 12, wherein the control means determines an execution method of the program based on a result of the analysis by the analysis means. [Configuration 15] The control means Executing the processing unit; determining whether a completion condition for the processing unit is satisfied; 15. The control device according to any one of configurations 12 to 14, characterized in that when the completion condition is satisfied, the control device controls so as to execute a next processing unit. [Configuration 16] The control means The control device according to configuration 15, characterized in that even if a completion condition for the processing unit is satisfied, the control device controls so as not to execute the next processing unit until a request to execute the next processing unit is received from the information processing device. [Configuration 17] The control means When the completion condition of the processing unit is satisfied, it is determined whether the next processing unit is designated as the processing unit to be stopped; 16. The control device according to configuration 15, wherein, when the next processing unit is designated as the processing unit to be stopped, execution of the designated processing unit is stopped. [Configuration 18] A method for controlling an information processing device that creates a program to be executed by a control device, comprising: transmitting the program and an execution method of the program to the control device; When the execution method of the program is the first execution method, the request for acquiring the execution status of the program is not transmitted to the control device; If the execution method of the program is a second execution method different from the first execution method, a step of sending a request to the control device to obtain the execution status of the program and displaying the execution status of the program received from the control device. [Configuration 19] A method for controlling a control device capable of communicating with an information processing device, comprising: receiving a program and an execution method of the program from the information processing device; Executing the program based on the execution method; The control method according to claim 1, wherein the executing step includes transmitting the execution status of the program to the information processing device in response to receiving a request for obtaining the execution status of the program from the information processing device. [Configuration 20] A program for causing a computer to function as the information processing device according to any one of claims 1 to 7. [Configuration 21] A program for causing a computer to function as the control device according to any one of configurations 8 to 17. [Explanation of symbols]

[0142] 100... development support device, 101... control unit, 108... communication IF, 150... control target, 162... control unit, 164... communication unit

Claims

1. An information processing device that creates a program to be executed by a control device, a transmission means for transmitting the program and an execution method of the program to the control device; When the execution method of the program is the first execution method, the request for acquiring the execution status of the program is not transmitted to the control device; and a control means for, when the execution method of the program is a second execution method different from the first execution method, sending a request to obtain the execution status of the program to the control device and displaying the execution status of the program received from the control device.

2. a programming tool for visually creating the program and specifying an execution method for the program; the first execution method is an execution method that allows the control device to execute the program, 2. The information processing apparatus according to claim 1, wherein the second execution method is an execution method capable of displaying an execution status of a program being executed in the control device.

3. The program includes at least one processing unit, 2. The information processing apparatus according to claim 1, wherein the processing unit includes the content of a process to be executed.

4. 4. The information processing apparatus according to claim 3, wherein the processing unit includes a condition for completing the processing.

5. 4. The information processing apparatus according to claim 3, wherein said control means displays the execution status of the program for each processing unit received from said control device.

6. 4. The information processing apparatus according to claim 3, wherein said control means transmits a request to said control apparatus to execute a next processing unit when a processing unit being executed in said control apparatus is completed.

7. 4. The information processing apparatus according to claim 3, wherein said control means designates one of a plurality of processing units whose execution is to be stopped and transmits this to said control device.

8. A control device capable of communicating with an information processing device, a receiving means for receiving a program and an execution method of the program from the information processing device; a control means for executing the program based on the execution method; The control device, characterized in that the control means transmits the execution status of the program to the information processing device in response to receiving a request for acquiring the execution status of the program from the information processing device.

9. 9. The control device according to claim 8, wherein the program execution method includes a first execution method capable of executing the program, and a second execution method capable of transmitting the execution status of the program to the information processing device.

10. A storage means for storing the program, The control means In the case of the first execution method, the program is stored so as not to be erased when the power is turned off; 10. The control device according to claim 9, wherein, in the second execution method, the program is temporarily stored so as to be erased when the power is turned off.

11. The program includes at least one processing unit, 9. The control device according to claim 8, wherein the processing unit includes the content of the processing to be executed.

12. The control device according to claim 11 , wherein the processing unit includes a condition for completing the processing.

13. 12. The control device according to claim 11, wherein the control means transmits the program execution status for each processing unit to the information processing device in response to receiving a request for acquiring the program execution status from the information processing device.

14. an analysis means for analyzing the program; The analysis means acquires the execution method of the program, the number of processing units constituting the program, configuration numbers, processing contents, and processing completion conditions; 13. The control device according to claim 12, wherein the control means determines an execution method of the program based on a result of the analysis by the analysis means.

15. The control means Executing the processing unit; determining whether a completion condition for the processing unit is satisfied; The control device according to claim 12, further comprising control for executing a next processing unit when the completion condition is satisfied.

16. The control means 16. The control device according to claim 15, further comprising control for preventing the next processing unit from being executed until a request for executing the next processing unit is received from the information processing device, even if a completion condition for the processing unit is satisfied.

17. The control means When the completion condition of the processing unit is satisfied, it is determined whether the next processing unit is designated as the processing unit to be stopped; 16. The control device according to claim 15, wherein, when the next processing unit is designated as the processing unit to be stopped, the control device stops execution of the designated processing unit.

18. A method for controlling an information processing device that creates a program to be executed by a control device, comprising: transmitting the program and an execution method of the program to the control device; When the execution method of the program is the first execution method, the request for acquiring the execution status of the program is not transmitted to the control device; If the execution method of the program is a second execution method different from the first execution method, a step of sending a request to the control device to obtain the execution status of the program and displaying the execution status of the program received from the control device.

19. A method for controlling a control device capable of communicating with an information processing device, comprising: receiving a program and an execution method of the program from the information processing device; Executing the program based on the execution method; The control method according to claim 1, wherein the executing step includes transmitting the execution status of the program to the information processing device in response to receiving a request for obtaining the execution status of the program from the information processing device.

20. A program for causing a computer to function as the information processing device according to any one of claims 1 to 7.

21. A program for causing a computer to function as the control device according to any one of claims 8 to 17.

Images