Video control system and video control method

Abstract

To allow video of a desired moment at a good frame rate to be captured while suppressing communication traffic.SOLUTION: A video control system according to the present application includes a reception unit that receives the setting of definition of an important scene in video captured by a camera, an estimation unit that estimates whether or not the defined important scene will be captured while the camera is capturing the video, and a frame rate change unit that changes the frame rate of the camera depending on whether the scene is important or unimportant.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a video control system and a video control method. [Background technology]

[0002] A widely known method is to vary the "frame rate," which indicates how many times a camera shutter is released per second, depending on the amount of movement of the subject in the image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2024-011715 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional methods, if the movement of the object is large in unimportant scenes and small in important scenes, this leads to an unnecessary increase in communication load in unimportant scenes, and makes it difficult to capture subtle changes in the object in important scenes.

[0005] The present application has been made in view of the above, and aims to make it possible to capture video of a desired moment at a good frame rate while suppressing communication traffic. [Means for solving the problem]

[0006] The video control system of the present application is characterized by comprising a reception unit that receives the setting of a definition of an important scene in a video captured by a camera, an estimation unit that estimates whether or not a defined important scene will be captured during shooting by the camera, and a frame rate change unit that changes the frame rate of the camera depending on whether the scene is important or unimportant. [Effects of the Invention]

[0007] According to one aspect of the embodiment, it is possible to capture video of a desired moment at a good frame rate while suppressing communication traffic. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a video control system according to an embodiment. [Figure 2] FIG. 2 shows the time series during processing and the frame rate at that time. [Figure 3] Figure 3 shows a table of examples of important scenes. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of the control server according to the embodiment. [Figure 5] FIG. 5 is a flowchart illustrating an example of the processing flow of the video control system according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] The video control system 1 according to the embodiment described below includes a receiving unit 332b, an estimation unit 332c, and a frame rate changing unit 332d. The receiving unit 332b receives a setting for the definition of an important scene in video captured by the camera 10. The estimation unit 332c estimates whether the defined important scene will be captured during shooting by the camera 10. The frame rate changing unit 332d changes the frame rate of the camera 10 depending on whether the scene is important or unimportant.

[0010] In addition, in the video control system 1 according to the embodiment described below, the frame rate change unit 332d changes the frame rate at which the camera 10 captures images to a frame rate equal to or higher than a predetermined threshold value when it is estimated that an important scene will be captured.

[0011] In addition, in the video control system 1 according to the embodiment described below, the frame rate change unit 332d changes the frame rate at which the camera 10 captures images to a frame rate below a predetermined threshold when it is estimated that an unimportant scene will be captured.

[0012] Furthermore, in the video control system 1 according to the embodiment described below, the frame rate change unit 332d changes the frame rate at which the camera 10 shoots to a higher frame rate when it is estimated that an unimportant scene will be shot, and changes the frame rate at which the camera 10 shoots to a lower frame rate when it is estimated that an unimportant scene will be shot.

[0013] Furthermore, in the video control system 1 according to the embodiment described below, the estimation unit 332c estimates whether or not an important scene will be captured in the video on the factory equipment side based on sensor information that detects the operation of the machine on the factory equipment side or the part to be processed.

[0014] Furthermore, in the video control system 1 according to the embodiment described below, the estimation unit 332c divides the operation program that controls the operation of the machinery on the factory equipment side into process steps, and estimates whether or not an important scene will be captured based on which process step is being performed.

[0015] In addition, in the video control system 1 according to the embodiment described below, the estimation unit 332c estimates whether an important scene will be filmed based on whether an action defined as an important scene among the actions listed in the itinerary is performed according to the timeline of the itinerary that has been determined in advance.

[0016] In addition, in the video control system 1 according to the embodiment described below, the receiving unit 332b receives a question asking whether a scene is important in VQA (Visual Question Answering) as a definition of an important scene. The estimation unit 332c uses VQA to evaluate the video in response to the question and estimates whether an important scene is captured.

[0017] Hereinafter, a video control system 1 according to an embodiment will be described with reference to the drawings. The same components in the embodiments are denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0018] FIG. 1 is a diagram illustrating an example of the configuration of a video control system according to an embodiment. As illustrated in FIG. 1, the video control system 1 includes a camera 10, a repeater 20, a control server 30, and a work terminal 40. The camera 10 is an imaging device with a photographing function. Alternatively, the camera 10 may be a camera-equipped LED light integrated with a lighting fixture. The repeater 20 is a relay device such as a router that relays communications. Alternatively, the repeater 20 may be a base station or the like. The control server 30 is a server device disposed on a network. The work terminal 40 is a terminal device such as a personal computer (PC) used by a user. Alternatively, the work terminal 40 may be a smart device such as a smartphone or tablet terminal. In reality, the user operating the work terminal 40 may be a worker, operator, work manager, supervisor, or the like.

[0019] Furthermore, the cameras 10 and the repeaters 20, the repeaters 20 and the control server 30, and the control server 30 and the work terminal 40 are connected wirelessly or by wire to enable direct or indirect communication. There may be a plurality of cameras 10 and a plurality of repeaters 20. The video control system 1 may be configured without the repeater 20 by directly connecting the cameras 10 and the control server 30. The control server 30 is installed on the cloud 2 and is connected to each device via a network such as a LAN (Local Area Network) or the Internet. The control server 30 may be configured by a plurality of servers on the cloud 2.

[0020] Furthermore, the control server 30 controls the operation of the camera 10 and machines in the factory equipment (production equipment) via the repeater 20 (or other communication path). For example, the control server 30 changes the frame rate of the camera 10. The control server 30 also controls the operation of the belt conveyor BC, the robot arm RA, and a machine tool MT different from the robot arm RA shown in FIG. 1. Furthermore, the control server 30 may control the operation (timing, position, number, etc.) of placing parts AS to be machined on the belt conveyor BC. In reality, the camera 10 and the machines in the factory equipment have the functions necessary for operation control, and may operate independently (under autonomous control) without going through the control server 30.

[0021] In this embodiment, a scene in which equipment is operating in a factory is assumed. In Fig. 1, a camera 10 captures the operation of machinery in the factory equipment. As shown in Fig. 1, the video (video) capturing the scene shows (1) a belt conveyor BC operating, (2) a robot arm RA operating, (3) a machine tool MT that is different from the robot arm RA that processes a part AS, and (4) the part AS being processed.

[0022] The belt conveyor BC is constantly moving, and parts AS flow on the belt conveyor BC, but even when parts AS are not visible within the field of view, they always flow regularly at regular intervals, with a certain distance between them. Parts AS are processed by the machine tool MT when they are visible within the field of view.

[0023] In the conventional method, the conveyor belt BC is always moving, so it is always filmed at a high frame rate. For example, even in an "unimportant scene" where parts AS are simply moving, the conveyor belt BC in the video is still moving, so it is filmed at a high frame rate.

[0024] Therefore, in this embodiment, by shooting "unimportant scenes" at a low frame rate and shooting only "important scenes" at a high frame rate, it is possible to reliably capture important scenes while reducing communication traffic. Note that the low frame rate may be set as the default (standard state, initial setting) frame rate. Also, a frame rate equal to or greater than a predetermined threshold may be set as a high frame rate, and a frame rate below the predetermined threshold may be set as a low frame rate.

[0025] The trigger for changing the frame rate is determined by whether or not it is an "important scene." Figure 2 shows the timeline of processing and the frame rates at that time. For example, as shown in Figure 2, the period from when part AS starts moving on the belt conveyor BC until processing of part AS begins (from after "part movement start" to before "part processing start") is an "unimportant scene" and a low frame rate is used; the period from when processing of part AS begins until processing ends (from after "part processing start" to "part processing end") is considered an "important scene" and a higher frame rate is used; and the period after processing ends (after "part processing end") is considered an "unimportant scene" and a lower frame rate is used.

[0026] In this embodiment, the definition of an "important scene" is when part AS is being processed. The server device 100 determines the start and end of processing based on an operation program created in advance. Alternatively, the server device 100 acquires trigger information for the start and end of processing by receiving, from the factory equipment, sensor information (operation information, operating information, etc.) that detects the operation of machines in the factory equipment and sensor information that detects the part to be processed.

[0027] Here, when determining the start and end of processing based on an operation program, for example, the server device 100 divides the operation program of the factory equipment into processes (or steps, or codes), and executes it by having the user input the processes from the start to the end of part processing using pull-down menus or check boxes. For example, when the server device 100 divides the operation program of the factory equipment into processes, it realizes this using a general-purpose GPT (Generative Pre-trained Transformer) in which the input value is the source code for operating the factory equipment and the output value is a sentence succinctly expressing each process. GPT is a text generation AI (Artificial Intelligence) and a language model capable of generating sentences using natural language processing.

[0028] In this way, the server device 100 divides the operation program into steps and presents them to the user via the work terminal 40. The user is then asked to register the range of high-rate (high frame rate) recording using a pull-down menu (or check boxes). Note that the trigger may not only be the range actually recorded at high rate, but also the next earlier range. The server device 100 receives information about which line of source code is active from devices (factory equipment machines, sensors, monitoring devices, etc.) and operates the camera 10 based on that information.

[0029] Additionally, the server device 100 can also determine important scenes by VQA from the video captured by the camera 10. For example, the server device 100 can receive a question such as "Does the factory equipment process parts with XX characteristics?" in advance from a user, and use VQA trained on a high-quality data set prepared in advance to evaluate each captured video in response to this question, thereby determining whether the condition is met as true or false, thereby identifying "important scenes" in real time.

[0030] When VQA is used, it is basically performed using one or more sentences. For example, the server device 100 may determine whether "an action is likely to be performed" using VQA, and if it determines that an action is likely to be performed, start shooting at a high rate. Alternatively, the server device 100 may make a determination using VQA using sentences that detect precursory actions.

[0031] Furthermore, in this embodiment, an example is described in which a scene in which equipment operates in a factory is filmed. However, other examples of important scenes that can be used as triggers include weddings, concerts, drama filming, etc. In this case, the filming is performed using a camera 10 installed at a wedding hall, concert venue, filming location, etc. The camera 10 may be a camera-equipped LED light. Based on the video captured by the camera 10, the server device 100 determines the start and end of an important scene (or the start and end of an unimportant scene) and uses this as a trigger to change the frame rate of the camera 10.

[0032] FIG. 3 is a table showing examples of important scenes. For example, as shown in FIG. 3, if the expected scene is a "factory," an important scene is "when parts are processed," and an unimportant scene is "when parts are transported." Furthermore, the server device 100 estimates (determines) whether a scene is important based on "sensor information from factory equipment" or "each process of the operation program" as the trigger determination source. Note that, when determining an important scene by VQA from the video captured by the camera 10, the server device 100 may use a question such as "Does the factory equipment process parts with XX characteristics?" as a question for VQA.

[0033] Furthermore, if the expected scene is a "wedding ceremony," an important scene is "the bride and groom," and an unimportant scene is "the movement of guests." The server device 100 estimates (determines) whether a scene is important based on whether an action corresponding to an important scene is occurring according to a "timeline based on a pre-determined schedule" as the trigger determination source. The server device 100 estimates which action is occurring among the actions listed in the schedule through image analysis, and switches the frame rate if the estimated action is a pre-determined action. When determining an important scene from the video captured by the camera 10 using VQA, the server device 100 may use a question such as "Are the bride and groom in the picture?" as a question for VQA.

[0034] Furthermore, if the expected scene is a "concert," important scenes are "when the singer sings" or "when the musicians play," and unimportant scenes are "between songs." The server device 100 estimates (determines) whether a scene is important based on whether an action corresponding to an important scene is occurring according to a "timeline based on a pre-determined schedule" as the trigger determination source. The server device 100 estimates which action is occurring among the actions listed in the schedule through image analysis, and switches the frame rate if the estimated action is a pre-determined action. When determining important scenes from the video captured by the camera 10 using VQA, the server device 100 may use questions such as "Is the singer performing?" or "Are the musicians performing?" as questions for VQA.

[0035] Furthermore, if the expected scene is "filming a drama," an important scene is "a scene or line related to the story or characters," and an unimportant scene is "a landscape depiction that does not affect the outline." Furthermore, the server device 100 estimates (determines) whether a scene is important based on whether an action corresponding to an important scene is occurring according to a "chronological order based on a pre-determined schedule" as the trigger determination source. The server device 100 estimates which action is occurring among the actions listed in the schedule through image analysis, and if the estimated action is a pre-determined action, switches the frame rate. Note that when determining an important scene using VQA from the video captured by the camera 10, the server device 100 may use a question such as "Are actors visible?" as a question for VQA.

[0036] However, since the scenes that are important to the viewer of the video may change, the above are merely examples and are not limited to these examples.

[0037] Furthermore, as a secondary effect of this embodiment, for example, when factory equipment stops due to abnormal operation, the tendency can be clearly captured at a high frame rate, and therefore, by using AI that has learned from the high frame rate video captured by the camera 10 to predict the tendency of abnormal situations, it becomes possible to carry out maintenance in advance.

[0038] In addition to sending information unilaterally from the factory equipment to the camera 10, if the operation of the machine tool MT that processes the part AS becomes slow due to aging or other reasons, the movement of the machine tool MT can be captured at a high frame rate and the information can be sent from the camera 10 to the factory equipment, making corrections.

[0039] Here, the method of calculating the machining time of the machine tool MT using the camera 10 can be realized, for example, by recording the time from when machining begins to when machining ends with the camera 10. In conventional methods, the frame rate is increased even for "unimportant scenes" if there is movement, and it is considered that the frame rate is limited even for "important scenes" due to communication volume considerations, but this embodiment makes it possible to reliably capture important scenes at a high frame rate, and it is also possible to relatively improve the effect on secondary aspects compared to conventional techniques.

[0040] An example of the configuration of the control server 30 will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the configuration of the control server according to the embodiment. As shown in Fig. 4, the control server 30 includes a communication unit 31, a storage unit 32, and a control unit 33.

[0041] The communication unit 31 is realized by, for example, a network interface card (NIC), etc. The communication unit 31 is also connected to each device so as to be able to communicate with them via a network N (see FIG. 4) such as a LAN or the Internet.

[0042] The storage unit 32 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as an HDD, an SSD, or an optical disk. The storage unit 32 may store captured image data, acquired sensor information, a factory work schedule, etc. In other words, the storage unit 32 can store output data from the camera 10 and factory equipment.

[0043] The control unit 33 is a controller, and is realized by, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like, executing various programs (corresponding to an example of an information processing program) stored in a storage device inside the control server 30 in a storage area such as a RAM. In the example shown in FIG. 4, the control unit 33 has an object control unit 331 and a video control unit 332.

[0044] The object control unit 331 includes an operation control unit 331a, an operation setting unit 331b, and an information input / output unit 331c.

[0045] The operation control unit 331a controls the operation of the factory equipment machines in accordance with the operation program. That is, the operation control unit 331a controls the operation of the factory equipment machines that are captured as targets by the camera 10 and that perform the main operations in the video.

[0046] The operation setting unit 331b determines the operation of the factory equipment machines in advance. That is, the operation setting unit 331b sets in advance the details of the operation control by the operation control unit 331a. The operation setting unit 331b may also receive and manage the input and settings of the operation program.

[0047] The information input / output unit 331c inputs and outputs information to and from the factory equipment. For example, the information input / output unit 331c receives, from the factory equipment, sensor information that detects the operation of machines in the factory equipment and sensor information that detects parts to be processed. The information input / output unit 331c also receives input from an operator and instructions (commands) from the work terminal 40, and outputs information related to the operation of machines in the factory equipment. The information input / output unit 331c also inputs and outputs information between the object control unit 331 and the video control unit 332. For example, the information input / output unit 331c inputs information from the video control unit 332 to the operation setting unit 331b, and outputs information from the operation setting unit 331b to the video control unit 332. The information input / output unit 331c may be linked to or integrated with each of the operation control unit 331a and the operation setting unit 331b.

[0048] In reality, the target control unit 331 may be independent of the control server 30, may be provided in a server device separate from the control server 30, or may be provided in each piece of factory equipment.

[0049] The video control unit 332 includes a shooting control unit 332a, a reception unit 332b, an estimation unit 332c, a frame rate change unit 332d, and an information input / output unit 332e.

[0050] The photographing control unit 332a controls the camera 10 to photograph an object. For example, the photographing control unit 332a photographs a machine in a factory facility or a part being processed as an object using the camera 10. The photographing control unit 332a may also control the position, angle, function, etc. of the camera 10.

[0051] The receiving unit 332b receives a setting of a definition of an important scene in a video captured by the camera 10. For example, the receiving unit 332b receives, from the work terminal 40, a designation of an important scene that will trigger a change in the frame rate. At this time, the receiving unit 332b may receive information (definition information) regarding the definition of an important scene as the designation of the important scene. Alternatively, the receiving unit 332b may display a UI (User Interface) for designating an important scene on the work terminal 40 and receive the designation of an important scene through an operation or input from the user via this UI. Alternatively, the receiving unit 332b may receive, as the setting of the definition of an important scene, a question asking whether the scene is important for VQA. For example, the question asking whether a specific object (including a person) or a specific action is captured, or whether there is a sign that such an object or action is likely to be captured, may be a question asking whether the scene is important for VQA.

[0052] The estimation unit 332c estimates whether a defined important scene will be captured during shooting by the camera 10. For example, the estimation unit 332c estimates whether a specified (defined) important scene will be captured based on the video captured by the camera 10 through image analysis, image recognition, or the like. Alternatively, the estimation unit 332c estimates whether an important scene will be captured in the video of the factory equipment based on sensor information detecting the operation of the factory equipment machine or the parts to be processed. Alternatively, the estimation unit 332c divides an operation program controlling the operation of the factory equipment machine into steps and estimates whether an important scene will be captured based on which step is being performed. Alternatively, the estimation unit 332c may estimate whether an important scene will be captured based on whether an action defined as an important scene is performed among actions listed in a predetermined itinerary according to the timeline of the itinerary. Alternatively, the estimation unit 332c may use VQA to evaluate the video in response to a question asking whether the scene is an important scene, and estimate whether an important scene will be captured.

[0053] The frame rate change unit 332d changes the frame rate of the camera 10 depending on whether the scene is important or unimportant. For example, when it is estimated that an important scene will be captured, the frame rate change unit 332d sets the frame rate at which the camera 10 captures images to a high frame rate equal to or higher than a predetermined threshold. On the other hand, when it is estimated that an unimportant scene will be captured, the frame rate change unit 332d sets the frame rate at which the camera 10 captures images to a low frame rate below the predetermined threshold.

[0054] The information input / output unit 332e inputs and outputs information to and from the camera 10. For example, the information input / output unit 332e transmits control information for controlling the operation of the camera 10 and setting information such as the frame rate of the camera, and receives video images (captured moving images) acquired by the camera 10. The information input / output unit 332e also receives input from an operator or instructions (commands) from the work terminal 40, and outputs information related to the operation of the camera 10 and the acquired video images. The information input / output unit 332e also inputs and outputs information related to the operation of the machine of the factory equipment from the object control unit 331, and outputs information related to the operation of the machine of the factory equipment to the object control unit 331. Furthermore, the information input / output unit 332e may be linked to or integrated with each of the shooting control unit 332a, the reception unit 332b, the estimation unit 332c, and the frame rate change unit 332d.

[0055] In reality, the video control unit 332 may be independent of the control server 30, may be provided in a server device other than the control server 30, or may be provided in the camera 10 itself.

[0056] Next, an example of the processing flow of the video control system 1 according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart illustrating an example of the processing flow of the video control system according to this embodiment.

[0057] First, the operation control unit 331a of the control server 30 controls the operation of the machinery in the factory equipment in accordance with the settings made by the operation setting unit 331b and the operation program (step S101).

[0058] Furthermore, the imaging control unit 332a of the control server 30 controls the camera 10 to capture images of the factory equipment machines and parts being processed as the target (step S102). That is, the camera 10 captures the target. Unless there is a particular instruction to stop, the camera 10 continues capturing images. Furthermore, the information input / output unit 332e of the control server 30 may continue to output the images captured by the camera 10 to the work terminal 40 in real time.

[0059] Next, the reception unit 332b of the control server 30 receives, from the work terminal 40, the setting of the definition of an important scene in the video captured by the camera 10 (step S103).

[0060] Next, the estimation unit 332c of the control server 30 estimates whether or not the defined important scene will be captured during shooting by the camera 10 (step S104).

[0061] For example, the estimation unit 332c estimates whether a predetermined important scene will be captured by image analysis based on the video captured by the camera 10. Alternatively, the estimation unit 332c estimates whether an important scene will be captured in the video of the factory equipment based on sensor information detecting the operation of the factory equipment's machines or the parts to be processed. Alternatively, the estimation unit 332c divides an operation program controlling the operation of the factory equipment's machines into steps and estimates whether an important scene will be captured based on which step is being performed. Furthermore, the estimation unit 332c may estimate whether an important scene will be captured based on whether an action defined as an important scene is performed among actions listed in a predetermined itinerary according to the timeline of the itinerary. Furthermore, the estimation unit 332c may use VQA to evaluate the video in response to a question asking whether the scene is important, and estimate whether an important scene will be captured.

[0062] At this time, if it is estimated that an important scene will be captured (step S104; Yes), the frame rate change unit 332d of the control server 30 sets the frame rate at which the camera 10 captures the image to a high frame rate equal to or higher than a predetermined threshold (step S105).

[0063] Furthermore, if it is estimated that an unimportant scene will be captured (step S104; No), the frame rate change unit 332d sets the frame rate at which the camera 10 captures images to a low frame rate that is less than a predetermined threshold (step S106).

[0064] Next, the photography control unit 332a of the control server 30, in cooperation with the operation control unit 331a, determines whether or not to continue photography by the camera 10 (step S107).

[0065] At this time, if the shooting control unit 332a determines that shooting with the camera 10 should be continued (step S107; Yes), the estimation unit 332c of the control server 30 estimates whether the defined important scene will be shot during subsequent shooting by the camera 10 (return to step S104). Note that the frame rate change unit 332d maintains a high frame rate while shooting an important scene, and switches to a low frame rate when switching from an important scene to an unimportant scene.

[0066] Furthermore, if the photography control unit 332a does not determine that photography by the camera 10 should continue (step S107; No), the information input / output unit 332e of the control server 30 brings photography by the camera 10 to an end and outputs the video captured by the camera 10 (step S108). For example, the photography control unit 332a determines that photography by the camera 10 should come to an end when a series of tasks at a factory is completed. The photography control unit 332a converts the video captured by the camera 10 into data and saves it. The information input / output unit 332e outputs the video data to the work terminal 40. The user checks the video data on the work terminal 40.

[0067] As described above, by linking with pre-set information, it is possible to clearly identify important scenes, thereby reducing communication volume while making it possible to capture video of desired moments at a good frame rate.

[0068] [Variations] In the above embodiment, the control server 30 may be configured to accept a setting for the definition of an unimportant scene in addition to an important scene in the video captured by the camera 10. Then, when it is estimated that an unimportant scene will be captured, the control server 30 may change the frame rate used when capturing images by the camera 10 to a frame rate lower than the frame rate in the standard state. For example, the control server 30 may change the frame rate to a higher frame rate than the standard state when it is estimated that an important scene will be captured, change the frame rate to a lower frame rate than the standard state when it is estimated that an unimportant scene will be captured, and change the frame rate to the standard state in all other cases.

[0069] Furthermore, in the above embodiment, the control server 30 may be configured to accept a setting of the importance of each scene (or subject) to be photographed as a setting for defining an important scene. Then, when it is estimated that each scene will be photographed, the control server 30 may change the frame rate used by the camera 10 when photographing to a frame rate that corresponds to the importance of that scene. In this case, the control server 30 changes the frame rate to a higher value as the importance of the scene to be photographed increases, and changes the frame rate to a lower value as the importance of the scene to be photographed decreases. In other words, the control server 30 may change the frame rate in stages according to the importance of the scene to be photographed.

[0070] Although an embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment is included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as set forth in the claims. Furthermore, this embodiment can be appropriately combined within the scope of the processing content. [Explanation of symbols]

[0071] 1. Video control system 10 Camera 20 Repeater 30 Control Server 31 Communications Department 32 Storage section 33 Control Unit 331 Target Control Unit 331a Operation control section 331b Operation setting section 331c Information input / output section 332 Video control section 332a Imaging control unit 332b Reception 332c Estimation Department 332d Frame rate change section 332e Information input / output section 40 Work Station

Claims

1. a reception unit that receives settings for defining important scenes in video captured by a camera; an estimation unit that estimates whether a defined important scene will be captured during shooting by the camera; a frame rate change unit that changes the frame rate of the camera depending on whether the scene is important or unimportant; A video control system comprising:

2. The frame rate change unit changes the frame rate at which the camera captures images to a frame rate equal to or higher than a predetermined threshold value when it is estimated that an important scene will be captured.

2. The video control system according to claim 1.

3. The frame rate change unit changes the frame rate at which the camera captures images to a frame rate below a predetermined threshold when it is estimated that an unimportant scene will be captured.

2. The video control system according to claim 1.

4. The frame rate change unit changes the frame rate at which the camera takes pictures to a higher frame rate when it is estimated that an unimportant scene will be shot, and changes the frame rate at which the camera takes pictures to a lower frame rate when it is estimated that an unimportant scene will be shot.

2. The video control system according to claim 1.

5. The estimation unit estimates whether an important scene will be captured in the video of the factory equipment based on sensor information that detects the operation of a machine on the factory equipment side or a part to be processed.

2. The video control system according to claim 1.

6. The estimation unit divides an operation program for controlling the operation of a machine on the factory equipment side into steps, and estimates whether an important scene will be captured based on which step is being performed.

2. The video control system according to claim 1.

7. The estimation unit estimates whether an important scene will be captured based on whether an action defined as an important scene among actions described in the itinerary is performed along a timeline of the itinerary that is determined in advance.

2. The video control system according to claim 1.

8. the reception unit receives a question asking whether a scene is important for VQA as a definition of the important scene; The estimation unit determines whether an important scene is captured in the video by using VQA for the question.

2. The video control system according to claim 1.

9. A video control method executed by an information processing device, A receiving step of receiving a setting of a definition of an important scene in an image captured by a camera; an estimation step of estimating whether a defined important scene will be captured during shooting by the camera; a frame rate changing step of changing the frame rate at which the camera captures an image to a frame rate equal to or higher than a predetermined threshold value when it is estimated that an important scene will be captured; A video control method comprising:

Images