Weather situation visualization system, control terminal and its program, and weather situation video generation device and its program

Abstract

We provide a weather visualization system that can superimpose weather conditions onto people using computer graphics. [Solution] The weather situation visualization system 1 comprises a camera 2 for photographing people, a control terminal 3 that generates instruction signals for visualizing weather conditions based on instructions via a control screen, and a weather situation video generation device 4 that generates the weather conditions instructed by the instruction signals using 3DCG in a pre-set virtual 3D space, projects people onto a predetermined plane in the virtual 3D space, and generates a weather situation video from the image virtually captured by the virtual camera.

Description

Technical Field

[0001] The present invention relates to a weather situation visualization system, a control terminal and its program, as well as a weather situation video generation device and its program.

Background Art

[0002] In recent years, large-scale floods have occurred due to record-breaking heavy rains across the country. Therefore, when bad weather is predicted, for disaster prevention and mitigation, announcers and meteorologists call for vigilance through weather explanations based on weather forecasts on broadcasts such as TV. Conventionally, when broadcasting weather situations, in addition to the weather across the country, local weather forecasts are also broadcast. In that case, weather forecasts are made by, for example, representing the amount of wind and rain numerically or the like for each region obtained by subdividing the local area, or changing the expression of the degree of wind and rain in words according to the weather situation for each region (see Patent Document 1 and Non-Patent Document 1). For example, regarding rain, the maximum rainfall and predicted rainfall for each region are represented numerically, graphically, etc. to provide information to viewers. Also, for example, regarding wind, the maximum wind speed for each region is represented numerically or the wind speed is represented by color-coded arrows on a map of the region to provide information to viewers.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when weather conditions such as rain and wind are described using numbers or words, it is difficult for viewers to understand the severity of the wind and rain. Therefore, with conventional methods, it is difficult for viewers to anticipate the occurrence of storms of disaster magnitude, and delays in response when a disaster occurs are anticipated to be a problem. Therefore, there was a need for a presentation method that would allow people to estimate the actual extent of weather conditions expressed in numerical values, words, etc.

[0006] This invention has been made in view of the above problems and needs, and aims to provide a weather condition visualization system, a control terminal and its program, and a weather condition video generation device and its program, which are capable of visualizing weather conditions using computer graphics superimposed on a person. [Means for solving the problem]

[0007] To solve the aforementioned problems, the weather situation visualization system according to the present invention is a weather situation visualization system that visualizes weather conditions superimposed on a person, and comprises a camera, a control terminal, and a weather situation video generation device.

[0008] In this configuration, the weather situation visualization system uses a camera to photograph people. Furthermore, the weather visualization system generates instruction signals, such as rainfall and wind volume, for visualizing weather conditions via a control terminal and instructions received through a control screen.

[0009] The weather situation visualization system uses a weather situation video generation device to generate 3DCG of the weather conditions indicated by the instruction signal in a pre-defined virtual 3D space. It then projects a person onto a predetermined plane in the virtual 3D space and generates weather situation video by virtually capturing images with a virtual camera. This allows the weather visualization system to superimpose rain, wind, and other weather conditions onto a person based on instruction signals from a control terminal.

[0010] Furthermore, in order to solve the above problems, the control device according to the present invention is a control terminal used in a weather situation visualization system comprising: a camera for photographing a person; a control terminal that generates instruction signals for instructing weather conditions via a control screen; and a weather situation video generation device that generates the weather conditions instructed by the instruction signals using 3DCG in a pre-set virtual 3D space, projects the person onto a predetermined plane in the virtual 3D space, and generates a weather situation video from the image virtually captured by the virtual camera, the control terminal comprising a screen display unit, an operation instruction unit, an instruction signal generation unit, and an instruction signal transmission unit.

[0011] In this configuration, the control terminal displays a control screen on its screen display unit for setting weather conditions such as rainfall and wind speed. Furthermore, the control terminal receives weather condition instructions via a control screen through the operation instruction unit. The control terminal then uses an instruction signal generation unit to generate an instruction signal based on the content instructed by the operation instruction unit. The control terminal then transmits an instruction signal to the weather condition video generation device via an instruction signal transmission unit. This allows the operator to notify the weather information video generation device of weather conditions by operating and giving instructions on the control screen. Furthermore, the control terminal can be operated using a program that enables the computer to function as a control device.

[0012] Furthermore, in order to solve the above problems, the weather condition video generation device according to the present invention is a weather condition video generation device used in a weather condition video visualization system comprising a camera for photographing people, a control terminal for generating and transmitting instruction signals that indicate weather conditions based on instructions via a control screen, and a weather condition video generation device, and is configured to include a video signal receiving unit, an instruction signal receiving unit, an instruction signal analysis unit, and a CG rendering and synthesis unit.

[0013] In this configuration, the weather condition video generation device receives a video signal of a person captured by a camera via a video signal receiving unit. In addition, the weather condition video generation device receives an instruction signal from a control terminal by an instruction signal receiving unit. This instruction signal includes weather conditions such as rainfall and wind volume. Then, the weather condition video generation device analyzes the instruction content included in the instruction signal by an instruction signal analysis unit.

[0014] Then, the weather condition video generation device generates 3D CG in a preset virtual three-dimensional space based on the instruction content by a CG drawing synthesis unit, projects a person onto a predetermined plane of the virtual three-dimensional space, and generates a video virtually captured by a virtual camera as a weather condition video. Thereby, the weather condition video generation device can generate a weather condition video in which rain, wind, etc. are visualized superimposed on a person according to an instruction signal from the control terminal. Note that the weather condition video generation device can be operated by a program for causing a computer to function as the weather condition video generation device.

Advantages of the Invention

[0015] According to the present invention, it is possible to visualize the degree of actual weather conditions expressed by numerical values, words, etc. superimposed on a person.

Brief Description of the Drawings

[0016] [Figure 1] It is a system configuration diagram showing the overall configuration of a weather condition video visualization system according to an embodiment of the present invention. [Figure 2] It is a block configuration diagram showing the configuration of a control terminal according to an embodiment of the present invention. [Figure 3] It is a diagram showing an example of a control screen displayed by a control terminal according to an embodiment of the present invention. [Figure 4] It is a flowchart showing the operation of a control terminal according to an embodiment of the present invention. [Figure 5] It is a block configuration diagram showing the configuration of a weather condition video generation device according to an embodiment of the present invention. [Figure 6]This diagram shows CG images at different times of day, with (a) showing an example of daytime (daytime) and (b) showing an example of evening (sunset). [Figure 7] This diagram shows CG images representing wind and rain, with (a) showing an example of rain, (b) showing an example of wind, and (c) showing an example of both rain and wind. [Figure 8] This diagram shows CG images with the virtual camera's position changed. (a) shows an example where the virtual camera is placed inside a car and the image is taken from there. (b) to (d) show examples where the camera is placed at three different locations in the city and the image is taken from there. [Figure 9] These are examples of CG images where the virtual camera is located inside a car, with (a) showing an example where the rainfall level is "0" and (b) showing an example where the rainfall level is "4". [Figure 10] This diagram shows computer-generated images of a disaster occurring, with (a) showing an example of water gushing from a manhole, (b) showing an example of a flooded road, and (c) showing an example of both water gushing from a manhole and a flooded road. [Figure 11] This is an explanatory diagram for explaining the image composited by the CG rendering and compositing unit. (a) is a diagram showing the relationship between a virtual 3D space that reproduces a city street, a green screen planar image, and a virtual camera. (b) is a diagram showing (a) after chroma casting. [Figure 12] The following are examples of weather condition images synthesized by the CG rendering and compositing unit: (a) is a diagram showing a sunny condition at point A, (b) is a diagram with rain, fountains, and flooding added to the condition in (a), and (c) is a diagram with the condition as seen from inside a car added to (b). [Figure 13] This is a flowchart showing the operation of a weather condition video generation device according to an embodiment of the present invention. [Modes for carrying out the invention]

[0017] Embodiments of the present invention will be described below with reference to the drawings. [Overall configuration of the weather situation visualization system] Referring to Figure 1, the overall configuration of the weather situation visualization system 1 will be explained. Weather Condition Visualization System 1 visualizes weather conditions using 3DCG (3-Dimensional Computer Graphics) superimposed on a person. The weather situation visualization system 1 is, for example, installed in a broadcasting station, and films a person O, generating a weather situation video CI showing the wind and rain conditions superimposed on person O.

[0018] Person O is filmed by camera 2 in studio S, which has a specific color as the background. Here, the specific color is green, and the person, such as a weather forecaster or presenter, is positioned against a green screen GB. Note that the specific color does not have to be green; it could be any color that can be chroma-casted, such as blue (blue screen).

[0019] A weather condition video CI is a video that superimposes the wind and rain conditions onto a person O, according to the level of wind and rain. Weather condition video CIs can be used when broadcasting weather forecast programs. Viewers can visually grasp the level (intensity and amount) of wind and rain through this weather condition video CI.

[0020] As shown in Figure 1, the weather situation visualization system 1 comprises a camera 2, a control terminal 3, and a weather situation video generation device 4. In this setup, camera 2 and weather image generation device 4 are connected by a wired cable (for example, an SDI cable that transmits SDI (Serial Digital Interface) signals), and video signals are transmitted.

[0021] Furthermore, signals are transmitted and received between the control terminal 3 and the weather condition video generation device 4 via the wireless router R. For example, the control terminal 3 and the wireless router R can be connected via WiFi (registered trademark), and the wireless router R and the weather condition video generation device 4 can be connected via a LAN cable. Note that the connection methods between the camera 2, the control terminal 3, and the weather condition video generation device 4 are examples only and are not limited to these connections. Furthermore, it is preferable that the control terminal 3 be configured to be remotely operated wirelessly when operated by a person O, such as a weather forecaster.

[0022] Camera 2 captures person O as live footage against a specific color background (in this case, green screen GB). Camera 2 is a standard video camera and transmits the captured live footage as a video signal to the weather condition video generation device 4. Furthermore, a chroma keyer may be provided between camera 2 and weather condition video generation device 4 to adjust the green portion of the green screen GB to have a uniform color.

[0023] The control terminal 3 generates instruction signals for visualizing weather conditions based on instructions received via the control screen. The control terminal 3 can be configured using a general-purpose mobile device, tablet, or the like. The signals used to visualize weather conditions include rainfall levels and wind speed levels, but details will be provided later. The control terminal 3 transmits the generated instruction signal to the weather condition video generation device 4 via the wireless router R.

[0024] The weather condition video generation device 4 superimposes the weather conditions indicated by the instruction signal transmitted from the control terminal 3 onto the person O captured by the camera 2, and generates a weather condition video CI. Here, the weather condition video generation device 4 generates 3DCG images of the weather conditions indicated by the instruction signal in a pre-configured virtual 3D space. The weather condition video generation device 4 then projects a person onto a predetermined plane in the virtual 3D space and generates the image virtually captured by a virtual camera as a weather condition video CI. The weather condition video generation device 4 outputs the generated weather condition video CI to an external source.

[0025] With the configuration described above, the weather situation visualization system 1 can visualize weather conditions using computer graphics (CG) superimposed on a person O. Furthermore, the weather situation visualization system 1 can quickly switch and visualize weather conditions based on instruction signals from the control terminal 3. The configuration and operation of the control terminal 3 and the weather condition video generation device 4 will be described below.

[0026] [Control terminal configuration] The configuration of the control terminal 3 will be explained with reference to Figure 2. The control terminal 3 comprises a user interface unit 30, an instruction signal generation unit 31, and an instruction signal transmission unit 32.

[0027] The user interface unit 30 provides an interface with the user (operator) via an operation screen. Here, the user interface unit 30 is equipped with a touch panel display TD, which displays the operation screen and receives operation instructions from the user. The user interface unit 30 comprises a screen display unit 300 and an operation instruction unit 301.

[0028] The screen display unit 300 displays a control screen for setting weather conditions. Here, the screen display unit 300 presents the control screen to the user via a touch panel display TD.

[0029] Here, with reference to Figure 3, an example of a control screen displayed by the screen display unit 300 will be described. As shown in Figure 3, the control screen SC is configured with a time zone change button group B1, a rainfall change button group B2, a wind speed change button group B3, a camera position change button group B4, a manhole discharge switch button B5, and a road flooding switch button B6.

[0030] The time zone change button group B1 is a group of buttons that change the time zone of the weather condition video generated by the weather condition video generation device 4. Here, the time zone change button group B1 consists of two buttons: "Daytime" and "Evening." The "Daytime" button sets the time zone to daytime (daytime) mode. The "Evening" button sets the time zone to evening (sunset) mode.

[0031] The rainfall amount change button group B2 is a group of buttons that change the amount of rainfall in the weather condition video generated by the weather condition video generation device 4. Here, the rainfall adjustment button group B2 consists of five buttons that indicate different rainfall levels (levels 0-4): "0mm", "10-20mm", "20-30mm", "30-50mm", and "50mm and above".

[0032] The airflow adjustment button group B3 is a group of buttons that change the airflow of the weather condition video generated by the weather condition video generation device 4. Here, the airflow adjustment button group B3 consists of five buttons that indicate different airflow levels (levels 0-4): "0m / s", "10-15m / s", "15-20m / s", "20-30m / s", and "30m / s and above".

[0033] The camera position change button group B4 is a group of buttons that change the position of the virtual camera placed in the virtual 3D space of the weather condition video generated by the weather condition video generation device 4. Here, the camera position change button group B4 consists of four buttons that indicate different positions (points A to D) for "Camera A," "Camera B," "Camera C," and "Camera D." "Camera A," "Camera B," "Camera C," and "Camera D" indicate pre-set positions (directions) in the virtual 3D space. "Camera A" indicates that the virtual camera is located inside a car. "Camera B," "Camera C," and "Camera D" indicate pre-defined positions within the virtual 3D space, specifically in a city.

[0034] The Manhole Spout Switching Button B5 is a button that switches the video effect of water gushing out of a manhole in the weather condition video generated by the Weather Condition Video Generation Device 4. This manhole spray toggle button B5 is a toggle button that switches the operation of starting (on) or stopping (off) the visual effect of water spraying from a manhole.

[0035] The road flooding toggle button B6 is a button that switches the road flooding effect in the weather condition video generated by the weather condition video generation device 4. This road flooding toggle button B6 is a toggle button that switches the operation of starting (on) or stopping (off) the road flooding visual effect. In other words, the manhole eruption toggle button B5 and the road flooding toggle button B6 are buttons that switch between displaying and hiding specific disaster footage. Of course, this control screen SC is just an example, and it could be a simpler screen that fixes the time period and the position of the virtual camera, and displays only the rainfall adjustment button group B2 and the wind speed adjustment button group B3. Returning to Figure 2, we will continue our explanation of the configuration of control terminal 3.

[0036] The operation instruction unit 301 receives weather condition instructions via the control screen. Here, the operation instruction unit 301 receives instructions when the user presses a button on the control screen SC (see Figure 3) via the touch panel display TD. In this case, the operation instruction unit 301 determines that a button has been pressed by a double tap, which is two consecutive taps on the screen within a certain time period (for example, 0.8 seconds), in order to prevent accidental operation. The operation instruction unit 301 outputs a predetermined identification code to the instruction signal generation unit 31 to identify the button that was pressed.

[0037] The instruction signal generation unit 31 generates an instruction signal based on the content instructed by the operation instruction unit 301. Here, the instruction signal generation unit 31 generates an instruction signal based on the button identification code output from the operation instruction unit 301. The instruction signals generated by the instruction signal generation unit 31 are not particularly limited, but for example, they can be OSC messages and OSC arguments of the OSC (Open Sound Control) protocol.

[0038] For example, the instruction signal generation unit 31 generates an instruction signal using labels that identify the time zone change button group B1, the rainfall change button group B2, the wind speed change button group B3, and the camera position change button group B4 of the control screen SC described in Figure 3 as OSC messages, and labels indicating which button within that button group was selected as OSC arguments.

[0039] Furthermore, for example, the instruction signal generation unit 31 generates instruction signals for the manhole discharge switching button B5 and the road flooding switching button B6 on the control screen SC described in Figure 3, using labels that identify each button as OSC messages and predetermined fixed values ​​that have no specific meaning as OSC arguments. In this case, the instruction signal may consist only of a message and not include any arguments. The instruction signal generation unit 31 outputs the generated instruction signal to the instruction signal transmission unit 32.

[0040] The instruction signal transmission unit 32 transmits the instruction signal generated by the instruction signal generation unit 31 to the weather condition video generation device 4. Here, the instruction signal transmission unit 32 adds an IP header to the instruction signal, which contains the IP address of the control terminal 3 (the source), the IP address of the weather condition video generation device 4 (the destination), and the port number of the application that generates the weather condition video. This IP header is then sent to the wireless router R via WiFi. The wireless router R then analyzes the IP address and transmits the instruction signal to the weather condition video generation device 4 via LAN. With the configuration described above, the control terminal 3 can send an instruction signal to the weather condition video generation device 4 to change the weather conditions in the weather condition video with a simple operation such as pressing a button.

[0041] [Control terminal operation] Next, the operation of the control terminal 3 will be explained with reference to Figure 4 (and Figures 2 and 3 as appropriate). In step S1, the screen display unit 300 displays a control screen SC (see Figure 3) for setting weather conditions on the touch panel display TD. In step S2, the operation instruction unit 301 waits until the user presses a button on the control screen SC. If a button on the control screen SC is pressed (Yes in step S2), the control terminal 3 proceeds to step S3. In step S3, the operation instruction unit 301 analyzes the position of the pressed button.

[0042] The instruction signal generation unit 31 generates the instruction signal below. In other words, if the pressed button is one of the time zone change buttons B1 (time zone change in step S3), in step S4, the instruction signal generation unit 31 sets a label indicating a time zone change in the OSC message. Also, in step S5, the instruction signal generation unit 31 sets a label indicating daytime mode or evening mode corresponding to the pressed button in the OSC argument to generate an instruction signal and proceeds to step S14.

[0043] Furthermore, if the pressed button is one of the rainfall change button group B2 (rainfall change in step S3), in step S6, the instruction signal generation unit 31 sets a label indicating the rainfall change in the OSC message. In step S7, the instruction signal generation unit 31 sets a label indicating the rainfall level corresponding to the pressed button in the OSC argument to generate an instruction signal and proceeds to step S14.

[0044] Furthermore, if the pressed button is one of the airflow change button group B3 (airflow change in step S3), in step S8, the instruction signal generation unit 31 sets a label indicating airflow change in the OSC message. In step S9, the instruction signal generation unit 31 sets a label indicating the airflow level corresponding to the pressed button in the OSC argument to generate an instruction signal and proceeds to step S14.

[0045] Furthermore, if the pressed button is one of the camera position change buttons B4 (camera position change in step S3), in step S10, the instruction signal generation unit 31 sets a label indicating camera position change in the OSC message. In step S11, the instruction signal generation unit 31 sets a label indicating the camera position corresponding to the pressed button in the OSC argument to generate an instruction signal and proceeds to step S14.

[0046] Furthermore, if the pressed button is the manhole ejection switching button B5 (ejection switching in step S3), in step S12, the instruction signal generation unit 31 sets a label indicating ejection switching in the OSC message and sets a fixed value in the OSC argument to generate an instruction signal, and proceeds to step S14.

[0047] Furthermore, if the pressed button is the road flooding switch button B6 (flooding switch in step S3), in step S13, the instruction signal generation unit 31 sets a label indicating flooding switch in the OSC message and sets a fixed value in the OSC argument to generate an instruction signal, and proceeds to step S14. In step S14, the instruction signal transmission unit 32 transmits the instruction signal generated by the instruction signal generation unit 31 to the weather condition video generation device 4. Through the above operations, the control terminal 3 can transmit the user's instructions as an instruction signal to the weather condition video generation device 4 with a simple operation, such as pressing a button.

[0048] [Configuration of the weather condition video generation device] Next, with reference to Figure 5, the configuration of the weather condition video generation device 4 will be described. The weather condition video generation device 4 comprises a setting information storage unit 40, a video signal receiving unit 41, an instruction signal receiving unit 42, an instruction signal analysis unit 43, and a CG rendering and synthesis unit 44.

[0049] The setting information storage unit 40 stores various setting information for generating weather condition images produced by the weather condition image generation device 4. The setting information storage unit 40 can be made of a general storage medium such as semiconductor memory. The setting information storage unit 40 stores the time of day (daytime mode or evening mode), rainfall (levels 0 to 4), wind speed (levels 0 to 4), camera position (points A to D), eruption effect (on or off), and flooding effect (on or off). The various setting information stored in the setting information storage unit 40 is changed by the instruction signal analysis unit 43, described later, based on instructions from the control terminal 3. The setting information storage unit 40 pre-stores the following initial values: time zone "daytime mode", rainfall "level 0", wind speed "level 0", camera position "location B", eruption effect "off", and flooding effect "off".

[0050] The video signal receiving unit 41 receives live video footage of a person, filmed against a specific color (in this case, green), as a video signal from camera 2. The video signal receiving unit 41 outputs the received video signal to the CG rendering and synthesis unit 44.

[0051] The instruction signal receiving unit 42 receives instruction signals from the control terminal 3. Here, the instruction signal receiving unit 42 receives the instruction signal transmitted from the control terminal 3 via the wireless router R. The instruction signal receiving unit 42 outputs the received instruction signal to the instruction signal analysis unit 43.

[0052] The instruction signal analysis unit 43 analyzes the instruction content contained in the instruction signal received by the instruction signal receiving unit 42. Here, the instruction signal analysis unit 43 acquires the type of instruction for time period change, rainfall change, wind speed change, camera position change, spray switch, or flood switch based on a label that identifies one of the time period change button group B1, rainfall change button group B2, wind speed change button group B3, camera position change button group B4, manhole spray switch button B5, or road flood switch button B6 shown in Figure 3, which are set as OSC messages of the OSC protocol. Furthermore, the instruction signal analysis unit 43 obtains the time period, rainfall level, wind speed level, or camera position from the OSC arguments added when the instruction type is a time period change, rainfall change, wind speed change, or camera position change. The instruction signal analysis unit 43 switches processing based on the type of instruction obtained from the analysis and the arguments added according to the instruction. Here, the instruction signal analysis unit 43 includes a time zone change unit 430, a rainfall change unit 431, an airflow change unit 432, a camera position change unit 433, a jet switching unit 434, and a flooding switching unit 435.

[0053] The time zone change unit 430 changes the time zone information in the settings. The time zone change unit 430 operates when the instruction type is a time zone change, and changes the time zone stored in the setting information storage unit 40 to either daytime mode or evening mode, as indicated by the argument attached to the instruction. When there is a change in the time zone, the time zone change unit 430 notifies the CG rendering and compositing unit 44 of the mode and instructs it to change the time zone.

[0054] The rainfall adjustment unit 431 modifies the rainfall amount in the setting information. The rainfall amount modification unit 431 operates when the instruction type is rainfall amount modification, and changes the rainfall amount stored in the setting information storage unit 40 to a level 0 to 4 notified by the argument attached to the instruction. The rainfall adjustment unit 431 notifies the CG rendering and synthesis unit 44 of the level when there is a change in the rainfall amount and instructs it to change the rainfall amount.

[0055] The airflow adjustment unit 432 changes the airflow setting information. The airflow change unit 432 operates when the instruction type is airflow change, and changes the airflow stored in the setting information storage unit 40 to level 0 to 4, which is notified by the argument attached to the instruction. When the airflow rate changes, the airflow rate change unit 432 notifies the CG rendering and synthesis unit 44 of the level and instructs it to change the airflow rate.

[0056] The camera position changing unit 433 changes the camera (virtual camera) position in the settings information. The camera position change unit 433 operates when the instruction type is camera position change, and changes the camera position stored in the setting information storage unit 40 to points A to D notified by the arguments attached to the instruction. When the camera position changes, the camera position change unit 433 notifies the CG rendering and compositing unit 44 of the location and instructs it to change the camera position.

[0057] The ejection switching unit 434 changes the ejection effect of the set information. The ejection switching unit 434 operates when the instruction type is ejection switching. If the ejection effect stored in the setting information storage unit 40 is off, the ejection switching unit 434 switches it on and instructs the CG rendering and synthesis unit 44 to start the ejection effect. Also, if the ejection effect stored in the setting information storage unit 40 is on, the ejection switching unit 434 switches it off and instructs the CG rendering and synthesis unit 44 to stop the ejection effect.

[0058] The flooding switching unit 435 changes the flooding effect of the setting information. The flooding switching unit 435 operates when the instruction type is flooding switching. If the flooding effect stored in the setting information storage unit 40 is off, the flooding switching unit 435 switches it on and instructs the CG rendering and synthesis unit 44 to start the flooding effect. Also, if the flooding effect stored in the setting information storage unit 40 is on, the flooding switching unit 435 switches it off and instructs the CG rendering and synthesis unit 44 to stop the flooding effect. In other words, the ejection switching unit 434 and the flooding switching unit 435 switch between displaying or hiding images of specific disasters.

[0059] The CG rendering and compositing unit 44 generates weather condition images based on the setting information. Here, the CG rendering and compositing unit 44 generates 3DCG based on the setting information in a pre-configured virtual 3D space. The CG rendering and compositing unit 44 then projects the person from the live-action video received by the video signal receiving unit 41 onto a predetermined plane in the virtual 3D space, and generates weather condition video from the video virtually captured by the virtual camera. This CG rendering and compositing unit 44 can be implemented using a general game engine (such as Unreal Engine®, Unity®, etc.). The following describes a specific example of CG image generation in the CG rendering and compositing unit 44.

[0060] (Time slot change) Please refer to Figure 6 for an example of changing the time zone. When the CG rendering and compositing unit 44 receives an instruction from the time zone change unit 430 to change the time zone, it sets a virtual sun position in a pre-configured virtual 3D space that matches the specified mode (daytime mode or evening mode), and generates 3DCG according to the time zone by representing the light from the virtual sun position. For example, Figure 6(a) shows an example of a computer-generated image representing daytime (daytime) by shining white light parallel to the top of a virtual 3D space in the vertical direction, as the position of a virtual sun. Figure 6(b) also shows an example of a CG image representing sunset (evening) by shining orange light parallel to the horizontal plane of a virtual 3D space, representing the position of a virtual sun. Figures 6(a) and 6(b) are images virtually captured from a pre-set virtual camera position, respectively.

[0061] (Rainfall and wind speed changes) Refer to Figure 7 for an example of changing rainfall and wind speed. When the CG rendering and compositing unit 44 receives an instruction from the rainfall change unit 431 to change the amount of rain, it displays rain clouds in a pre-set virtual 3D space and generates 3DCG that represents rain according to the specified rainfall level. For example, Figure 7(a) shows an example of a CG image representing rain. Rainfall levels 1-4 can be represented by smoothly changing the size and number of particles, as well as the size of the splashes when they collide with the ground, etc.

[0062] Furthermore, when the CG rendering and compositing unit 44 receives an instruction from the airflow change unit 432 to change the airflow, it generates 3DCG in a pre-configured virtual 3D space that represents the wind according to the specified airflow level. For example, Figure 7(b) shows an example of a CG image representing wind. Wind speed levels 1 to 4 can be represented by randomly generating flying objects FO and smoothly changing their direction and speed. Furthermore, if trees exist in the virtual 3D space, wind speed levels can also be represented by changing the direction and magnitude of the trees' swaying.

[0063] The flying objects may also be changed depending on the wind speed. For example, if the wind speed is level 0, all flying objects will be removed. If the wind speed is level 1, leaves will be blown away. If the wind speed is level 2, leaves and cardboard boxes will be blown away. If the wind speed is level 3, leaves, traffic cones, and wooden planks will be blown away. If the wind speed is level 4, leaves, signs, and cargo pallets will be blown away. In this way, by moving objects affected by wind according to the wind speed, it is possible to visualize the wind.

[0064] The CG rendering and compositing unit 44 receives individual instructions from the instruction signal analysis unit 43 regarding the time period, rainfall, wind speed, camera position, and jet switching or flooding switching. However, the CG rendering and compositing unit 44 generates 3DCG by retaining the previous state for events other than those for which a change has been instructed. For example, if it is raining and there is an instruction to change the wind speed, the CG rendering and compositing unit 44 generates 3DCG that represents the rain and wind at the specified levels, as shown in Figure 7(c). Figures 7(a) to (c) are images virtually taken from a pre-set virtual camera position, respectively.

[0065] (Camera position changed) Refer to Figure 8 for an example of changing the camera position. When the CG rendering and compositing unit 44 receives an instruction from the camera position change unit 433 to change the camera (virtual camera) position, it sets the camera position to a specified location (points A to D) in a pre-set virtual 3D space and generates 3DCG corresponding to the specified camera position by virtually taking a picture.

[0066] For example, Figures 8(a) to 8(d) show images virtually captured by placing virtual cameras at points A to D, respectively. Points A to D have pre-defined positions and directions set in a virtual 3D space. Here, point A is the location inside the car, and the CG rendering and compositing unit 44 renders the car's interior equipment, such as the mirror MR and dashboard DB, as shown in Figure 9(a), onto the generated 3DCG. Furthermore, if the location changes from location A to locations B through D, the CG rendering and compositing unit 44 will hide the equipment inside the vehicle.

[0067] Furthermore, if it is raining and the camera position is changed to point A, the CG rendering and compositing unit 44 maintains the rainy state while changing the camera position and operates the wiper WP in four stages (weak, medium, strong, and maximum) according to the rainfall level 1 to 4. Of course, even if the camera position is at point A and there is an instruction to change the rainfall, the CG rendering and compositing unit 44 will operate the wiper WP according to the rainfall level.

[0068] (Spray switching / Flood switching) Refer to Figure 10 to explain examples of jet switching and flooding switching. When the CG rendering and compositing unit 44 receives an instruction from the ejection switching unit 434 to turn on the ejection effect, it plays an animation in which the manhole cover shakes and is thrown upward, and the height of the water column object gradually increases. For example, Figure 10(a) shows an example of CG footage in which the manhole cover comes off and water is ejected. When the CG rendering and compositing unit 44 receives an instruction from the ejection switching unit 434 to turn off the ejection effect, it plays an animation in which the height of the water column object gradually decreases, and finally the manhole cover is placed back in its original position.

[0069] Furthermore, when the CG rendering and compositing unit 44 receives an instruction from the flooding switching unit 435 to turn on the flooding effect, it plays an animation in which the height of the water object covering the ground gradually increases until the water reaches about the ankles of a person. For example, Figure 10(b) shows an example of CG footage of water overflowing onto a road. When the CG rendering and compositing unit 44 receives an instruction from the flooding switching unit 435 to turn off the flooding effect, it plays an animation in which the height of the water covering the ground gradually decreases until the ground is revealed.

[0070] Furthermore, if the CG rendering and compositing unit 44 is in a state where water is gushing out of the manhole (gushing effect on) and receives an instruction to turn on the flooding effect, the CG rendering and compositing unit 44 will play an animation in which water gushes out of the manhole while the road is flooded, as shown in Figure 10(c).

[0071] (Chroma processing) The CG rendering and compositing unit 44 composites the generated 3DCG with a person (green screen footage) filmed by camera 2 against a specific color (in this case, green) background, using chroma casting.

[0072] In other words, as shown in Figure 11(a), the CG rendering and compositing unit 44 projects a person (green screen image GBI) onto a plane at a predetermined relative position relative to a virtual camera VC placed at a specified camera position in a virtual three-dimensional space. Then, as shown in Figure 11(b), the CG rendering and compositing unit 44 performs chroma processing to virtually photograph the person O with the virtual camera VC. The plane on which the green screen image (GBI) is placed is a fixed position predetermined from the virtual camera (VC). Therefore, if the camera position is changed, the position of the plane on which the green screen image (GBI) is placed will also move relatively. As a result, the CG rendering and compositing unit 44 can generate an image in which a person is placed in a virtual three-dimensional space, as shown in Figure 12(a).

[0073] In this case, if the CG rendering and compositing unit 44 receives instructions from the instruction signal analysis unit 43 such as a change in time period, rainfall amount, wind speed, camera position, fountain switching, or flooding switching, it will visualize the weather conditions by superimposing them onto the person, as shown in Figure 12(b). Furthermore, when location A (inside the vehicle) is specified as the camera position change, the CG rendering and compositing unit 44 renders the in-vehicle equipment such as the mirror MR and dashboard DB, as shown in Figure 12(c). With the configuration described above, the weather condition video generation device 4 can generate video (weather condition video) showing altered weather conditions in response to instructions from the control terminal 3.

[0074] [Weather condition video generation device operation] Next, the operation of the weather condition video generation device 4 will be explained with reference to Figure 13 (and Figure 5 as appropriate). It is assumed that the setting information storage unit 40 has initial values ​​set in advance. Furthermore, it is assumed that the video signal receiving unit 41 is constantly receiving live video (green screen video) from camera 2.

[0075] In step S20, the instruction signal receiving unit 42 waits until it receives an instruction signal from the control terminal 3 via the wireless router R. If an instruction signal is received (Yes in step S20), the weather condition video generation device 4 proceeds to step S21.

[0076] In step S21, the instruction signal analysis unit 43 analyzes the instruction content contained in the instruction signal received in step S20. The instruction signal analysis unit 43 then performs control according to the instruction content (OSC message, OSC argument).

[0077] In other words, if the instruction is to change the time zone (time zone change in step S21), in step S22, the time zone change unit 430 changes the time zone stored in the setting information storage unit 40 to the daytime mode or evening mode set in the OSC argument, and instructs the CG drawing and compositing unit 44 to change the time zone. Then, the operation proceeds to step S28.

[0078] Furthermore, if the instruction is to change the amount of rainfall (rainfall change in step S21), in step S23, the rainfall change unit 431 changes the amount of rainfall stored in the setting information storage unit 40 to the level of rainfall set in the OSC argument, and instructs the CG drawing and synthesis unit 44 to change the wind speed. Then, the operation proceeds to step S28.

[0079] Furthermore, if the instruction is to change the airflow (airflow change in step S21), in step S24, the airflow change unit 432 changes the airflow stored in the setting information storage unit 40 to the level of the airflow set in the OSC argument, and instructs the CG rendering and synthesis unit 44 to change the airflow. Then, the operation proceeds to step S28.

[0080] Furthermore, if the instruction is to change the camera position (camera position change in step S21), in step S25, the camera position change unit 433 changes the camera position stored in the setting information storage unit 40 to points A to D set in the OSC arguments, and instructs the CG rendering and compositing unit 44 to change the camera position. Then, the operation proceeds to step S28.

[0081] Furthermore, if the instruction is to switch the spray output (spray output switching in step S21), in step S26, the spray output switching unit 434 switches the spray effect stored in the setting information storage unit 40 to ON if it is OFF, and instructs the CG rendering and synthesis unit 44 to start the spray effect. Also, if the spray effect stored in the setting information storage unit 40 is ON, the spray output switching unit 434 switches it to OFF, and instructs the CG rendering and synthesis unit 44 to stop the spray effect. Then, the operation proceeds to step S28.

[0082] Furthermore, if the instruction is to switch to flooding mode (flooding mode is switched in step S21), in step S27, the flooding mode switching unit 435 switches the flooding effect stored in the setting information storage unit 40 to ON if it is OFF, and instructs the CG rendering and synthesis unit 44 to start the flooding effect. Also, if the flooding effect stored in the setting information storage unit 40 is ON, the flooding mode switching unit 435 switches it to OFF, and instructs the CG rendering and synthesis unit 44 to stop the flooding effect. Then, the operation proceeds to step S28.

[0083] In step S28, the CG rendering and compositing unit 44 generates 3DCG based on the setting information set by the instruction signal analysis unit 43, and combines it with the live-action video (green screen video) received by the video signal receiving unit 41 by applying chroma processing to generate a weather condition video. In step S29, the instruction signal receiving unit 42 determines whether or not it has received an instruction signal from the control terminal 3. If no instruction signal is received at this point (No in step S29), the weather condition video generation device 4 returns to step S28 and continues its operation. On the other hand, if an instruction signal is received (Yes in step S29), the weather condition video generation device 4 returns to step S21 and continues its operation. Through the above operations, the weather condition video generation device 4 can generate video (weather condition video) showing altered weather conditions based on instruction signals transmitted from the control terminal 3 at any given time.

[0084] Although embodiments of the present invention have been described in detail above, the present invention is not limited to the embodiments described above, and includes design changes and the like that that do not depart from the spirit of the present invention. Furthermore, although the above-described embodiment assumed that the control terminal 3 and the weather condition video generation device 4 were independent hardware components, the present invention is not limited thereto. For example, the present invention can also be implemented using hardware resources such as the CPU, memory, and hard disk of a computer as programs for functioning as the control terminal 3 and the weather condition video generation device 4, respectively. These programs may be distributed via a communication line or written to a recording medium such as a CD-ROM or flash memory and distributed. [Explanation of symbols]

[0085] 1. Weather situation visualization system 2 cameras 3. Control terminal 30. User Interface Section 300 Screen display section 301 Operation instruction section 31 Instruction signal generation section 32 Instruction signal transmission unit 4. Weather Condition Video Generation Device 40. Configuration Information Storage Unit 41 Video signal receiving unit 42 Instruction signal receiving unit 43 Instruction signal analysis section 44 CG drawing synthesis section R Wireless Router SC control screen CI Weather Conditions Video

Claims

1. A weather situation visualization system that superimposes weather conditions onto a person, A camera for photographing the aforementioned person, A control terminal that generates instruction signals for visualizing the weather conditions via instructions on a control screen, A weather condition video generation device generates weather conditions indicated by the aforementioned instruction signal using 3D CG in a pre-set virtual 3D space, projects the person onto a predetermined plane in the virtual 3D space, and generates a weather condition video from the image virtually captured by a virtual camera. A weather condition visualization system characterized by having the following features.

2. If the instruction signal includes an instruction to change the rainfall amount, including the rainfall level, The weather situation visualization system according to claim 1, characterized in that the weather situation video generation device generates the 3DCG with the amount of rainfall changed according to the rainfall level.

3. If the instruction signal includes an instruction to change the airflow level, The weather condition visualization system according to claim 1, characterized in that the weather condition video generation device generates the 3D CG with the airflow rate changed according to the airflow level.

4. If the instruction signal indicates a change in the position of the virtual camera, The weather condition video generation device is characterized in that it places the virtual camera at the indicated location and virtually takes pictures, as described in claim 1.

5. If the aforementioned instruction signal instructs the display or hiding of images of the disaster, The weather situation video generation device generates the 3DCG by displaying or hiding images of a specified disaster, as described in claim 1.

6. The control terminal is used in a weather situation visualization system comprising: a camera for photographing a person; a control terminal that generates instruction signals for weather conditions based on instructions via a control screen; and a weather situation video generation device that generates the weather conditions indicated by the instruction signals using 3D CG in a pre-set virtual 3D space, projects the person onto a predetermined plane in the virtual 3D space, and generates a weather situation video from the image virtually captured by the virtual camera. A screen display unit that displays a control screen for setting the aforementioned weather conditions, An operation instruction unit that receives instructions regarding the weather conditions via the control screen, An instruction signal generation unit generates an instruction signal based on the content instructed by the operation instruction unit, An instruction signal transmission unit that transmits the instruction signal to the weather condition video generation device, A control terminal characterized by being equipped with the following features.

7. A program for causing a computer to function as a control terminal as described in claim 6.

8. A weather condition video generation device used in a weather condition video visualization system comprising a camera for photographing people, a control terminal for generating and transmitting instruction signals to indicate weather conditions via a control screen, and a weather condition video generation device, A video signal receiving unit that receives a video signal of the person captured by the camera, An instruction signal receiving unit that receives the instruction signal from the control terminal, An instruction signal analysis unit that analyzes the instruction content contained in the instruction signal, Based on the above instructions, a CG rendering and synthesis unit generates 3D CG in a pre-set virtual 3D space, projects the person onto a predetermined plane in the virtual 3D space, and generates weather condition video from images virtually captured by a virtual camera. A weather condition video generation device characterized by being equipped with the following features.

9. A program for causing a computer to function as a weather condition video generation device according to claim 8.

Images