Live camera system, server, GUI information generation method and program
The live camera system addresses image display failures by generating GUI information with hidden indicators for unavailable images, minimizing user inquiries and system burden through proactive image freshness determination.
Patent Information
- Application Number
- JP2020210615
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-12-18
- Publication Date
- 2024-11-20
- Estimated Expiration
- 2040-12-18
AI Technical Summary
Existing live camera systems often fail to display moving images due to network or camera malfunctions, leading to user inquiries and complaints when thumbnail images are selected but cannot be converted to video.
A live camera system with a server that acquires, updates, and determines the freshness of thumbnail images, generating GUI information that includes hidden indicators for non-display if images are outdated or unavailable, reducing user inquiries by preemptively indicating non-display.
The system automatically indicates when images cannot be displayed, reducing user complaints and system burden by proactively managing image availability through GUI updates.
Smart Images

Figure 0007587336000001 
Figure 0007587336000002 
Figure 0007587336000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a live camera system that enables the most recent images captured by cameras installed at multiple locations to be displayed on a user's terminal device, and in particular to improvements to the graphical user interface (GUI) displayed on the terminal device. [Background technology]
[0002] In recent years, live camera systems have become widespread, in which multiple cameras are installed around roads and public facilities, and the latest images taken by each camera can be viewed by an unspecified number of users via a communication network. By accessing the live camera system via the Internet using a terminal device owned by the user, such as a personal computer (PC) or smartphone, the user can learn about the congestion situation on the roads he or she plans to drive on, or the weather conditions around public facilities he or she plans to visit, through images.
[0003] A live camera system is a system that is used irregularly by an unspecified number of users using their own terminal devices. In other words, the system needs to be designed with the assumption that it will be used by users who do not know where to install the cameras or how to use the live camera system, and an intuitive and easy-to-understand GUI is required.
[0004] For example, in the live camera system disclosed in Patent Document 1, a user first accesses the web server of the live camera system using a browser function of a terminal device. The terminal device acquires GUI information such as a web page from the web server, and displays a GUI screen on the screen of the terminal device, in which a map of the shooting area and an icon showing the installation location of the camera are superimposed on the map.
[0005] When a user clicks on an icon on the map on the GUI screen, the terminal device displays a list of thumbnail images taken by each camera installed at the position indicated by the icon on the map on the basis of the acquired GUI information on the screen of the terminal device.
[0006] When the user selects one of the thumbnail images displayed in the list, the terminal device retrieves the video image from the camera associated with the selected thumbnail image from the live camera system's server (referred to as a cloud server in Patent Document 1) and displays it.
[0007] Note that converting images captured by a camera successively into thumbnail images reduced to a desired size and having the camera periodically transmit the thumbnail images is not limited to live camera systems but is a common practice. For example, by linking the thumbnail images received from the camera to GUI information, the terminal device of the user that has acquired the GUI information can display the most recently captured thumbnail image on the GUI screen displayed based on the GUI information. [Prior art documents] [Patent documents]
[0008] [Patent Document 1] JP 2019-165319 A Summary of the Invention [Problem to be solved by the invention]
[0009] However, even if thumbnail images are displayed on the GUI screen of a terminal device and a user selects one of the thumbnail images as in the live camera system disclosed in Patent Document 1, there are cases in which the video image from the camera associated with the selected thumbnail image cannot be displayed on the user's terminal device.
[0010] One possible reason why video images cannot be displayed is a malfunction of the network between the camera and the server, or a malfunction of the camera itself.
[0011] Many users of live camera systems do not know how the system works, and therefore, if they are unable to display video from a selected camera on their terminal device, they may file a complaint with the provider of the live camera system.
[0012] Therefore, the present invention aims to provide a live camera system, server, GUI information generation method, and program that, when there is a possibility that the latest image from a specific camera cannot be transmitted from the server to a terminal device, automatically notifies the user before the user selects a camera that the image from the camera in question will not be displayed, thereby reducing inquiries from the user. [Means for solving the problem]
[0013] (1) In order to solve the above problems, a live camera system which is one aspect of the present invention includes a plurality of cameras which sequentially capture images, a server which receives video data generated based on the images from each camera via a network, and a terminal device which is capable of sequentially obtaining streaming data converted from the video data from the server and displaying the images.
[0014] The server includes an acquisition unit, an update unit, a determination unit, a generation unit, and a server communication unit. The acquisition unit sequentially acquires thumbnail images obtained by reducing images captured by each camera. The thumbnail images include identification information for identifying the camera that captured the images. When the acquisition unit acquires the thumbnail images, the update unit updates the thumbnail images having the same identification information, and stores the thumbnail images in association with an update date and time indicating the date and time of the update. The determination unit determines at a predetermined timing whether a difference between the update date and time associated with the updated thumbnail images and a current date and time is within a predetermined time. When the determination unit determines that the difference is within the predetermined time, the generation unit generates GUI information including the thumbnail images and a reception component that receives a request to transmit streaming data of the camera identified by the identification information associated with the thumbnail images, and when the determination unit determines that the difference is not within the predetermined time, generates GUI information including non-display information indicating that the latest image of the camera identified by the identification information associated with the thumbnail images cannot be displayed instead of the thumbnail images. When the server communication unit receives a GUI request for GUI information from the terminal device via the network, the server communication unit transmits the GUI information to the terminal device.
[0015] The terminal device includes a terminal communication unit and a GUI unit. The terminal communication unit transmits a GUI request to the server and receives GUI information from the server. The GUI unit displays a GUI screen including thumbnail images and reception components or non-display information based on the GUI information, and receives a selection for the reception components.
[0016] (2) In one aspect of the present invention, in the live camera system of (1) above, the GUI information includes a map image and camera icons indicating the location where each camera is installed, and the GUI unit displays a plurality of camera icons superimposed on the map image based on the GUI information, and when a selection of one of the plurality of camera icons is accepted, a thumbnail image and a reception part of the camera installed at the location indicated by the camera icon, or hidden information, is displayed.
[0017] (3) In one aspect of the present invention, in the live camera system of (1) above, the GUI information includes a map image and location icons indicating locations where multiple cameras are installed, and the GUI unit displays the multiple location icons superimposed on the map image based on the GUI information, and when a selection of one of the multiple location icons is accepted, a list including thumbnail images and reception parts, or hidden information, of each camera installed at the location indicated by the location icon is displayed based on the GUI information.
[0018] (4) In one aspect of the present invention, in the live camera system described in any one of (1) to (3) above, the thumbnail image may further include a shooting date and time indicating the date and time when the image was taken, and the determination unit may use the shooting date and time instead of the update date and time.
[0019] (5) In one aspect of the present invention, in the live camera system described in any one of (1) to (4) above, the server communication unit may further be configured to transmit the GUI information to a terminal device via a network when the generation unit generates GUI information.
[0020] (6) In order to solve the above problems, a server according to one aspect of the present invention receives, via a network, moving image data generated based on images from a plurality of cameras that sequentially capture images, and when a request to transmit streaming data converted from the moving image data is received from a terminal device, the server sequentially transmits the streaming data to the terminal device via the network. The server includes an acquisition unit, an update unit, a determination unit, a generation unit, and a server communication unit. The acquisition unit sequentially acquires thumbnail images obtained by reducing images captured by each camera. The thumbnail images include identification information that identifies the camera that captured the images. When the acquisition unit acquires the thumbnail images, the update unit updates the thumbnail images having the same identification information, and stores the thumbnail images in association with an update date and time indicating the date and time of the update. The determination unit determines, at a predetermined timing, whether or not a difference between the update date and time associated with the updated thumbnail image and the current date and time is within a predetermined time. The generation unit generates GUI information including a thumbnail image and a reception component that receives a request to transmit streaming data of the camera identified by the identification information associated with the thumbnail image when the determination unit determines that the difference is not within the predetermined time, and generates GUI information including non-display information indicating that the latest image captured by the camera identified by the identification information associated with the thumbnail image cannot be displayed instead of the thumbnail image when the determination unit determines that the difference is not within the predetermined time. The server communication unit transmits the GUI information to the terminal device when a GUI request requesting GUI information is received from the terminal device via the network.
[0021] (7) In order to solve the above problems, a GUI information generation method according to one aspect of the present invention is a GUI information generation method used in a server that receives, via a network, moving image data generated based on images from a plurality of cameras that sequentially capture images, and that, upon receiving a request from a terminal device to transmit streaming data converted from the moving image data, sequentially transmits the streaming data to the terminal device via the network. The GUI information generation method includes an acquisition step, an update step, a determination step, and a generation step. The acquisition step sequentially acquires thumbnail images that are reduced images from each camera. The acquired thumbnail images include identification information that identifies the camera that captured the images. When a thumbnail image is acquired in the acquisition step, the update step updates a thumbnail image having the same identification information, and stores the thumbnail image in association with an update date and time indicating the date and time of the update. The determination step determines, at a predetermined time, whether or not a difference between the update date and time associated with the updated thumbnail image and the current date and time is within a predetermined time. In the generation step, if it is determined in the determination step that the difference is within a predetermined time, GUI information including a thumbnail image and a reception component that receives a request to send streaming data of the camera identified by the identification information associated with the thumbnail image is generated, and if it is determined that the difference is not within the predetermined time, GUI information is generated instead of the thumbnail image that includes non-display information indicating that the latest image of the camera identified by the identification information associated with the thumbnail image cannot be displayed.
[0022] (8) In order to solve the above problem, a program according to one aspect of the present invention is a program that receives, via a network, moving image data generated based on images from a plurality of cameras that sequentially capture images, and when a request for transmitting streaming data converted from the moving image data is received from a terminal device, causes the program to function as a server that sequentially transmits the streaming data to the terminal device via the network, and causes the server to execute a process for generating GUI information. The generation process includes an acquisition step, an update step, a determination step, and a generation step. The acquisition step sequentially acquires thumbnail images that are reduced images from each camera. The acquired thumbnail images include identification information that identifies the camera that captured the images. When a thumbnail image is acquired in the acquisition step, the update step updates a thumbnail image having the same identification information, and stores the thumbnail image in association with an update date and time indicating the date and time of the update. The determination step determines, at a predetermined time, whether or not a difference between the update date and time associated with the updated thumbnail image and the current date and time is within a predetermined time. In the generation step, if it is determined in the determination step that the difference is within a predetermined time, GUI information including a thumbnail image and a reception component that receives a request to send streaming data of the camera identified by the identification information associated with the thumbnail image is generated, and if it is determined that the difference is not within the predetermined time, GUI information is generated instead of the thumbnail image that includes non-display information indicating that the latest image of the camera identified by the identification information associated with the thumbnail image cannot be displayed. Effect of the Invention
[0023] In a live camera system to which the present invention is applied, if there is a possibility that the latest image from a specific camera cannot be transmitted from the server to the terminal device, a message is automatically displayed indicating that the latest image from that camera will not be displayed before the user selects a camera, thereby reducing inquiries from the user.
[0024] When a terminal device receives GUI information from a server to which the present invention is applied, if there is a possibility that the server will not be able to send the latest images from a specific camera to the terminal device, the terminal device automatically notifies the user before the user selects a camera that the latest images from the camera in question will not be displayed, thereby reducing inquiries from the user.
[0025] A terminal device that receives GUI information from a server that executes a GUI information generation method or program to which the present invention is applied automatically notifies the user before the user selects a camera that the latest image from the camera in question will not be displayed if there is a possibility that the server will not be able to send the latest image from a specific camera to the terminal device, thereby reducing inquiries from the user. [Brief description of the drawings]
[0026] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a live camera system 10. [Diagram 2] FIG. 2 is a diagram illustrating an example of a functional configuration of a terminal device 4. [Diagram 3] FIG. 2 illustrates an example of a functional configuration of a server 3. [Figure 4] FIG. 2 is a sequence diagram of the live camera system 10. [Diagram 5] FIG. 4 is a diagram showing an example of a GUI screen based on GUI information of the present invention. [Figure 6] FIG. 4 is a diagram showing an example of a GUI screen based on GUI information of the present invention. [Figure 7] FIG. 1 is a diagram showing an example of a GUI screen based on conventional GUI information. [Figure 8] FIG. 4 is a diagram showing an example of a GUI screen based on GUI information of the present invention. [Figure 9] FIG. 13 is a diagram illustrating a modified example of the functional configuration of the server 3. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0027] [composition] A live camera system 10 according to an embodiment of the present invention will be described with reference to the drawings. As shown in FIG. 1, the live camera system 10 is composed of a camera 1, a network 2, a server 3, and a terminal device 4. The camera 1 is a general term for the first camera 1a or the second camera 1b. The network 2 is a general term for the first network 2a, the second network 2b, or the third network 2c. The terminal device 4 is a general term for the first terminal device 4a or the second terminal device 4b.
[0028] Camera 1 is a so-called network camera. Camera 1 has a function of sequentially capturing images and generating still image data or video data based on the captured images. Camera 1 also has a function of generating thumbnail images, which are still image data obtained by reducing the captured images to a predetermined size, by setting the function in advance.
[0029] Camera 1 has a communication function for communicating with other communication devices via network 2, and transmits the generated still image data, video data, or thumbnail images to a predetermined network device. In this embodiment, each camera 1 transmits the generated still image data, video data, or thumbnail images to server 3 via network 2.
[0030] A plurality of cameras 1 are installed in a given area for the purpose of sequentially capturing images of road congestion and the weather in the vicinity of the installation location. The cameras 1 are installed, for example, on poles erected at the side of the road or at high places in facilities.
[0031] The first camera 1a and the second camera 1b are examples of the existence of multiple cameras 1 and the case where the communication forms are different, and are not limited to two cameras. The first camera 1a and the second camera 1b are installed so as to capture images of different shooting ranges. For example, the first camera 1a and the second camera 1b may be installed at different points, or may be installed at the same point but with different shooting viewpoints and shooting directions.
[0032] The first camera 1a is connected to the first network 2a by a wired LAN cable. The second camera 1b is wirelessly connected to the second network 2b, which is a wireless LAN. The area in which the second camera 1b is installed has a poor wireless environment, so communication via the second network 2b sometimes fails. The second network 2b is connected to the first network 2a.
[0033] The server 3 has a function of receiving still image data, video data, or thumbnail images sequentially transmitted from the camera 1 via the network 2. When the server 3 receives a request from a terminal device 4, it has a function of transmitting, to the requesting terminal device 4, GUI information including the received thumbnail image or streaming data obtained by converting the still image data or video data. The detailed configuration and operation of the server 3 will be described later.
[0034] The terminal device 4 is a PC or a smartphone owned or used by a user. The terminal device 4 has a communication function and communicates with the server 3 via the network 2. The terminal device 4 receives GUI information and streaming data from the server 3 by transmitting a GUI request or a streaming request to the server 3. The terminal device 4 has a function of displaying a GUI screen based on the GUI information received from the server 3 via the network 2, and displaying an image captured by a selected camera 1 based on the received streaming data.
[0035] The first terminal device 4a and the second terminal device 4b are examples showing that there are a plurality of terminal devices 4 and that the communication form may differ depending on the terminal device 4. That is, the number of terminal devices 4 is not limited to two. The first terminal device 4a is a so-called PC. The first terminal device 4a is connected to the first network 2a by a wired LAN cable.
[0036] The second terminal device 4b is a so-called smartphone. The second terminal device 4b has a wireless communication function for connecting to a third network 2c, which is a wireless communication network. The third network 2c is connected to the first network 2a. The detailed configuration and operation of the terminal device 4 will be described later.
[0037] [Server 3 configuration] As shown in Fig. 2, the server 3 is composed of a web server 30, an FTP server 31, and a streaming server 32. The hardware of the server 3 is a computer equipped with a CPU, a memory, and a large-capacity storage medium. The server 3 may be a server group composed of a plurality of computers. The server 3 may also be a virtual machine constructed using the server group.
[0038] The web server 30 has a function of generating and transmitting GUI information, which is a web page written in HTML (HyperText Markup Language) or the like. When the web server 30 receives a request specifying a URI (Uniform Resource Identifier) of the GUI information from the terminal device 4 via HTTP (HyperText Transfer Protocol) communication (hereinafter referred to as a "GUI request"), the web server 30 transmits the specified GUI information to the terminal device 4 via HTTP communication.
[0039] The web server 30 includes a storage unit 33, a timer unit 34, an acquisition unit 35, an update unit 36, a determination unit 37, a generation unit 38, and a server communication unit 39. The storage unit 33 has a function of storing various information. The storage unit 33 stores, for example, various images constituting the GUI information, the latest thumbnail images captured by each camera 1, non-display information, reception components such as icons, programs related to the generation process of GUI information, and the like.
[0040] The clock unit 34 has a function of measuring the current date and time. The hardware of the clock unit 34 is configured with an RTC (Real Time Clock) circuit.
[0041] The acquiring unit 35 has a function of sequentially acquiring thumbnail images obtained by reducing the images captured by each camera 1. Specifically, the acquiring unit 35 stores the thumbnail images received from each camera 1 by the FTP server 31, which will be described later, in a public folder provided in the storage unit 33.
[0042] The thumbnail image may be an image compressed in JPEG format, or may be in another image format, such as PNG, BMP, GIF, or another format. The thumbnail image includes identification information that identifies the camera 1 that took the image. If the thumbnail image is a JPEG image, the identification information and the date and time of shooting are included in the Exif information. The file format, image size, and transmission cycle of the thumbnail image are set for the camera 1.
[0043] When the acquisition unit 35 acquires a thumbnail image, the update unit 36 updates a thumbnail image with the same identification information that has already been stored in the public folder, and stores the updated date and time indicating the date and time of the update in association with the updated thumbnail image. The updated date and time is stored in a known file system of the web server 30 using the current date and time measured by the clock unit 34. If a thumbnail image with the same identification information is not in the public folder, the thumbnail image is stored as is. Updating a thumbnail image means replacing a thumbnail image with an older update date and time with a newer one for the same identification information. Therefore, when a thumbnail image with a newer update date and time is acquired, the thumbnail image with the older update date and time is deleted from the storage unit 33.
[0044] The determination unit 37 has a function of determining, at a predetermined timing, whether the difference between the update date and time associated with the acquired thumbnail image and the current date and time is within a predetermined time or not. The predetermined time may be the same as or different from the cycle at which thumbnail images are transmitted from each camera 1 to the FTP server 31.
[0045] The predetermined time may be, for example, 1.5 times longer than the transmission cycle of thumbnail images. The cycles at which each camera 1 transmits thumbnail images may be the same, but the timing of transmission may be different. In addition, assuming that some network delay is tolerable, the predetermined time for determination may be set to a time obtained by adding the network delay time to the transmission cycle of camera 1.
[0046] For example, since the second camera 1b is wirelessly connected to the second network 2b, it is possible that communication may be interrupted due to the influence of jamming radio waves or the like. However, communication may be restored after one minute. In this way, when there is a possibility that communication may be restored in a short period of time, the determination unit 37 can adjust the predetermined time used for the determination to perform an appropriate determination.
[0047] The generating unit 38 has a function of generating GUI information including the acquired thumbnail image, or GUI information including non-display information instead of the acquired thumbnail image. When the determining unit 37 determines that the difference between the update date and time of the acquired thumbnail image and the current time is within a predetermined time, the generating unit 38 generates GUI information including the thumbnail image and a reception component that receives a transmission request for streaming data of the camera 1 identified by the identification information associated with the thumbnail image.
[0048] On the other hand, if the determination unit 37 determines that the difference is not within the specified time, the generation unit 38 generates, instead of the thumbnail image, GUI information including non-display information indicating that the latest image of camera 1 identified by the identification information associated with the thumbnail image cannot be displayed.
[0049] As described above, the server communication unit 39 has a communication function of transmitting the GUI information generated by the generation unit 38 when a GUI request is received from each terminal device 4 via HTTP communication.
[0050] The FTP server 31 has a function of sequentially receiving thumbnail images generated at a predetermined cycle from each camera 1 via FTP communication. The thumbnail images sequentially received by the FTP server 31 are temporarily stored in an FTP folder provided in a storage unit (not shown) within the FTP server 31. The acquisition unit 35 monitors the FTP folder, and when a new thumbnail image is stored, it acquires it and stores it in a public folder in the storage unit 33. When the acquisition unit 35 acquires a thumbnail image from the FTP folder, it deletes the thumbnail image from the FTP folder.
[0051] The streaming server 32 has a function of sequentially receiving moving image data from each camera 1 via the network 2 and converting the received moving image data into streaming data of a predetermined streaming format, and a communication function of sequentially transmitting the specified streaming data to the terminal device 4 upon receiving a request to send specified streaming data from the terminal device 4 (hereinafter referred to as a "streaming request").
[0052] The streaming method used by the streaming server 32 is preferably, for example, an HTTP streaming method, and more specifically, a method compatible with standards such as HLS (HTTP Live Streaming) and MPEG DASH. Note that a unique protocol other than the HTTP streaming method may be used. The function of the streaming server 32 relating to conversion to streaming data and transmission itself is a publicly known technology, as described in, for example, JP 2020-136702 A, and therefore a detailed description thereof will be omitted.
[0053] [Configuration of terminal device 4] 3, the terminal device 4 is composed of a terminal communication unit 41, a terminal storage unit 42, a terminal control unit 43, and a GUI unit 44. As described above, the terminal device 4 is a computer equipped with a CPU, a memory, and a storage medium, and each functional unit is realized by the CPU executing a control program stored in the storage medium.
[0054] The terminal communication unit 41 has a communication function of transmitting a GUI request or a streaming request to the server 3 via the network 2 and receiving GUI information or streaming data from the server 3.
[0055] The terminal storage unit 42 has the above-mentioned storage medium, and stores the received GUI information and streaming data in addition to the control program for controlling the terminal device 4. The terminal control unit 43 is composed of the above-mentioned CPU and memory, and controls each functional unit of the terminal device 4.
[0056] Although not shown, the GUI unit 44 has, as hardware, a display for displaying information and an input function for receiving selections from a user. For example, the hardware of the GUI unit 44 in the first terminal device 4a may be a display, a mouse, and a keyboard. In addition, the GUI unit 44 in the second terminal device 4b, which is a smartphone, may be a touch panel display.
[0057] The GUI unit 44 has a function of displaying a GUI screen including thumbnail images and reception parts or non-display information and accepting selection of the reception parts based on the GUI information received by the terminal communication unit 41. The GUI unit 44 also has a function of displaying images captured by the camera 1 based on the received streaming data.
[0058] [Operation] The operation of the live camera system 10 will be described with reference to Fig. 4. Note that the symbol "S" used in the description of the operation means a step. First, as shown in Fig. 4, when the acquisition unit 35 acquires a thumbnail image (acquisition step, S0: Yes), the server 3 proceeds to processing S1, and when the acquisition unit 35 does not acquire a thumbnail image (acquisition step, S0: No), the server 3 proceeds to processing S3.
[0059] In the process of S1, if there is a thumbnail image associated with the same identification information as the identification information associated with the acquired thumbnail image (S1: Yes), the update unit 36 updates it to the newly acquired thumbnail image (update step, S2). That is, the update unit 36 deletes the old thumbnail image and stores the newly acquired thumbnail image and the update date and time. Then, the process proceeds to S3.
[0060] In the process of S1, if there is no thumbnail image associated with the same identification information as the identification information associated with the acquired thumbnail image (S1: No), the process proceeds to the process of S3.
[0061] In the process of S3, if the timing of the timer unit 34 indicates that a predetermined time has elapsed (S3: Yes), the process proceeds to the process of S4, which is a judgment step; if the predetermined time has not elapsed (S3: No), the process waits until the time has elapsed.
[0062] In the process of S4, the determination unit 37 determines whether the difference between the update date and time associated with the acquired thumbnail image and the current date and time is within a predetermined time. If the determination unit 37 determines that the difference is within the predetermined time (determination step, S4: Yes), the process proceeds to the generation step of S5. If the determination unit 37 determines that the difference is not within the predetermined time (determination step, S4: No), the process proceeds to the process of S6.
[0063] In S5 (generation step), the generation unit 38 generates GUI information including a thumbnail image and a reception component for receiving a transmission request for streaming data of the camera 1 identified by the identification information associated with the thumbnail image.
[0064] In S6 (generation step), the generation unit 38 generates, instead of the thumbnail image, GUI information including non-display information indicating that the latest image of the camera 1 identified by the identification information associated with the thumbnail image cannot be displayed.
[0065] After the generation step of S5 or S6, when the GUI unit of the terminal device 4 accepts a GUI request (S7), the server communication unit 39 of the server 3 receives the GUI request. The server communication unit 39 transmits the GUI information generated by the generation unit 38 to the terminal device 4 (S8).
[0066] The GUI unit 44 of the terminal device 4 displays a GUI screen including a thumbnail image or non-display information based on the received GUI information (S9).
[0067] The GUI screen displayed by the GUI unit 44 of the terminal device 4 will be described with reference to the drawings. Fig. 5 is a diagram showing an example of a GUI screen displayed by the GUI unit 44 based on GUI information received by the terminal device 4. The GUI screen shown in Fig. 5 includes a map image 50, a first camera icon 51, a second camera icon 52, a first window 100, a first thumbnail image 101, and a reception component 102. The GUI unit 44 displays the first camera icon 51 and the second camera icon 52 superimposed on the map image 50 based on the GUI information.
[0068] The first camera icon 51 and the first window 100 are associated with the GUI information received by the terminal device 4. When the terminal device 4 receives an operation of the user selecting the first camera icon 51, the terminal device 4 pops up and displays the first window 100. The first window 100 includes a first thumbnail image 101 and a reception component 102 that displays "Watch Video."
[0069] In the GUI information, the reception component 102 is associated with the identification information of the camera 1 that captured the first thumbnail image 101. When the user performs an operation to select the reception component 102, a streaming request including the identification information is transmitted to the streaming server 32. Upon receiving the streaming request, the streaming server 32 transmits the corresponding streaming data to the terminal device 4.
[0070] An example of a GUI screen when the user performs an operation to select the second camera icon 52 is shown in Fig. 6. As shown in Fig. 6, a first window 100 is popped up, but does not include a thumbnail image and a reception component 102, and instead includes first hidden information 103. The first hidden information 103 displays "Live video cannot be displayed because the camera is out of order." In other words, it indicates that the latest image from the camera 1 corresponding to the second camera icon 52 cannot be displayed.
[0071] An example of a GUI screen in the conventional technology is shown in Fig. 7. In the conventional technology, the determination unit 37 does not perform a determination process, and the generation unit 38 does not perform a generation process of GUI information. Therefore, even if a communication failure occurs between the corresponding camera 1 and the server 3, the second thumbnail image 104 and the reception part 102 that have not been updated are displayed as shown in Fig. 7.
[0072] 7, there is a high possibility that a user viewing this will select the reception component 102. Then, since the video display does not start despite the operation of selecting the reception component 102, it is quite conceivable that the user will make a complaint to the provider of the live camera system 10.
[0073] On the other hand, in the live camera system 10 of the present invention, when there is a possibility that images from a particular camera 1 cannot be transmitted, as shown in FIG. 6, a message is automatically displayed indicating that video images from the camera 1 will not be displayed before the user selects the camera 1, thereby reducing complaint inquiries.
[0074] Furthermore, when the malfunction of the network 2 is improved and the latest thumbnail image is received again from the camera 1, the determination unit 37 determines that the difference between the update date and time and the current date and time is within a predetermined time. As a result, the generation unit 38 generates GUI information including the thumbnail image and the reception part. In other words, when the malfunction of the network 2 is improved, the thumbnail image and the reception part are automatically displayed on the GUI screen, so that the burden on the provider of the live camera system 10 can be reduced.
[0075] Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and the components of the present invention can be modified and embodied without departing from the gist of the present invention. In addition, various inventions can be formed by appropriately combining multiple components disclosed in the above embodiment. In addition, some components may be deleted from the components of the above embodiment or the modified examples described below.
[0076] For example, the display mode of the GUI screen according to the present invention includes display modes other than the display modes described in the above embodiment. For example, Fig. 8 is a diagram showing another example of a GUI screen displayed by the GUI unit 44 based on the GUI information received by the terminal device 4 according to the present invention.
[0077] 8 includes a map image 50, a first location icon 61, a second location icon 62, a second window 200, a third thumbnail image 201, a second non-displayed information 202, and a fourth thumbnail image 203. The GUI unit 44 displays the first location icon 61 and the second location icon 62 superimposed on the map image 50 based on the GUI information.
[0078] The second window 200 is popped up when the user performs an operation to select the first location icon 61. The second window 200 includes a third thumbnail image 201, second non-display information 202, and a fourth thumbnail image 203. In other words, this indicates that three cameras 1 are installed at the location indicated by the first location icon 61, and that the images captured by each camera 1 can be selected to be displayed on the terminal device 4.
[0079] Since the first location icon 61 and the second window 200 are associated with the GUI information, when the user performs an operation to select the first location icon 61, the second window 200 pops up and is displayed. In the second window 200, a third thumbnail image 201, second non-display information 202, and a fourth thumbnail image 203 corresponding to the three cameras 1, respectively, are displayed in a list.
[0080] The third thumbnail image 201 and the fourth thumbnail image 203 are associated with identification information of the corresponding camera 1. The second non-display information 202 is not associated with identification information of the corresponding camera 1. The second non-display information 202 displays the message "Live video cannot be displayed due to camera malfunction."
[0081] When the user performs an operation to select the third thumbnail image 201 or the fourth thumbnail image 203, a streaming request including the identification information is transmitted to the streaming server 32. Upon receiving the streaming request, the streaming server 32 transmits corresponding streaming data to the terminal device 4. In other words, the third thumbnail image 201 and the fourth thumbnail image 203 have the same function as the reception component 102 described above.
[0082] On the other hand, since the second hidden information 202 is not associated with the identification information of the corresponding camera 1, the terminal device 4 does not accept anything and does not operate even if the user performs an operation to select the second hidden information 202. Also, since the second hidden information 202 contains a description that a video image will not be displayed, it is expected that the user will select the third thumbnail image 201 or the fourth thumbnail image 203 without selecting the second hidden information 202.
[0083] In the above embodiment, the FTP server 31 receives thumbnail images from each camera 1. However, as shown in FIG. 9, instead of acquiring thumbnail images from each camera 1, the streaming server 32 may generate thumbnail images of each camera 1 based on the video data acquired from each camera 1, and the generated thumbnail images may be acquired by the acquisition unit 35.
[0084] In this case, the FTP server 31 is not necessary and each camera 1 only needs to generate video data, reducing the burden on each camera 1. On the other hand, the streaming server 32 has to perform an additional process of generating thumbnail images from the video data of each camera 1. It is desirable to select one of the appropriate methods depending on the operating environment of the camera 1, network 2, and server 3 that constitute the system.
[0085] In the above embodiment, the determination unit 37 determines whether the acquired thumbnail image is newer than a predetermined time based on the difference between the update date and time of the thumbnail image and the current date and time, but if the thumbnail image is a JPEG format image, the Exif information includes the shooting date and time, so the determination may be made using the shooting date and time instead of the update date and time. In other words, the determination unit 37 may determine whether the difference between the shooting date and time and the current date and time is within a predetermined time.
[0086] As for the judgment method, a similar judgment can be made by a method other than that described in the above embodiment. For example, since the transmission period of thumbnail images transmitted from each camera 1 to the server 3 is constant and the transmission period is known, the server 3 calculates a judgment date and time by adding the transmission period to the update date and time of the acquired thumbnail image, and stores the calculated judgment date and time in the storage unit 33. The server 3 may judge the malfunction of the camera 1 at the next update of the thumbnail image based on whether the absolute value of the difference between the judgment date and time and the update date and time is within a predetermined time. In this judgment, if the thumbnail image is updated at the timing of the transmission period, it is determined that the camera 1 is normal, and if not, it is determined that a malfunction has occurred in the camera 1. By judging based on the absolute value, it is also possible to detect an abnormality such as transmission earlier than the default transmission period.
[0087] For example, if the transmission cycle of thumbnail images is one minute and the update date and time of an acquired thumbnail image is 15:00 on December 15, 2020, the determination date and time is 15:01 on December 15, 2020. If the next time a thumbnail image is acquired and updated, and the update date and time is 15:01:36 on December 15, 2020, the absolute value of the difference from the determination date and time will be 36 seconds. If the predetermined time used for determination is 30 seconds, the determination unit 37 determines that camera 1 is malfunctioning because the absolute value of the difference is within 30 seconds, and generation unit 38 generates GUI information including non-hidden information.
[0088] Furthermore, if the thumbnail image is a JPEG format image, the Exif information for the thumbnail image includes the shooting date and time, so the difference between the update date and time of the thumbnail image and the shooting date and time can be calculated, and if the absolute value of this difference is within a specified time, it can be determined that camera 1 is normal, and if it is not within the specified time, it can be determined that there is an abnormality in camera 1.
[0089] In the above embodiment, the terminal device 4 transmits a GUI request to the server 3, and the server 3 transmits GUI information to the terminal device 4 in a pull type manner. However, the GUI information may be transmitted to the terminal device 4 in a push type manner at the timing when the generating unit 38 generates the GUI information. In this case, the terminal device 4 needs to register the address of the terminal device 4 in advance. By using the push type, the terminal device 4 can display a GUI screen based on the latest GUI information.
[0090] For example, as shown in Fig. 7, it is assumed that the terminal device 4 displays a first window 100 including a second thumbnail image 104 and a reception component 102 as a GUI screen. After a few seconds, the communication environment between the camera 1 and the server 3 becomes poor, and new GUI information including hidden information is generated from the server 3. The server 3 pushes the generated GUI information to the terminal device 4. The terminal device 4 receives the push-transmitted GUI information and pops up the first window 100 including the first hidden information 103 as shown in Fig. 6.
[0091] By sending GUI information by push transmission from the server 3, the terminal device 4 can present the fact that the video image of the camera in question is not being displayed even if the camera 1 is not able to send images to the server 3 recently due to a malfunction of the network 2. In other words, it is possible to suppress complaints and inquiries from users.
[0092] The present invention may be the server 3 constituting the live camera system 10, or may be a GUI information generating method used in the server 3. The present invention may also be a program that causes the server 3 to execute a process for generating GUI information. It goes without saying that various modifications and applications are possible without departing from the spirit of the invention. [Industrial Applicability]
[0093] The present invention is applicable to a live camera system. [Explanation of symbols]
[0094] 1 Camera 1a First Camera 1b Second Camera 2 Network 2a First Network 2b Second Network 2c The third network 3 Server 4 Terminal Equipment 4a First terminal device 4b Second terminal device 10 Live Camera System 30 Web Server 31 FTP Server 32 Streaming Server 33 Storage section 34 Timing section 35 Acquisition Department 36 Update section 37 Judgment section 38 Generation part 39 Server Communication Department 41 Terminal communication unit 42 Terminal memory section 43 Terminal control unit 44 GUI section 50 Map Images 51 First Camera Icon 52 Secondary Camera Icon 61 First Location Icon 62 Second Location Icon 100 First Window 101 1st thumbnail image 102 Reception parts 103 First Hidden Information 104 Second thumbnail image 200 Second Window 201 3rd thumbnail image 202 Second hidden information 203 4th thumbnail image
Claims
1. A live camera system including a plurality of cameras which sequentially capture images, a server which receives moving image data generated based on the images from each of the cameras via a network, and a terminal device which sequentially obtains streaming data converted from the moving image data from the server and is capable of displaying the images, The server, an acquisition unit that sequentially acquires thumbnail images obtained by reducing the images captured by each of the cameras; The thumbnail image includes identification information for identifying the camera that captured the image, an update unit that updates the thumbnail image having the same identification information when the acquisition unit acquires the thumbnail image, and stores the updated image in association with an update date and time indicating a date and time of the update; a determination unit that determines, at a predetermined timing, whether a difference between the update date and time associated with the updated thumbnail image and a current date and time is within a predetermined time; a generation unit that generates GUI information including the thumbnail image and an acceptance component that accepts a transmission request of the streaming data of the camera identified by the identification information associated with the thumbnail image when the determination unit determines that the difference is within the predetermined time, and generates the GUI information including non-display information indicating that the latest image of the camera identified by the identification information associated with the thumbnail image cannot be displayed when the determination unit determines that the difference is not within the predetermined time, instead of the thumbnail image; a server communication unit that transmits the GUI information to the terminal device when a GUI request for the GUI information is received from the terminal device via the network; The terminal device a terminal communication unit that transmits the GUI request to the server and receives the GUI information from the server; a GUI unit that displays a GUI screen including the thumbnail image and the reception component or the non-display information based on the GUI information and receives a selection for the reception component. Live camera system.
2. the GUI information includes a map image and a camera icon indicating a position where each of the cameras is installed; the GUI unit displays a plurality of the camera icons superimposed on the map image based on the GUI information, and when a selection of any one of the plurality of the camera icons is accepted, displays the thumbnail image of the camera installed at the position indicated by the camera icon and the acceptance part, or the non-display information. The live camera system according to claim 1.
3. the GUI information includes a map image and a location icon indicating a location where each of the cameras is installed; the GUI unit displays the plurality of point icons superimposed on the map image based on the GUI information, and when a selection of any one of the plurality of point icons is accepted, displays a list including the thumbnail images of the cameras installed at the points indicated by the point icons and the reception parts, or the non-display information, based on the GUI information. The live camera system according to claim 1.
4. The thumbnail image further includes a shooting date and time indicating the date and time when the image was shot. The determination unit uses the shooting date and time instead of the update date and time. The live camera system according to any one of claims 1 to 3.
5. A server which receives, via a network, moving image data generated based on images from a plurality of cameras which sequentially capture images, and which upon receiving a request from a terminal device to transmit streaming data converted from the moving image data, sequentially transmits the streaming data to the terminal device via the network, an acquisition unit that sequentially acquires thumbnail images obtained by reducing the images captured by each of the cameras; The thumbnail image includes identification information for identifying the camera that captured the image, an update unit that updates the thumbnail image having the same identification information when the acquisition unit acquires the thumbnail image, and stores the updated image in association with an update date and time indicating a date and time of the update; a determination unit that determines, at a predetermined timing, whether a difference between the update date and time associated with the updated thumbnail image and a current date and time is within a predetermined time; a generation unit that generates GUI information including the thumbnail image and an acceptance component that accepts a transmission request for the streaming data of the camera identified by the identification information associated with the thumbnail image when the determination unit determines that the difference is within the predetermined time, and generates the GUI information including non-display information indicating that the latest image of the camera identified by the identification information associated with the thumbnail image cannot be displayed when the determination unit determines that the difference is not within the predetermined time, instead of the thumbnail image; a server communication unit that transmits the GUI information to the terminal device when a GUI request for the GUI information is received from the terminal device via the network.
Citation Information
Patent Citations
Video communication system
JP2004248069A
Video monitoring system, video information distribution apparatus, and client apparatus
JP2010161632A
Image providing system, image display device, and image display program
JP2012034216A
Electronic apparatus and method
JP2016139387A
Captured image display device, captured image display method, and captured image display program
JP2019062530A
