Video management device, video management method, and program

Abstract

This invention provides a video management device, video management method, and program that enable continued monitoring of a target even if a sudden malfunction occurs in one of multiple network cameras. [Solution] When the video management device 1 determines that the acquired image information is in the specified format at each image transmission time, it stores the acquired image information in the composite image buffer 122 in association with the composite image identification information corresponding to the arrived image transmission time. If image information cannot be acquired, or if it is determined that the acquired image information is not in the specified format, it stores dummy image information indicating a dummy image in the composite image buffer 122 in association with the composite image identification information corresponding to the arrived image transmission time.

Description

Technical Field

[0001] The present invention relates to a video management apparatus, a video management method, and a program.

Background Art

[0002] A monitoring system has been proposed that includes a plurality of network cameras, a log management device for managing logs, and an integrated console that divides a display screen into three areas: an image content switching area, a content display menu area, and a content display area, and displays a list of contents that can be displayed in the image content switching area and a list of content display methods in the content display menu area, and displays the content in the content display area using the selected content and display method (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in a system including a plurality of network cameras as described above, even when a situation occurs where an image cannot be obtained suddenly with any of the plurality of network cameras, it is required to continue monitoring by continuously obtaining and displaying images of other network cameras in which no abnormality has occurred on the display screen.

[0005] The present invention has been made in view of the above, and an object thereof is to provide a video management apparatus, a video management method, and a program that can continue monitoring a monitoring target even when a sudden abnormality occurs in any of a plurality of network cameras.

Means for Solving the Problems

[0006] To achieve the above objective, the video management device according to the present invention is A composite image buffer stores image information representing captured images or dummy images captured by each of the multiple network cameras, associated with camera identification information that identifies each of the multiple network cameras, and composite image identification information that identifies a composite image formed using the captured images or dummy images represented by each of the multiple image information components. An image acquisition unit acquires the image information indicating the captured image transmitted from each of the multiple network cameras at the image transmission time, which occurs repeatedly at a predetermined interval and in which each of the multiple network cameras transmits the image information. When the aforementioned image information is acquired, a format determination unit determines whether or not the acquired image information is in a pre-set specified format, The system includes an image management unit that, at each of the aforementioned image transmission times, if it is determined that the acquired image information is in the specified format, stores the acquired image information in the composite image buffer in association with the camera identification information that identifies the source of the image information and the composite image identification information corresponding to the arrived image transmission time. If the image information is not acquired, or if it is determined that the acquired image information is not in the specified format, stores dummy image information representing the dummy image in the composite image buffer in association with the camera identification information that identifies a network camera that does not transmit the image information and the composite image identification information corresponding to the arrived image transmission time.

[0007] From another perspective, the video management device according to the present invention is: A composite image buffer stores image information representing captured images or dummy images captured by each of the multiple network cameras, associated with camera identification information that identifies each of the multiple network cameras, and composite image identification information that identifies a composite image formed using the captured images or dummy images represented by each of the multiple image information components. An image acquisition unit that, at the transmission time when it transmits image request information to each of the multiple network cameras that repeatedly arrive at a predetermined period, transmits the image request information to the multiple network cameras, thereby acquiring the image information that represents the captured image transmitted from each of the multiple network cameras, When the aforementioned image information is acquired, a format determination unit determines whether or not the acquired image information is in a pre-set specified format, The system includes an image management unit that, at each of the aforementioned transmission times, if it is determined that the acquired image information is in the specified format, stores the acquired image information in the composite image buffer in association with the camera identification information that identifies the source of the image information and the composite image identification information corresponding to the arrival of the transmission time. If the image information is not acquired within a preset waiting time after the transmission of the image request information, or if it is determined that the acquired image information is not in the specified format, stores dummy image information representing the dummy image in the composite image buffer in association with the camera identification information that identifies a network camera that does not transmit the image information and the composite image identification information corresponding to the arrival of the transmission time. [Effects of the Invention]

[0008] According to the present invention, if the image management unit is unable to acquire image information from a network camera, or if it determines that the image information acquired from the network camera is not in the specified format, it stores dummy image information indicating a dummy image in the composite image buffer, associating it with the camera identification information of the network camera under evaluation and the composite image identification information. As a result, even if an abnormality occurs in one of the multiple network cameras, the system can continue to provide the user with a composite image that includes images acquired from other network cameras that are not experiencing abnormalities, thus enabling continuous monitoring of the target object by multiple network cameras. [Brief explanation of the drawing]

[0009] [Figure 1] This diagram shows the configuration of a video management system according to Embodiment 1 of the present invention. [Figure 2] This figure shows the hardware configuration of the video management system according to Embodiment 1. [Figure 3] This diagram shows the functional configuration of the video management system according to Embodiment 1. [Figure 4] This is a sequence diagram showing the operation of the video management system according to Embodiment 1. [Figure 5] This figure shows an example of a display image shown on the display unit of the terminal device according to Embodiment 1. [Figure 6] This is a sequence diagram showing the operation of the video management system according to Embodiment 1. [Figure 7] This figure shows an example of a display image shown on the display unit of the terminal device according to Embodiment 1. [Figure 8] This is an explanatory diagram illustrating the status of the image buffer to be synthesized in the video management system according to Embodiment 1. [Figure 9] This flowchart shows an example of the video management process flow performed by the video management device according to Embodiment 1. [Figure 10] This figure shows the functional configuration of a video management system according to Embodiment 2 of the present invention. [Figure 11] This is a sequence diagram showing the operation of the video management system according to Embodiment 2. [Figure 12] This is a sequence diagram showing the operation of the video management system according to Embodiment 2. [Figure 13] This is a sequence diagram showing the operation of the video management system according to Embodiment 2. [Figure 14] This flowchart shows an example of the video management process flow performed by the video management device according to Embodiment 2. [Modes for carrying out the invention]

[0010] (Embodiment 1) Hereinafter, a video management apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings. The video management system according to this embodiment includes a composite target image buffer, an image acquisition unit, a format determination unit, and an image management unit. The composite target image buffer stores, in association with each other, image information indicating captured images or dummy images captured by each of a plurality of network cameras, camera identification information for identifying each of the plurality of network cameras, and composite image identification information for identifying a composite image formed using the captured images or dummy images indicated by each of the plurality of image information. The image acquisition unit acquires, at the image transmission timing when each of a plurality of network cameras repeatedly arrives at a preset period and transmits image information, image information indicating the captured images transmitted from each of the plurality of network cameras. When the image information is acquired, the format determination unit determines whether the acquired image information is in a preset specified format. When it is determined that the acquired image information is in the specified format at each image transmission timing, the image management unit stores the acquired image information in the composite target image buffer in association with camera identification information for identifying the transmission source of the image information and composite image identification information corresponding to the arrived image transmission timing. When no image information is acquired, the image management unit stores a dummy image in the composite target image buffer in association with camera identification information for identifying a network camera that does not transmit image information and composite image identification information corresponding to the arrived image transmission timing. Further, when it is determined that the acquired image information is not in the specified format, the image management unit stores a dummy image in the composite target image buffer in association with camera identification information for identifying a network camera that does not transmit image information and composite image identification information corresponding to the arrived image transmission timing. Hereinafter, this embodiment will be described in detail. Note that the image information in the video management system according to an embodiment of the present invention may include not only still images but also moving images.

[0011] The video management system according to this embodiment, as shown in Figure 1, for example, comprises four network cameras 2A, 2B, 2C, and 2D (hereinafter also referred to as network cameras 2, etc.), a video management device 1 connected to these network cameras 2, etc. via a network NW, and a terminal device 3. The network NW is, for example, a wired LAN (Local Area Network) conforming to the Ethernet standard. The network cameras 2A, 2B, 2C, and 2D are used, for example, as surveillance cameras to monitor production equipment. As shown in Figure 2, the network cameras 2A, 2B, 2C, and 2D each comprise a CPU (Central Processing Unit) 201, a main memory unit 202, an auxiliary memory unit 203, an imaging unit 204, a wired communication interface 206, and a bus 209 connecting each unit. The main memory unit 202 is composed of volatile memory such as RAM (Random Access Memory). The auxiliary storage unit 203 consists of non-volatile memory such as ROM (Read Only Memory) and stores programs for controlling the network cameras 2A, 2B, 2C, and 2D, which are the devices themselves. The imaging unit 204 has, for example, a lens barrel that holds at least one lens and an image sensor (neither shown) positioned at the imaging position of at least one lens, and outputs image information indicating the captured image. The wired communication interface 206 is connected to the network NW and transmits and receives data via the network NW. The CPU 201 reads the program stored in the auxiliary storage unit 203 into the main memory unit 202 and executes it, so as shown in Figure 3, it functions as an encoding unit 211, a connection processing unit 212, and an image transmission unit 213. Furthermore, the main memory unit 202 shown in Figure 2 has a compressed image buffer 221 that temporarily stores image information of images captured by the imaging unit 204 compressed using a pre-set compression encoding scheme, as shown in Figure 3. Here, as a compression encoding scheme, for example, H.261, H.263, H.264, MPEG-1, MPEG-2, MPEG-3, etc., which utilize discrete cosine transforms, can be adopted. Alternatively, a compression encoding scheme using subband coding or wavelet transform may be adopted.

[0012] When the image information is input from the imaging unit 204, the encoding unit 211 compresses the input image information by the above-described compression encoding method and stores it in the compressed image buffer 221.

[0013] The connection processing unit 212 performs processing to establish a link with the video management device 1 and processing to establish a session. Specifically, the connection processing unit 212 acquires the FLP (First Link Pulse) burst transmitted from the video management device 1, and if the FLP burst matches the communication speed and communication mode corresponding to the network cameras 2A, 2B, 2C, and 2D indicated, negotiation with the video management device 1 is successful and the link is established. Here, the communication mode indicates either full-duplex or half-duplex communication. The connection processing unit 212 also transmits the FLP burst corresponding to the communication speed and communication mode of the network cameras 2A, 2B, 2C, and 2D to the video management device 1. After the link is established with the video management device 1, the connection processing unit 212 acquires connection request information from the video management device 1 requesting the transmission of connection information necessary to establish a session with the video management device 1, and in response, transmits the requested connection information about its own device to the video management device 1 along with response information. Here, connection information is information that defines the connection method, transmission method, media type, etc., as described by, for example, SDP (Session Description Protocol). Next, when the connection processing unit 212 receives communication start information transmitted from the video management device 1, it sends response information accordingly. Subsequently, when the connection processing unit 212 receives image transmission request information transmitted from the video distribution device 1, it generates response information accordingly and sends it to the video distribution device 1. Here, the image transmission request information is information transmitted from the video management device 1, and includes information such as the video distribution device 1 notifying the network camera 2, etc. that it is now able to receive image information captured by the network camera 2, etc. (reception permission) after communication has become possible between the video management device 1 and the network camera 2, etc. On the other hand, the response information is the response of the network camera 2, etc. to the image transmission request information. After this, image information may be transmitted from the network camera 2, etc. to the video management device 1.

[0014] After establishing a session with the video management device 1, the image transmission unit 213 receives image transmission request information. Whenever a pre-set image transmission period occurs, it identifies the image information to be transmitted from the image information stored in the compressed image buffer 221 and transmits the identified image information to the video management device 1. Here, the image transmission unit 213 sequentially identifies one image to be transmitted from the group of image information stored in the compressed image buffer 221, for example, using a FIFO (First In First Out) method. The image transmission period is set to occur at intervals of, for example, 33.33 msec. Furthermore, the image transmission period is set to be synchronized across the four network cameras 2A, 2B, 2C, and 2D. The image transmission unit 213 transmits the image information using, for example, UDP (User Datagram Protocol), RTP (Real-time Transport Protocol) / RTCP (Real-time Transport Control Protocol), or RTSP (Real Time Streaming Protocol). In this embodiment, the communication protocol used by the image transmission unit 213 is described using the four communication protocols mentioned above as examples, but it is not limited to these. Any communication protocol that can be used for audio or video data is acceptable.

[0015] Returning to Figure 2, the video management device 1 comprises a CPU 101, a main memory unit 102, an auxiliary memory unit 103, a wired communication interface 106, a wireless module 107, and a bus 109 connecting the various parts. The main memory unit 102 is composed of volatile memory such as RAM and is used as the working area for the CPU 101. The auxiliary memory unit 103 is composed of non-volatile memory such as ROM, or an SSD (Solid State Drive), HDD (Hard Disk Drive), etc., and stores a program for controlling the video management device 1. The wired communication interface 106 is connected to a network NW and transmits and receives data to and from network cameras 2, etc. via the network NW. The wireless module 107 wirelessly communicates with terminal devices 3 using a communication method that conforms to a wireless LAN standard such as IEEE 802.11.

[0016] The CPU 101 reads the program stored in the auxiliary storage unit 103 into the main storage unit 102 and executes it, so as shown in Figure 3, it functions as a connection processing unit 111, a link status determination unit 112, a session determination unit 113, an image acquisition unit 114, a decoding unit 115, a format determination unit 116, an image acquisition status determination unit 117, an image management unit 118, an image synthesis unit 119, an encoding unit 120, and an image transmission unit 140. Furthermore, the main storage unit 102 shown in Figure 2 has a compressed image buffer 121 for temporarily storing image information transmitted from network cameras 2A, 2B, 2C, and 2D, a synthesis target image buffer 122, and a transmission image buffer 123 for temporarily storing image information to be transmitted to the terminal device 3, as shown in Figure 3. The composite image buffer 122 stores image information representing captured images or dummy images (described later) taken by each of the network cameras 2A, 2B, 2C, and 2D, which are the targets for forming the composite image to be displayed on the display unit 304 of the terminal device 3. This information is associated with camera identification information that identifies each of the network cameras 2A, 2B, 2C, and 2D, and composite image identification information that identifies the composite image. Here, the composite image identification information can include, for example, time information indicating a pre-set image transmission time, or sequence number information indicating a sequence number assigned separately to each image transmission time that occurs after a session has been established between the network cameras 2A, 2B, 2C, and 2D and the video management device 1.

[0017] The connection processing unit 111 performs processes to establish a link with network cameras 2A, 2B, 2C, and 2D, and processes to establish a session. Specifically, the connection processing unit 111 acquires the FLP burst transmitted from each of the network cameras 2A, 2B, 2C, and 2D, and if it matches the communication speed and communication mode corresponding to its own device, negotiation with network cameras 2A, 2B, 2C, and 2D is successful and the link is established.

[0018] Furthermore, the connection processing unit 111 transmits an FLP burst corresponding to the communication speed and communication mode of the video management device 1 to the network cameras 2A, 2B, 2C, and 2D. Then, the connection processing unit 111 generates communication start information and transmits it to the network cameras 2A, 2B, 2C, and 2D. Next, when the connection processing unit 111 receives response information from the network cameras 2A, 2B, 2C, and 2D, it transmits image transmission request information to the aforementioned network cameras 2A, 2B, 2C, and 2D requesting the transmission of images stored in the compressed image buffer 221 of each network camera. After that, it receives response information transmitted from the network cameras 2A, 2B, 2C, and 2D and then begins to acquire image information transmitted from the network cameras 2A, 2B, 2C, and 2D. In addition, when a session release event occurs that triggers the release of the session, the connection processing unit 111 transmits session release request information to the network cameras 2A, 2B, 2C, and 2D, thereby releasing the session established with the network cameras 2A, 2B, 2C, and 2D.

[0019] The link status determination unit 112 determines the link status with each of the network cameras 2A, 2B, 2C, and 2D. Specifically, the link status determination unit 112 determines that a link is established if the communication speed and communication mode corresponding to network cameras 2A, 2B, 2C, and 2D indicated by the FLP burst acquired by the connection processing unit 111 match the communication speed and communication mode corresponding to the video management device 1, and negotiation with the video management device 1 is successful. On the other hand, the link status determination unit 112 determines that the link is broken if the connection processing unit 111 cannot acquire an FLP burst from network cameras 2A, 2B, 2C, or 2D, or if the communication speed and communication mode indicated by the FLP burst acquired by the connection processing unit 111 do not match the communication speed and communication mode corresponding to the video management device 1, and negotiation with the video management device 1 is not successful.

[0020] The session determination unit 113 determines whether a session can be established with network cameras 2A, 2B, 2C, and 2D after the link status determination unit 112 has determined that a link is established and a session establishment event occurs. The session determination unit 113 determines that a session has been established if, for example, the network cameras 2A, 2B, 2C, and 2D are RTP-compatible when the connection processing unit 111 receives response information corresponding to the communication start information transmitted from network cameras 2A, 2B, 2C, and 2D, based on the received response information. On the other hand, the session determination unit 113 determines that a session has not been established if, for example, the network cameras 2A, 2B, 2C, and 2D are not RTP-compatible when the connection processing unit 111 receives response information transmitted from network cameras 2A, 2B, 2C, and 2D, based on the received response information.

[0021] When a link and session are established between the video management device 1 and the network cameras 2A, 2B, 2C, and 2D, the image acquisition unit 114 acquires image information transmitted from each of the network cameras 2A, 2B, 2C, and 2D each time a pre-set image transmission time arrives. The image acquisition unit 114 stores the acquired image information in the compressed image buffer 121. Here, the image acquisition unit 114 acquires image information using UDP, RTP / RTCP, and RTSP. In this embodiment, the communication protocols used by the image acquisition unit 114 are described using the three communication protocols mentioned above as examples, but are not limited to these. Any communication protocol that can be used for audio and video data may be used.

[0022] The decoding unit 115 generates restored image information by performing a restoration process on the image information stored in the compressed image buffer 121, and notifies the image management unit 118 of the generated image information. The format determination unit 116 determines whether the image information has been successfully restored as a result of the restoration process by the decoding unit 115, and if successful, determines that the image information is in the specified format. Each time the decoding unit 115 performs a restoration process, the format determination unit 116 notifies the image management unit 118 of the decoding success or failure information indicating whether the image information has been successfully restored.

[0023] The image acquisition status determination unit 117 determines the acquisition status of image information transmitted from network cameras 2A, 2B, 2C, and 2D by the image acquisition unit 114 at the aforementioned image transmission time, after a link and session have been established between the video management device 1 and network cameras 2A, 2B, 2C, and 2D.

[0024] The image management unit 118, at each image transmission time, if it determines that the acquired image information is in the specified format, stores the acquired image information in the composite image buffer 122, associating it with camera identification information that identifies the source of the image information and composite image identification information corresponding to the arrived image transmission time. Furthermore, if the image acquisition status determination unit 117 determines that no image information is acquired, the image management unit 118 stores dummy image information, which represents a dummy image in place of the image information, in the composite image buffer 122, associating it with camera identification information that identifies the network camera from which no image information was acquired and composite image identification information corresponding to the arrived image transmission time. Moreover, if the image management unit 118 determines that the acquired image information is not in the specified format, it stores the aforementioned dummy image information in the composite image buffer 122, associating it with camera identification information that identifies the source of the image information and composite image identification information corresponding to the arrived image transmission time.

[0025] The image synthesis unit 119 uses the image information or dummy image information corresponding to each network camera 2A, 2B, 2C, and 2D stored in the image buffer 122 to be synthesized, along with camera identification information, to form a composite image, and notifies the encoding unit 120 of the composite image information indicating the formed composite image, along with the composite image identification information. The encoding unit 120 compresses the composite image information notified by the image synthesis unit 119 using the aforementioned compression encoding method and stores it in the transmission image buffer 123 in association with the composite image identification information. The image transmission unit 140 transmits the display image information, including the compressed composite image information and composite image identification information stored in the transmission image buffer 123, to the terminal device 3.

[0026] Returning to Figure 2, the terminal device 3 is a personal computer capable of wireless communication with the video management device 1 using a communication method conforming to a wireless LAN standard such as IEEE 802.11, and comprises a CPU 301, a main memory unit 302, an auxiliary memory unit 303, a display unit 304, a wireless module 307, and a bus 309 connecting the parts. The main memory unit 302 is composed of volatile memory such as RAM and is used as the working area for the CPU 301. The auxiliary memory unit 303 is composed of non-volatile memory such as ROM, or an SSD, HDD, etc., and stores a program for controlling the terminal device 3. The display unit 304 is a display device having, for example, a liquid crystal display. The wireless module 307 wirelessly communicates with the wireless module 107 of the video management device 1 using a communication method conforming to a wireless LAN standard such as IEEE 802.11.

[0027] The CPU 301 functions as an image acquisition unit 311, a decoding unit 312, and a display control unit 313, as shown in Figure 3, by reading the program stored in the auxiliary storage unit 303 into the main storage unit 302 and executing it. The main storage unit 302, shown in Figure 2, has a compressed image buffer 321 that temporarily stores image information transmitted from the video management device 1, and a display image storage unit 322 that stores display image information indicating the display image to be displayed on the display unit 304, as shown in Figure 3. The display image storage unit 322 stores the composite image information obtained by restoring the compressed composite image information stored in the compressed image buffer 321 by the decoding unit 312, in association with composite image identification information.

[0028] The image acquisition unit 311 acquires display image information transmitted from the video management device 1, associates the compressed composite image information and composite image identification information contained in the acquired display image information with each other, and stores them in the compressed image buffer 321. The decoding unit 312 generates restored composite image information by performing a restoration process on the composite image information stored in the compressed image buffer 321, and stores the generated composite image information in the display image storage unit 322, associating it with the composite image identification information. The display control unit 313 displays the display image indicated by the display image information stored in the display image storage unit 322 on the display unit 304.

[0029] Next, the operation of the video management system according to this embodiment will be described with reference to Figures 4 to 8. First, when power is supplied to the video management device 1, an FLP burst indicating the communication speed and communication mode of the video management device 1 is transmitted from the video management device 1 to the network cameras 2A, 2B, 2C, and 2D (step S1). Meanwhile, an FLP burst indicating the communication speed and communication mode of the network cameras 2A, 2B, 2C, and 2D is also transmitted from the network cameras 2A, 2B, 2C, and 2D to the video management device 1 (step S2). At this point, if the communication speed and communication mode indicated by the FLP burst transmitted from the network cameras 2A, 2B, 2C, and 2D match the communication speed and communication mode of the video management device 1 and negotiation is successful, the video management device 1 determines that a link with the network cameras 2A, 2B, 2C, and 2D has been established (step S3).

[0030] Next, let's assume that the aforementioned session establishment event occurs. Here, a session establishment event is, for example, a display operation in terminal device 3 to display images captured by network cameras 2A, 2B, 2C, and 2D on display unit 304. In this case, connection information request information is sent from video management device 1 to network cameras 2A, 2B, 2C, and 2D (step S5). Here, connection information is information that defines, for example, the connection method, transmission method, media type, etc., as described by SDP. On the other hand, when network cameras 2A, 2B, 2C, and 2D receive the connection information request information, they send the aforementioned connection information to video management device 1 accordingly (step S6). On the other hand, video management device 1 generates communication start information based on the connection information obtained from network cameras 2A, 2B, 2C, and 2D (step S7), and the generated communication start information is sent from video management device 1 to network cameras 2A, 2B, 2C, and 2D (step S8). Meanwhile, when network cameras 2A, 2B, 2C, and 2D acquire communication start information, they transmit response information to the video management device 1, which then acquires it (step S9).

[0031] On the other hand, the video management device 1 determines, based on the acquired response information, that network cameras 2A, 2B, 2C, and 2D are RTP compatible and that session establishment is possible (step S10).

[0032] Furthermore, network cameras 2A, 2B, 2C, and 2D capture images using the imaging unit 204. Then, the image information representing the captured image is sequentially encoded using a pre-set compression encoding method. The image is then stored in the compressed image buffer 221 (step S11).

[0033] Then, when the video management device 1 acquires response information in step S9, it transmits image transmission request information from the video management device 1 to the network cameras 2A, 2B, 2C, and 2D accordingly (step S12). On the other hand, when the network cameras 2A, 2B, 2C, and 2D acquire image transmission request information, they transmit response information from the network cameras 2A, 2B, 2C, and 2D to the video management device 1 accordingly (step S13).

[0034] Next, when the aforementioned image transmission time arrives, network cameras 2A, 2B, 2C, and 2D identify the image information to be transmitted to the video distribution device 1 from the image information stored in the compressed image buffer 221 (step S14). Subsequently, compressed image information, including the identified image information stored in the compressed image buffer 221 and camera identification information that identifies network cameras 2A, 2B, 2C, and 2D, is transmitted from network cameras 2A, 2B, 2C, and 2D to the video management device 1 (step S15).

[0035] Meanwhile, when the video management device 1 acquires compressed image information at the aforementioned image transmission time, it associates the image information contained in the acquired compressed image information with the camera identification information and stores it in the compressed image buffer 121 (step S16). Next, the video management device 1 performs a restoration process on the image information stored in the compressed image buffer 121 (step S17). Here, if the video management device 1 determines that the restoration of the image information has been successful, it associates the restored image information with the camera identification information and stores it in the composite image buffer 122 (step S18). Subsequently, the video management device 1 combines the images indicated by the image information corresponding to each network camera 2A, 2B, 2C, and 2D based on the camera identification information associated with each image information to form a composite image (step S19). After that, the video management device 1 encodes the composite image information indicating the formed composite image, associates it with the composite image identification information, and stores it in the transmission image buffer 123 as display image information (step S20). Next, assume that transmission command information instructing the video management device 1 to transmit image information to the terminal device 3 is transmitted from the terminal device 3 to the video management device 1 (step S20-2). In this case, the display image information stored in the transmission image buffer 123 is transmitted from the video management device 1 to the terminal device 3 (step S21). Meanwhile, when the terminal device 3 acquires the display image information with the image acquisition unit 311, it associates the composite image information and composite image identification information contained in the acquired display image information with each other and stores them in the compressed image buffer 321 (step S22).

[0036] Next, the terminal device 3 performs a decompression process on the display image information stored in the compressed image buffer 321 using the decoding unit 312, and stores the display image information obtained through the decompression process in the display image storage unit 322 (step S23). Subsequently, the terminal device 3 displays the display image indicated by the display image information stored in the display image storage unit 322 on the display unit 304 (step S24). Here, the terminal device 3 displays a composite image CI[j] on the display unit 304, which is a two-dimensional arrangement of images I1[j], I2[j], I3[j], I4[j] (where j is a positive integer) indicated by the image information corresponding to network cameras 2A, 2B, 2C, and 2D, as shown in Figure 5.

[0037] For example, as shown in Figure 6, suppose the video management device 1 determines that it was unable to acquire image information transmitted from the network camera 2B when the image transmission time (reception time) arrives (step S25). In this case, the video management device 1 stores the aforementioned dummy image information in the composite image buffer 122, associating it with camera identification information that identifies the network camera from which image information could not be acquired, and composite image identification information corresponding to the arrived image transmission time (step S26). The video management device 1 also performs a restoration process on the image information that was successfully received from a camera other than the network camera 2B and stored in the compressed image buffer 121 in step S16 (step S17). If it determines that the restoration of the image information was successful, it stores the restored image information in the composite image buffer 122, associating it with the camera identification information (step S18). Subsequently, as shown in steps S27 to S29, the same process as in steps S19 to S21 in Figure 4 is executed. Then, the process in steps S22 to S24 in Figure 4 is executed.

[0038] Here, if the terminal device 3 is unable to acquire image information from, for example, the network camera 2B, it displays a composite image CI[i] on the display unit 304. This composite image CI[i] is formed by arranging images I1[i], I3[i], and I4[i] corresponding to the image information of the network cameras 2A, 2C, and 2D, respectively, as shown in Figure 7, and a dummy image DI2[i] corresponding to the network camera 2B in a two-dimensional array. The dummy image DI2[i] can be, for example, an image filled with a pre-set color or pattern; in Figure 7, it is set to an image filled with gray. Note that the dummy image is not limited to an image filled with a color or pattern, but may also be an image containing specific images or text characters.

[0039] Returning to Figure 6, we will now explain what happens if, for example, an abnormality occurs in the encoding process of the captured image in the network camera 2B. In this case, the network camera 2B encodes image information indicating the image captured by the imaging unit 204 and stores it in the compressed image buffer 221 (step S31). Subsequently, the compressed image information, which includes the compressed image information stored in the compressed image buffer 221 and camera identification information that identifies the network camera 2B, is transmitted from the network camera 2B to the video management device 1 (step S32).

[0040] On the other hand, when the video management device 1 acquires the aforementioned compressed image information, it associates the compressed image information contained in the acquired compressed image information with the camera identification information and stores it in the compressed image buffer 121 (step S33). Next, the video management device 1 performs a restoration process on the image information stored in the compressed image buffer 121 (step S34). Here, suppose the video management device 1 determines that it has failed to restore the image information transmitted from the network camera 2B (step S35). In this case, the video management device 1 associates the aforementioned dummy image information with the camera identification information that identifies the network camera 2B that transmitted the image information that failed to be restored, and with the composite image identification information corresponding to the arrival of the image transmission time, and stores it in the composite image buffer 122 (step S36). On the other hand, when the video management device 1 determines that it has succeeded in restoring the image information transmitted from the network cameras 2A, 2C, and 2D stored in the compressed image buffer 121, it associates the restored image information with the camera identification information and stores it in the composite image buffer 122 (step S37). Next, as shown in steps S38 to S40, the same process as in steps S19 to S21 in Figure 4 is executed. After that, the process in steps S22 to S24 in Figure 4 is executed.

[0041] Furthermore, in the video management system according to this embodiment, it is assumed that at time T1, image information transmitted from network camera 2B becomes unavailable, and then at time T2, image information transmitted from network camera 2B becomes available again. It is also assumed that image information transmitted from other network cameras 2A, 2C, and 2D was continuously available between time T1 and time T2. In this case, for example, as shown in Figure 8, dummy images DI2[i], ..., DI2[i+18] corresponding to each image transmission time between time T1 and time T2 for network camera 2B are stored in the image buffer 122 to be synthesized. In addition, captured images I1[i], I3[i], I4[i] corresponding to each image transmission time for the other network cameras 2A, 2C, and 2D, and the dummy image DI2[i] corresponding to network camera 2B are stored in the image buffer 122 to be synthesized, associated with the same synthesized image identification information. By storing the images in the image buffer 122 in this manner, it is possible to associate the image information transmitted from other network cameras 2A, 2C, and 2D with the dummy image in chronological order from the time when image information transmitted from network camera 2B can no longer be acquired until image information can be acquired again. This allows for the formation of an appropriate composite image for the user using terminal device 3.

[0042] Next, the video management process performed by the video management device 1 according to this embodiment will be described in detail with reference to Figure 9. This video management process is started, for example, when power is turned on to the video management device 1. First, the link status determination unit 112 determines the link status with each of the network cameras 2A, 2B, 2C, and 2D (step S101). Here, the link status determination unit 112 determines, based on the FLP burst acquired by the connection processing unit 111, that negotiation with the network cameras 2A, 2B, 2C, and 2D has not been established and the link is broken (step S101: No), so the process in step S101 is executed again.

[0043] On the other hand, the link status determination unit 112 determines, based on the FLP burst acquired by the connection processing unit 111, that negotiation with network cameras 2A, 2B, 2C, and 2D has been successful and a link has been established (step S101: Yes). In this case, the connection processing unit 111 determines whether or not the aforementioned session establishment event has occurred (step S102). If the connection processing unit 111 determines that the session establishment event has not occurred (step S102: No), the process in step S101 is executed again. On the other hand, if the connection processing unit 111 determines that the session establishment event has occurred (step S102: Yes), it sends the aforementioned connection information request information to network cameras 2a, 2B, 2C, and 2D (step S103). Then, when the connection processing unit 111 obtains the aforementioned connection information from the network cameras 2A, 2B, 2C, and 2D (step S104), it generates communication start information based on the obtained connection information and sends it to the network cameras 2A, 2B, 2C, and 2D (step S105). Subsequently, when the connection processing unit 111 obtains the response information sent from the network cameras 2A, 2B, 2C, and 2D (step S106), the session determination unit 113 determines whether a session can be established with the network cameras 2A, 2B, 2C, and 2D (step S107). Here, suppose the session determination unit 113 determines, based on the response information obtained by the connection processing unit 111 from the network cameras 2A, 2B, 2C, and 2D, that one of the network cameras 2A, 2B, 2C, and 2D does not support RTP and therefore session establishment is impossible (step S107: No). In this case, the process in step S101 is executed again.

[0044] On the other hand, the session determination unit 113 determines, based on the response information obtained by the connection processing unit 111 from the network cameras 2A, 2B, 2C, and 2D, that the network cameras 2A, 2B, 2C, and 2D support RTP and that session establishment is possible (Step S107: Yes). In this case, the connection processing unit 111 sends the aforementioned image transmission request information to the network cameras 2A, 2B, 2C, and 2D (Step S108).

[0045] Subsequently, the image acquisition status determination unit 117 determines whether or not the aforementioned image transmission time has arrived (step S109). If the image acquisition status determination unit 117 determines that the image transmission time has not yet arrived (step S109: No), the process of step S120 described later is executed. On the other hand, if the image acquisition status determination unit 117 determines that the image transmission time has arrived (step S109: Yes), it determines whether or not the image information transmitted from network cameras 2A, 2B, 2C, and 2D has been acquired by the image acquisition unit 114 (step S110). If the image acquisition status determination unit 117 determines that the image acquisition unit 114 was unable to acquire image information from any of the network cameras (step S110: No), the image management unit 118 generates the aforementioned dummy image information and stores it in the composite image buffer 122, associating it with camera identification information that identifies network cameras that do not transmit image information and composite image identification information corresponding to the arrived image transmission time (step S111). Next, the process described in step S116 below is executed.

[0046] On the other hand, if the image acquisition status determination unit 117 determines that image information has been acquired from the network camera by the image acquisition unit 114 (step S110: Yes), the image acquisition unit 114 stores the acquired image information in the compressed image buffer 121 (step S112). Subsequently, the decoding unit 115 performs a restoration process on the image information stored in the compressed image buffer 121 (step S113). After that, the format determination unit 116 determines whether or not the image information has been successfully restored as a result of the restoration process by the decoding unit 115 (step S114). Here, suppose the format determination unit 116 determines that the decoding unit 115 failed to restore the image information (step S114: No). In this case, the image management unit 118 executes the process in step S111 described above, and then proceeds to the process in step S116 described below. On the other hand, suppose the format determination unit 116 determines that the decoding unit 115 has successfully restored the image information (step S114: Yes). In this case, the image management unit 118 stores image information indicating the restored image in the composite image buffer 122, associating it with camera identification information that identifies the source of the image information transmission and composite image identification information corresponding to the arrival time of the image transmission (step S115).

[0047] Next, the image synthesis unit 119 determines whether it has stored the corresponding image information or dummy image information for all network cameras 2A, 2B, 2C, and 2D in the image buffer 122 to be synthesized (step S116). Suppose the image synthesis unit 119 determines that it has not stored the corresponding image information or dummy image information for any one of the network cameras 2A, 2B, 2C, or 2D in the image buffer 122 to be synthesized (step S116: No). In this case, the process in step S110 is executed again. On the other hand, suppose the image synthesis unit 119 determines that it has stored the corresponding image information or dummy image information for all of the network cameras 2A, 2B, 2C, and 2D in the image buffer 122 to be synthesized (step S116: Yes). In this case, the image synthesis unit 119 uses the image information or dummy image information corresponding to each network camera 2A, 2B, 2C, and 2D stored in the image buffer 122 to be synthesized, along with camera identification information, to form a composite image (step S117). Subsequently, the encoding unit 120 encodes the composite image information representing the composite image formed by the image synthesis unit 119 using the aforementioned compression encoding method, and stores it in the transmission image buffer 123 in association with the composite image identification information (step S118). At this point, if the image transmission unit 140 obtains the aforementioned transmission command information from the terminal device 3, it transmits the display image information, including the compressed composite image information and composite image identification information stored in the transmission image buffer 123, to the terminal device 3.

[0048] Next, the connection processing unit 111 determines whether or not the aforementioned session release event has occurred (step S120). If the connection processing unit 111 determines that the session release event has not occurred (step S120: No), the process in step S109 is executed again, and the system waits for the timing to acquire the next image. On the other hand, if the connection processing unit 111 determines that the session release event has occurred (step S120: Yes), the process in step S101 is executed again.

[0049] As described above, according to the video management device 1 of this embodiment, if the image management unit 118 is unable to acquire image information from any (or more) network cameras 2A, 2B, 2C, 2D, or if it determines that any (or more) of the image information acquired from network cameras 2A, 2B, 2C, 2D is not in the specified format and the image cannot be restored, it stores dummy image information indicating a dummy image in the composite image buffer 122, associating it with the camera identification information of the network camera 2A, 2B, 2C, 2D that is the source of the image information that could not be received or was determined not to be in the specified format, and the composite image identification information. As a result, even if an abnormality occurs in any of the multiple network cameras 2A, 2B, 2C, 2D, it is possible to continue providing the user with a composite image that includes images acquired from other network cameras that are not experiencing abnormalities. This allows for continued monitoring of the target of surveillance by multiple network cameras 2A, 2B, 2C, 2D, and makes it easy to determine which network camera's images cannot be displayed.

[0050] (Embodiment 2) The video management system according to this embodiment differs from Embodiment 1, in that each of the multiple network cameras autonomously transmits captured images to the video management device, in that the video management device obtains image information indicating the captured images transmitted from each of the multiple network cameras by sending image request information to the multiple network cameras at transmission times that occur repeatedly at a predetermined interval. This embodiment will be described in detail below.

[0051] As shown in Figure 10, the imaging system according to this embodiment comprises network cameras 2002A, 2002B, 2002C, and 2002D (hereinafter, the four network cameras are collectively referred to as network camera 2002), a video management device 2001, and a terminal device 3. In Figure 10, components similar to those in Embodiment 1 are denoted by the same reference numerals as in Figure 3. Furthermore, the hardware configuration of the video management system according to this embodiment is the same as the hardware configuration of the video management system described in Embodiment 1. Therefore, the hardware configuration according to this embodiment will be described using the reference numerals shown in Figure 2 as appropriate.

[0052] In each network camera 2002, the CPU 201 functions as the encoding unit 211, connection processing unit 212, and image transmission unit 2213 by reading a program stored in the auxiliary storage unit 203 into the main storage unit 202 and executing it. When the image transmission unit 2213 receives image request information requesting the transmission of image information to be sent from the video management device 2001 to each network camera 2002, it identifies the image information to be sent to the video management device 2001 from the image information stored in the compressed image buffer 221 and transmits the identified image information to the video management device 2001. In this way, each network camera 2002 transmits the identified image information to the video management device 2001 in response to the acquired image request information.

[0053] In the video management device 2001, the CPU 101 functions as a connection processing unit 111, a link status determination unit 112, a session determination unit 113, an image acquisition unit 2114, a decoding unit 115, a format determination unit 116, an image acquisition status determination unit 2117, an image management unit 118, an image synthesis unit 119, an encoding unit 120, and an image transmission unit 140 by reading a program stored in the auxiliary storage unit 103 into the main storage unit 102 and executing it.

[0054] When a link and session are established between the video management device 2001 and the network camera 2002, the image acquisition unit 2114 transmits the aforementioned image request information to the network camera 2002 at the transmission time, which occurs repeatedly at a predetermined interval, thereby acquiring image information transmitted from each of the network cameras 2002. The image acquisition unit 2114 then stores the acquired image information in the compressed image buffer 121.

[0055] The image acquisition status determination unit 2117 determines whether or not it has acquired image information transmitted from the network camera 2002 within a predetermined time after the image acquisition unit 2114 has transmitted image request information to the network camera 2002, following the establishment of a link and session between the video management device 2001 and the network camera 2002. The network camera 2002, the remaining parts of the video management device 2001, and the terminal device 3 are the same as in Embodiment 1, so their description is omitted.

[0056] Next, the operation of the video management system according to this embodiment will be described with reference to Figures 11 to 13. Figure 11 corresponds to Figure 4 in Embodiment 1, and Figures 12 and 13 correspond to Figure 6 in Embodiment 1, with the same reference numerals used for the same operations. First, when power is supplied to the video management device 2001, a series of processes from step S1 to step S9 described in Embodiment 1 are executed. Then, the video management device 2001 determines, based on the acquired response information, that the network camera 2002 supports RTP and that session establishment is possible (step S11). Also, a series of processes from step S12 to S14 described in Embodiment 1 are executed. Next, in Embodiment 2, unlike Embodiment 1, when the time for image acquisition arrives, image request information is sent from the video management device 2001 to each of the network cameras 2002 (step S2001). Meanwhile, when each network camera 2002 receives image request information, it identifies the image information to be sent to the video management device 2001 from the image information stored in the compressed image buffer 221 (step S2002). Next, compressed image information, including the identified image information and camera identification information that identifies the network camera 2002, is sent from the network camera 2002 to the video management device 2001 (step S2003).

[0057] On the other hand, when the video management device 2001 acquires compressed image information, it associates the compressed image information contained in the acquired compressed image information with the camera identification information and stores them in the compressed image buffer 121 (step S16), and then the series of processes from steps S17 to S24 described in Embodiment 1 are executed.

[0058] In contrast, as shown in Figure 12, suppose that after the image acquisition time arrives and the video management device 2001 executes the process of sending the aforementioned image request information to the network cameras 2002A to 2002D (step S2004), it is determined that only 2002B was unable to acquire compressed image information within a predetermined time (step S2005). In this case, the video management device 2001 stores dummy image information similar to that in Embodiment 1 in the image buffer 122, associating it with camera identification information that identifies the network camera (2002B) from which compressed image information could not be acquired, and composite image identification information corresponding to the arrived image transmission time (step S2006). Furthermore, the video management device 2001 performs a restoration process on the image information transmitted from network cameras 2002A, 2002C, and 2002D stored in the compressed image buffer 121 (step S17). If it determines that the restoration of the image information has been successful, it stores the restored image information in the composite image buffer 122, associating it with the camera identification information (step S18). Subsequently, the video management device 2001 combines the images shown by the image information corresponding to each of the network cameras 2002A, 2002C, and 2002D with a dummy image corresponding to network camera 2002B to form a composite image (step S27). Next, the video management device 2001 encodes the composite image information showing the formed composite image and stores it in the transmission image buffer 123, associating it with the composite image identification information (step S28). Subsequently, the display image information, including the composite image information and composite image identification information stored in the transmission image buffer 123, is transmitted from the video management device 2001 to the terminal device 3 (step S29). Meanwhile, when terminal device 3 acquires display image information, it associates the composite image information and composite image identification information contained in the acquired display image information with each other and stores them in the compressed image buffer 321 (step S30). After that, the decoding and storage (step S23) and display (step S24) processes are executed, as in Embodiment 1.

[0059] Figure 12 also illustrates the sequence of events in the network camera 2002B when, for example, an encoding error occurs in the captured image. When it is time to acquire an image, the aforementioned image request information is sent from the video management device 2001 to the network camera 2002B (step S2007). Upon receiving the image request information, the network camera 2002B identifies the image information to be sent to the video management device 2001 from the image information stored in the compressed image buffer 221 (step S2008). Next, compressed image information, including the identified image information and camera identification information that identifies the network camera 2002B, is sent from the network camera 2002B to the video management device 2001 (step S2009).

[0060] On the other hand, when the video management device 2001 acquires the aforementioned compressed image information, it associates the compressed image information contained in the acquired compressed image information with the camera identification information and stores it in the compressed image buffer 121 (step S33). Next, the video management device 2001 performs a restoration process on the image information stored in the compressed image buffer 121 (step S34). Here, suppose the video management device 2001 determines that it has failed to restore the image information transmitted from the network camera 2002B (step S35). In this case, the video management device 2001 associates the aforementioned dummy image information with the camera identification information that identifies the network camera 2002B that transmitted the image information that failed to be restored, and with the composite image identification information corresponding to the arrival of the image transmission time, and stores it in the composite image buffer 122 (step S36). After that, as shown in Figure 13, a series of processes from step S37 onwards are executed, similar to Embodiment 1.

[0061] Next, the video management processing performed by the video management device 2001 according to this embodiment will be described in detail with reference to Figure 14. Here, in Figure 14, the same reference numerals as in Figure 9 are used for processing that is the same as in Embodiment 1. First, after a series of processes from step S101 to step S106 are executed, the session determination unit 113 determines, based on the response information obtained by the connection processing unit 111 from each of the network cameras 2002, that network cameras 2002A, 2002B, 2002C, and 2002D are RTP compatible and that session establishment is possible (step S107: Yes). In this case, the connection processing unit 111 sends the aforementioned image transmission request information to each of the network cameras 2002 (step S108). Next, the image acquisition unit 2114 determines whether or not the aforementioned image acquisition time has arrived (step S2101). Here, if the image acquisition unit 2114 determines that the image acquisition time has not yet arrived (step S2101: No), the process of step S120, which will be described later, is executed. On the other hand, when the image acquisition unit 2114 determines that the time for image acquisition has arrived (step S2101: Yes), it sends the aforementioned image request information to each of the network cameras 2002 (step S2102). Next, the image acquisition status determination unit 2117 determines whether or not it has acquired the image information transmitted from each of the network cameras 2002 within a predetermined waiting time after the image acquisition unit 2114 has sent the image request information (step S2103). Here, suppose the image acquisition status determination unit 2117 determines that it was not able to acquire the image information transmitted from each of the network cameras 2002 within the aforementioned waiting time (step S2103: No). In this case, the image management unit 118 generates the aforementioned dummy image information and stores it in the composite image buffer 122, associating it with camera identification information that identifies network cameras that do not transmit image information and composite image identification information corresponding to the arrived image transmission time (step S111), and the process in step S116 is executed. On the other hand, if the image acquisition status determination unit 2117 determines that image information has been acquired by the image acquisition unit 2114 within the waiting time (step S2103: Yes), the image acquisition unit 2114 stores the acquired image information in the compressed image buffer 121 (step S2104).Next, the decoding unit 115 performs a restoration process on the image information stored in the compressed image buffer 121 (step S113). After that, the format determination unit 116 determines whether or not the image information has been successfully restored as a result of the restoration process by the decoding unit 115 (step S114). Here, suppose the format determination unit 116 determines that the decoding unit 115 has failed to restore the image information (step S114: No). In this case, the image management unit 118 performs the process in step S111 described above, and then performs the process in step S116 described below. On the other hand, suppose the format determination unit 116 determines that the decoding unit 115 has successfully restored the image information (step S114: Yes). In this case, the image management unit 118 stores the image information representing the restored image in the composite image buffer 122, associating it with camera identification information that identifies the source of the image information and composite image identification information corresponding to the arrival of the image transmission time (step S115). Subsequently, the processes from step S116 onwards, as described in Embodiment 1, are executed.

[0062] As described above, even if an abnormality occurs in any of the network cameras 2002A, 2002B, 2002C, or 2002D, the video management device 2001 according to this embodiment can continue to provide users with a composite image that includes images acquired from the other network cameras 2002A, 2002B, 2002C, or 2002D that are not experiencing any abnormalities. This allows for continued monitoring of the target being monitored by the network cameras 2002A, 2002B, 2002C, or 2002D, and makes it easy to determine which network cameras' images cannot be displayed.

[0063] (Another embodiment) Although various embodiments of the present invention have been described above, the present invention is not limited to the configurations of the embodiments described above. For example, the network NW connecting the video management device 1, 2001 and the network cameras 2A, 2B, 2C, 2D, 2002A, 2002B, 2002C, 2002D may be a wireless LAN. In this case, for example, in Embodiment 1, when power is supplied to the video management device 1, probe response information including probe request information for searching for network cameras 2A, 2B, 2C, 2D, SSID information of network cameras 2A, 2B, 2C, 2D, channel information used, etc., authentication information for authenticating the video management device 1, and association information including information indicating that each of the network cameras 2A, 2B, 2C, 2D is a device that communicates wirelessly with the video management device 1 are transmitted and received between the video management device 1 and the network cameras 2A, 2B, 2C, 2D. After that, a series of processes from step S4 onwards are executed. The video management device 1 may also be configured to communicate with network cameras 2A, 2B, 2C, and 2D via an access point (not shown) that is wired to the network cameras 2A, 2B, 2C, and 2D. In this case, the video management device 1 may determine that a link with network cameras 2A, 2B, 2C, and 2D has been established after sending probe request information to the access point, obtaining probe response information, and then sending authentication information and connection request information to the access point, and obtaining connection response information from the access point. With this configuration, the same effects as in the embodiment can be obtained even when the network NW is configured as a wireless LAN.

[0064] In each embodiment, an example was described in which the session determination unit 113 determines whether a session can be established based on connection information acquired by the connection processing unit 111 from the network cameras 2A, 2B, 2C, and 2D, and whether the network cameras 2A, 2B, 2C, and 2D support RTP. However, the invention is not limited to this, and for example, the session determination unit 113 may determine whether a session can be established based on whether the header information corresponding to the application layer protocol included in the image information acquired by the image acquisition unit 114 conforms to RTP.

[0065] Alternatively, the session determination unit 113 may determine whether a session can be established based on whether the data format of the image information payload matches a pre-set format when the image acquisition unit 114 acquires image information. Alternatively, the session determination unit 113 may determine whether a session can be established based on whether a coding error is detected based on the checksum contained in the header information corresponding to the UDP contained in the image information when the image acquisition unit 114 acquires image information.

[0066] Furthermore, in each embodiment, the format determination unit 116 determines whether the image information is in a specified format by having the decoding unit 115 attempt to restore the image information acquired from the network camera. For example, the format determination unit may determine whether it is in a specified format based on header information corresponding to the application layer protocol included in the image information acquired by the image acquisition unit 114. In this case, the format determination unit 116 may be provided before the decoding unit 115. Also, for example, suppose the captured image is a color image, and the network camera transmits the image information, which includes information corresponding to each of the RGB values, to the video management device in a predetermined transmission order. In this case, the format determination unit 116 may determine that it is in a specified format if the transmission order of the information corresponding to each of the RGB values ​​included in the image information transmitted from the network camera is in a predetermined order, that is, if the arrangement order of the information corresponding to each of the RGB values ​​included in the image information is in a predetermined arrangement order.

[0067] Furthermore, it is not essential for the network camera to encode and compress the captured images and transmit them to the video management device, and for the video management device to decode and restore them, in order to implement the present invention. The network camera may send uncompressed images to the video management device. Of course, transmitting uncompressed images increases the network load between the network camera and the video management device, but it simplifies the functions of both devices and avoids the risk of restoration failure due to mismatch in compression encoding formats.

[0068] The various functions of the video management device 1, 2001 according to the present invention can be realized using a computer system equipped with a wireless communication module, without relying on a dedicated system. For example, a program for performing the above operations may be stored on a non-temporary recording medium (such as a CD-ROM) that can be read by the computer system and distributed to a computer connected to a network, and the video management device 1 and terminal device 3 that perform the above processing may be configured by installing the program on the computer system.

[0069] Furthermore, the method of providing the program to the computer is arbitrary. For example, the program may be uploaded to a server on a communication line and distributed to the computer via the communication line. The computer then launches this program and executes it under the control of the OS, just like any other application. In this way, the computer functions as a video management device 1, 2001 that performs the processes described above.

[0070] Although embodiments and variations of the present invention have been described above, the present invention is not limited thereto. The present invention includes embodiments and variations that are appropriately combined, and those that are appropriately modified thereto. [Industrial applicability]

[0071] This invention is suitable for surveillance systems using multiple network cameras. [Explanation of Symbols]

[0072] 1, 2001: Video management device, 2A, 2B, 2C, 2D, 2002A, 2002B, 2002C, 2002D: Network camera, 3: Terminal device, 101, 201, 301: CPU, 102, 202, 302: Main memory unit, 103, 203, 303: Auxiliary memory unit, 106, 206: Wired communication interface, 107, 307: Wireless module, 109, 209, 309: Bus, 111, 212: Connection processing unit, 112: Link status determination unit, 113: Session determination unit, 114, 311, 2114: Image acquisition unit, 115, 312: Decode unit, 116: Format Image acquisition status determination unit, 117, 2117: Image acquisition status determination unit, 118: Image management unit, 119: Image synthesis unit, 120, 211: Encoding unit, 121, 221, 321: Compressed image buffer, 122: Image buffer to be synthesized, 123: Transmitted image buffer, 140, 213, 2213: Image transmission unit, 204: Imaging unit, 304: Display unit, 313: Display control unit, 322: Display image storage unit, IC[*]: Synthesized image, I1[*], I2[*], I3[*], I4[*]: Image, DI2[*]: Dummy image, NW: Network

Claims

1. A composite image buffer stores image information representing captured images or dummy images captured by each of the multiple network cameras, associated with camera identification information that identifies each of the multiple network cameras, and composite image identification information that identifies a composite image formed using the captured images or dummy images represented by each of the multiple image information components. An image acquisition unit acquires the image information representing the captured image transmitted from each of the multiple network cameras at the image transmission time, which occurs repeatedly at a predetermined interval and in which each of the multiple network cameras transmits the image information. When the aforementioned image information is acquired, a format determination unit determines whether or not the acquired image information is in a pre-set specified format, The image management unit, when it is determined that the acquired image information is in the specified format at each of the aforementioned image transmission times, stores the acquired image information in the composite image buffer in association with the camera identification information that identifies the source of the image information and the composite image identification information corresponding to the arrived image transmission time. If the image information is not acquired, or when it is determined that the acquired image information is not in the specified format, stores dummy image information representing the dummy image in the composite image buffer in association with the camera identification information that identifies a network camera that does not transmit the image information and the composite image identification information corresponding to the arrived image transmission time. Video management device.

2. A composite image buffer stores image information representing captured images or dummy images captured by each of the multiple network cameras, associated with camera identification information that identifies each of the multiple network cameras, and composite image identification information that identifies a composite image formed using the captured images or dummy images represented by each of the multiple image information components. An image acquisition unit that, at the transmission time when it transmits image request information to each of the multiple network cameras that repeatedly arrive at a predetermined period, transmits the image request information to the multiple network cameras, thereby acquiring the image information that represents the captured image transmitted from each of the multiple network cameras, When the aforementioned image information is acquired, a format determination unit determines whether or not the acquired image information is in a pre-set specified format, The system includes an image management unit that, at each of the aforementioned transmission times, if it is determined that the acquired image information is in the specified format, stores the acquired image information in the composite image buffer in association with the camera identification information that identifies the source of the image information and the composite image identification information corresponding to the arrival of the transmission time; and if the image information is not acquired within a preset waiting time after the transmission of the image request information, or if it is determined that the acquired image information is not in the specified format, stores dummy image information representing the dummy image in the composite image buffer in association with the camera identification information that identifies a network camera that does not transmit the image information and the composite image identification information corresponding to the arrival of the transmission time. Video management device.

3. The image synthesis unit further comprises an image synthesis unit that forms the synthesized image using captured images or dummy images indicated by each of the multiple pieces of image information corresponding to the same synthesized image identification information stored in the image buffer to be synthesized. The video management device according to claim 1 or 2.

4. The specified format refers to a data format that has been set in advance for the image information. The format determination unit determines that the image information is in the specified format if it can restore the captured image by performing a restoration process on the image information that corresponds to the data format. The video management device according to claim 1 or 2.

5. The aforementioned specified format indicates a format that is compressed using a pre-configured compression encoding scheme. The format determination unit determines that the image information has been compressed and encoded in the specified format if the captured image can be restored using the restoration process corresponding to the compression encoding method. The video management device according to claim 1 or 2.

6. The steps include: acquiring image information indicating images captured by each of the multiple network cameras transmitted from each of the multiple network cameras at the image transmission time, which occurs repeatedly at a predetermined interval and where each of the multiple network cameras transmits image information indicating images captured by each of the multiple network cameras; If the image information is acquired during the image transmission period, the step of determining whether the acquired image information is in a pre-set specified format, When it is determined that the acquired image information is in the specified format, the acquired image information is stored in a composite image buffer, associating it with camera identification information that identifies the network camera that transmitted the image information, and composite image identification information that identifies a composite image formed using the captured images or dummy images shown by each of the multiple pieces of image information corresponding to the arrival of the image transmission time. If it is determined that the acquired image information is not in the specified format, the process involves storing dummy image information representing the dummy image in the composite image buffer, associating it with the camera identification information that identifies the source of the image information and the composite image identification information that corresponds to the time of the image transmission. If the image information is not acquired at the image transmission time, the step includes storing the dummy image information in the composite image buffer in association with the camera identification information that identifies a network camera that does not transmit the image information and the composite image identification information that corresponds to the arrived image transmission time. Video management methods.

7. The process involves transmitting image request information to multiple network cameras that repeatedly occur at a predetermined interval, requesting the transmission of image information indicating captured images from each of the multiple network cameras, and obtaining the image information indicating the captured images transmitted from each of the multiple network cameras by transmitting the image request information to the multiple network cameras. If the image information is acquired at the time of transmission, the step of determining whether the acquired image information is in a pre-set specified format, When it is determined that the acquired image information is in the specified format, the acquired image information is stored in a composite image buffer, associating it with camera identification information that identifies the network camera that transmitted the image information, and composite image identification information that identifies a composite image formed using the captured images or dummy images shown by each of the multiple pieces of image information corresponding to the time of transmission. If it is determined that the acquired image information is not in the specified format, the process involves storing dummy image information representing the dummy image in the composite image buffer, associating it with the camera identification information that identifies the source of the image information and the composite image identification information corresponding to the time of transmission. If, at the transmission time, the image information is not acquired within a predetermined waiting time after the image request information has been transmitted, the dummy image information is stored in the composite image buffer in association with camera identification information that identifies a network camera that does not transmit the image information and composite image identification information corresponding to the arrived transmission time. Video management methods.

8. Computers, A composite image buffer stores image information representing captured images or dummy images captured by each of multiple network cameras, associating it with camera identification information that identifies each of the multiple network cameras and composite image identification information that identifies a composite image formed using the captured images or dummy images represented by each of the multiple image information components. An image acquisition unit acquires the image information representing the captured image transmitted from each of the multiple network cameras at the image transmission time, which occurs repeatedly at a predetermined interval and in which each of the multiple network cameras transmits the image information. When the aforementioned image information is acquired, a format determination unit determines whether or not the acquired image information is in a pre-set specified format. At each of the aforementioned image transmission times, if it is determined that the acquired image information is in the specified format, the image management unit stores the acquired image information in the composite image buffer in association with the camera identification information that identifies the source of the image information and the composite image identification information corresponding to the arrived image transmission time. If the image information is not acquired, or if it is determined that the acquired image information is not in the specified format, the image management unit stores dummy image information indicating the dummy image in the composite image buffer in association with the camera identification information that identifies a network camera that does not transmit the image information and the composite image identification information corresponding to the arrived image transmission time. A program designed to function as such.

9. Computers, A composite image buffer stores image information representing captured images or dummy images captured by each of multiple network cameras, associating it with camera identification information that identifies each of the multiple network cameras and composite image identification information that identifies a composite image formed using the captured images or dummy images represented by each of the multiple image information components. An image acquisition unit, which transmits image request information to each of the multiple network cameras that repeatedly arrive at a predetermined period, to each of the multiple network cameras, thereby acquiring the image information that represents the captured image transmitted from each of the multiple network cameras, at the transmission time when it transmits image request information to each of the multiple network cameras that repeatedly arrive at a predetermined period. When the aforementioned image information is acquired, a format determination unit determines whether or not the acquired image information is in a pre-set specified format. At each of the aforementioned transmission times, if it is determined that the acquired image information is in the specified format, the image management unit stores the acquired image information in the composite image buffer, associating it with the camera identification information that identifies the source of the image information and the composite image identification information corresponding to the arrival of the transmission time. If the image information is not acquired within a predetermined waiting time after the transmission of the image request information, or if it is determined that the acquired image information is not in the specified format, the image management unit stores dummy image information representing the dummy image in the composite image buffer, associating it with the camera identification information that identifies a network camera that does not transmit the image information and the composite image identification information corresponding to the arrival of the transmission time. A program designed to function as such.

Images