Distribution server, remote monitoring system, remote monitoring method and program

Abstract

To provide a distribution server, remote monitoring system, remote monitoring method, and program that use a plurality of networks.SOLUTION: A distribution server according to an embodiment includes: a network camera processing unit that receives moving image data output by a network camera in real time and converts the moving image data using one or more parameters to generate distribution data; a recorded data processing unit that records the moving image data and generates the recorded data; and a web server that outputs the distribution data to an external device in real time. The web server outputs the recorded data in response to a download request from the external device.SELECTED DRAWING: Figure 2

Description

Technical Field

[0005]

[0001] Embodiments of this invention relate to a distribution server, a remote monitoring system, a remote monitoring method, and a program applicable to remote monitoring.

Background Art

[0002] In recent years, the opportunities for work management supervisors to be absent from the site in factories and plants have been increasing, and it has become necessary to establish a mechanism to prevent the quantity and quality of on-site safety management from deteriorating. As an example, there is a remote monitoring system that mounts a network camera on a trackless AGV (Automated Guided Vehicle: unmanned transport vehicle), automatically patrols the site, and distributes real-time moving images to a remote terminal, thereby remotely monitoring on-site workers and safely patrolling the work site.

[0003] In a conventional remote monitoring system, real-time moving images of a network camera can be reproduced by a reproduction means on a web browser, so that they can be freely viewed from a remote terminal connected to an intranet or a general internet used by administrators of factories, plants, etc. In recent years, with the introduction of local 5G and the like, the network connecting the network camera has become larger in capacity, and the transmission of higher-definition moving images is expected.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

[0006] The problem that this invention aims to solve is to provide a distribution server, a remote monitoring system, a remote monitoring method, and a program that utilize multiple networks. [Means for solving the problem]

[0007] The distribution server according to this embodiment receives video data output by the network camera in real time and distributes the video data. At least the video bitrate, video resolution, video frame rate, video compression method, and image size. A network camera processing unit that converts and generates distribution data using one or more parameters, and a recording data processing unit that records the video data and generates recording data, A web screen for changing the parameter settings is generated and output to an external device, The aforementioned distribution data The aforementioned It includes a web server that outputs to external devices in real time, The network camera processing unit generates multiple distribution data converted with different combinations of the parameters, and switches the output distribution data according to the combination of parameters selected from the web screen for changing the parameter settings on the external device. The web server outputs the recorded data in response to a download request from the external device. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is an illustrative diagram showing a remote monitoring system according to an embodiment. [Figure 2] Figure 2 is a configuration diagram showing an example of the configuration of the distribution server functions according to the first embodiment. [Figure 3] Figure 3 is a flowchart showing the processing of the distribution server according to the first embodiment. [Figure 4] Figure 4 is a configuration diagram showing an example of the configuration of the distribution server functions according to the second embodiment. [Figure 5] Figure 5 is a configuration diagram showing an example of the configuration of the distribution server functions according to the third embodiment. [Figure 6] Figure 6 is a flowchart showing the processing of the distribution server according to the third embodiment. [Figure 7]FIG. 7 is a configuration diagram showing a configuration example of the functions of a distribution server according to the fourth embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a tag data input / display screen generated by the distribution server according to the fourth embodiment. [Figure 9] FIG. 9 is a flowchart showing the processing of the distribution server according to the fourth embodiment. [Figure 10] FIG. 10 is a configuration diagram showing a configuration example of the functions of a distribution server according to the fifth embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a distribution screen generated by the distribution server according to the fifth embodiment. [Figure 12] FIG. 12 is a flowchart showing the processing of the distribution server according to the fifth embodiment. [Figure 13] FIG. 13 is an image diagram showing a remote monitoring system according to the sixth embodiment. [Figure 14] FIG. 14 is a configuration diagram showing a configuration example of the functions of a distribution server according to the sixth embodiment. [Figure 15] FIG. 15 is a flowchart showing the first processing of the distribution server according to the sixth embodiment. [Figure 16] FIG. 16 is a diagram showing an example of data acquired by the distribution server according to the sixth embodiment. [Figure 17] FIG. 17 is a flowchart showing the second processing of the distribution server according to the sixth embodiment. [Figure 18] FIG. 18 is a diagram showing an example of a tag data display screen generated by the distribution server according to a modification.

MODE FOR CARRYING OUT THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. (First Embodiment) The distribution server in the remote monitoring system of this embodiment compresses and converts (reduces the data volume) the data of high-definition moving images output by the network camera connected to Network A to generate distribution data and performs stream distribution. The remote terminal connected to Network B with a smaller communication capacity than Network A can view the moving images by receiving the distribution data. At the same time, the distribution server records the moving image data. The remote terminal downloads the recorded data to view high-definition moving images.

[0010] FIG. 1 is an image diagram showing a remote monitoring system according to an embodiment.

[0011] The remote monitoring system of this embodiment includes a network camera 1, a distribution server 2, and a plurality of remote terminals 3. The network camera 1 and the distribution server 2 are connected via Network 4A, and the distribution server 2 and the remote terminal 3 are connected via Network 4B. Basically, in the remote monitoring system, the monitor views the moving images captured by the network camera 1 installed remotely on the monitor of the remote terminal 3 or the like, and performs monitoring related to the installation location of the network camera 1. Hereinafter, the operator of the remote terminal 3 is referred to as the user or the monitor.

[0012] The network camera 1 is a network camera capable of capturing moving images, and includes communication means for outputting the captured moving image data, video stream, etc. to an external device. The network camera 1 in this embodiment is connected to Network 4A via a router 15. The network camera 1 includes an interface such as a USB or a LAN cable for connecting to the router 15.

[0013] Router 15 is a router for connecting network camera 1 to network 4A, and is equipped with interfaces for connecting network camera 1 and network 4A. Router 15 is equipped with interfaces such as USB and LAN cables for connecting to network camera 1. Router 15 is also equipped with an interface for connecting to network 4A and associated communication means 4A. Communication means 4A is not limited to wired or wireless, and may be any communication method such as Ethernet, Bluetooth®, WiFi®, or local 5G based on 5G communication (5th Generation communication). If communication means 4A is wireless, the interface will also include an antenna. Note that network camera 1 may also include the functions of router 15 and be directly connected to network 4A equipped with communication means 4A.

[0014] The distribution server 2 is a server that distributes various content such as video and audio, video streams, and web pages to the client remote terminal 3, and may be composed of a computer equipped with a CPU and memory, for example. The distribution server 2 acquires video and image data captured by the network camera 1 via the network 4, generates distribution data from the acquired video and image data, and outputs it to the remote terminal 3.

[0015] One or more remote terminals 3-11, 3-12, and 3-13 (referred to as remote terminal 3 unless otherwise specified) are terminals for remote monitoring by a monitorer by acquiring video data, video streams, etc., from the network camera 1. Remote terminal 3 may be composed of a computer equipped with a CPU, memory, etc., such as a PC or smartphone. Remote terminal 3 is equipped with an interface that can connect to network 4B and associated communication means 4A. Remote terminal 3 is equipped with data input means such as a mouse and keyboard (may include a touch panel, etc.) and a monitor, and a web browser displayed on the monitor can be operated from the data input means.

[0016] Network 4A is the network to which Router 15 and BS40 are connected. The communication method used in Network 4A is referred to as Communication Method 4A. That is, Router 15 and BS40 are equipped with Communication Method 4A. Communication Method 4A is not limited to wired or wireless, and may be any communication method, such as Ethernet, Bluetooth®, WiFi®, or local 5G based on 5G communication (5th Generation communication). If Network 4A is wireless, the connected devices are equipped with antennas to receive radio waves.

[0017] Network 4B refers to the network to which the distribution server 2 and one or more remote terminals 3 are connected. The communication means used in network 4B is referred to as communication means 4B. That is, the distribution server 2 and remote terminals 3 are equipped with communication means 4B. Communication means 4B is not limited to wired or wireless, and may be any communication method such as Ethernet, Bluetooth®, or WiFi®. If network 4B is wireless, the connected devices are equipped with antennas to receive radio waves. In this embodiment, network 4B is assumed to be a general internet network, for example, while network 4A is assumed to be capable of higher capacity communication than network 4B.

[0018] BS40 is a transceiver equipped with a communication means 4A, which receives video data output by the network camera 1 via the network 4A and outputs it to the distribution server 2. Depending on the communication means 4A, BS40 may be, for example, a WiFi access point or a local 5G base station.

[0019] Figure 2 is a configuration diagram showing an example of the configuration of the distribution server functions according to the first embodiment.

[0020] Distribution Server 2 is a server that provides streaming distribution and various web services, and may be composed of a computer equipped with a CPU, memory, etc. Distribution Server 2 has network camera data processing functions, live data distribution functions, and web server functions. Distribution Server 2 also has functions to output distribution data to a web screen, functions to output recorded data to a web screen in a list, and functions to download recorded data. Distribution Server 2 also outputs a web screen for setting and changing various parameters.

[0021] The network camera data processing unit 21 receives video data from the network camera 1, processes the video data according to various setting parameters, and outputs distribution data and recorded data. The network camera data processing unit 21 receives video data from the network camera 1 as stream data using communication protocols such as RTP (Real-time Transport Protocol) or RTSP (Real Time Streaming Protocol).

[0022] The data generation unit 211 outputs the received video data in two parts: distribution data and recorded data. For the distribution data, the data generation unit 211 converts it according to predetermined setting parameters and converts it into a format that can be played back by the network camera live video distribution screen processing unit 221 on the web server. More specifically, the data generation unit 211 compresses the video data according to the setting parameters to generate the distribution data. For example, setting parameters such as the method for compressing video data can be set by the user of the remote terminal 3 from the remote terminal 3, or by the administrator of the distribution server 2 directly on the web server 22.

[0023] Web server 22 is a commonly used web server that provides various web services, such as various screen data, to external devices using communication protocols such as HTTP (Hypertext Transfer Protocol). Web server 22 also streams received stream data in real time.

[0024] The recording data processing unit 23 stores the recording data output by the network camera data processing unit 21 into the storage unit 24 and creates a database of the recording data.

[0025] The network camera live video streaming screen processing unit 221 generates a network camera live video streaming screen and outputs it to a client such as the remote terminal 3. The network camera live video streaming screen processing unit 221 also receives the streaming data output by the network camera data processing unit 21, displays the streaming data on the network camera live video streaming screen, and outputs it to the remote terminal 3.

[0026] The recording data list / download screen processing unit 222 generates a recording data list / download screen that displays a list of recorded data, and outputs it to a client such as the remote terminal 3. The recording data list / download screen displayed on the monitor of the remote terminal 3 may allow the user to select the recording data they wish to download, and the user of the remote terminal 3 can download the selected recording data on the recording data list / download screen to the remote terminal 3 and view it.

[0027] The distribution settings change management screen processing unit 223 generates a screen displaying the parameters that can be set or changed for the distribution server 2, and outputs it to a client such as a remote terminal 3. The user of the remote terminal 3 sets various parameters from the distribution settings change management screen. The distribution settings change management screen processing unit 223 sets the received parameters in the target part of the distribution server 2 or stores them in the storage unit 23.

[0028] The distribution settings change management screen processing unit 223 sets the received parameters in the network camera data processing unit 21. More specifically, the distribution settings change management screen processing unit 223 includes the parameters in a video distribution / recording settings file and outputs it to the network camera data processing unit 21 for setting. The video distribution / recording settings file may include one or more of the following parameters: video bitrate, video resolution, video frame rate, video compression method, audio sampling rate, audio bit depth, audio bitrate, and audio compression method. The network camera data processing unit 21 processes the video image data using the parameters included in the video distribution / recording settings file and generates distribution data or recorded data.

[0029] The compression ratio of the video data may also be included as a parameter in the video distribution / recording setting file. The compression ratio is defined as the ratio of the compressed video data volume to the original video data volume. For example, a compression ratio of 1 means no compression. The compression ratio (or an approximate value thereof) for video data may be achieved by the various parameters described above, or by a combination thereof. For example, the compression ratio and the various parameters for achieving it may be linked and stored as a database in the storage unit 24. In this case, when the network camera data processing unit 21 receives the compression ratio as a setting parameter, it may refer to the database to determine the method for converting the video data.

[0030] The video distribution / recording configuration file 2231 is an example of a video distribution / recording configuration file, and it is an example of instructing the network camera data processing unit 21 to acquire video data from network camera 1 and convert the video data into distribution data with a compression ratio of CR1. At the same time, it is an example of instructing the network camera data processing unit 21 to output the video data acquired from network camera 1 as recording data with a compression ratio of 1 (no compression). If there are multiple video acquisition sources, the video distribution / recording configuration file 2231 may include data that can identify the video acquisition source, such as a unique ID number for network camera 1, in the video acquisition source field.

[0031] The memory unit 24 refers to a storage device that includes non-volatile memory where various types of data are stored. Recording data and various parameters are stored in the memory unit 24.

[0032] The I / F section 25 is an interface that connects to the BS40 for data communication, and is not limited to wired or wireless connections; various cables such as USB may also be connected.

[0033] The I / F unit 26 is an interface equipped with communication means 4B, connected to network 4B, and used for data communication with external devices such as remote terminals 3.

[0034] Figure 3 is a flowchart showing the processing related to the distribution server in the first embodiment.

[0035] The user of remote terminal 3 accesses the distribution settings change management screen provided by the distribution settings change management screen processing unit 223 via a web browser and sets various parameters. The user of remote terminal 3 also accesses the network camera live video distribution screen provided by the network camera live video distribution screen processing unit 211 and requests the distribution server 2 to distribute live video from the network camera. The distribution settings change management screen processing unit 223 outputs and sets the received various parameters in the video distribution / recording settings file to the network camera data processing unit 21. The network camera live video distribution screen processing unit 221 outputs a command to the network camera data processing unit 21 to receive video data based on the video distribution / recording settings file and output distribution data. In this embodiment, the configured video distribution / recording settings file is the video distribution / recording settings file 2231 shown in Figure 2.

[0036] When the network camera data processing unit 21 receives a command to output distribution data (step S100), it receives video data from the network camera 1 as stream data using a communication protocol such as RTP or RTSP, based on the video distribution / recording setting file 2231 (step S101).

[0037] In the network camera data processing unit 21, the data generation unit 211 converts the received stream data to extract video data (step S102) and outputs the recorded data to the recording data processing unit 23 (step S103). The data generation unit 211 obtains the parameters of the video distribution / recording setting file 2231 (distribution data in step S104, step S105) and converts the distribution data based on the parameters (S106).

[0038] More specifically, in step S106, the data generation unit 211 compresses and converts the video data according to the compression ratio CR1, converting it into a format playable on the network camera live video distribution screen provided by the Web server 22 to generate distribution data. The setting parameter, compression ratio CR1, may be set from the remote terminal 3, for example, taking into account the performance of network 4B, so that the amount of data for distribution is sufficient to transmit. Alternatively, the distribution setting change management screen processing unit 223 may calculate the compression ratio CR1 by measuring network 4B and using the measured value. Considering that network 4A has a smaller communication capacity than network 4B, it is desirable to determine the compression ratio CR1 so that the amount of data for distribution is smaller than that of video data. Based on the compression ratio CR1, the data generation unit 211 determines the data compression method, resolution, etc., and performs the compression conversion process.

[0039] The network camera live video streaming screen processing unit 221 displays the generated streaming data on the network camera live video streaming screen (step S107).

[0040] On the other hand, the recording data is stored in the storage unit 24 as is, with the video data transmitted from the network camera 1 (recording data in step S104, S111). The recording data processing unit 23 generates recording data information, which is information about the recording data, along with the recording data, and generates a data list based on the recording data information (step S112), and stores it in the storage unit 24 (step S113). The recording data information in step S112 may be, for example, the name of the recording data.

[0041] By following the above procedure, the user of remote terminal 3 can view the live video streaming screen of the network camera in real time by viewing it via a web browser, thereby enabling real-time viewing of the video output from network camera 3 connected to the high-capacity network 4A. Simultaneously, the user of remote terminal 3 can download recorded data from the recorded data list / download screen provided by the web server 22, and receive and view the original high-definition, large-data-volume video image data output by network camera 1, thereby enabling detailed remote monitoring. In this embodiment, the case where network 4A has a larger communication capacity than network 4B is shown as an example, but this embodiment is not limited to that case and can of course be applied to other situations as well.

[0042] In this embodiment, an example is shown where the compression ratio of video data is set as a parameter, but the video bitrate, video resolution, video frame rate, video compression method, etc., may also be used as parameters. Furthermore, the audio sampling rate, audio bit depth, audio bitrate, audio compression method, etc., for audio data received simultaneously with the video data and video stream may be set as parameters to compress the audio data and reduce the amount of data transmitted. (Second embodiment) The distribution server in this embodiment of the remote monitoring system receives video data from a network camera, processes the video data with different parameters, generates multiple distribution data, and outputs them. For example, it may be a combination of distribution data with high compression using a compression method with a high compression ratio and distribution data with almost no compression using a compression method with a low compression ratio. By allowing the remote terminal 3 connected to a network with low communication capacity to access the highly compressed distribution data, smooth real-time video viewing becomes possible.

[0043] Figure 4 is a configuration diagram showing an example of the configuration of the distribution server functions according to the second embodiment.

[0044] The distribution server 2A is assumed to have the same configuration as the distribution server 2 in Figure 2, except for the network camera data processing unit 21A. Unlike the network camera data processing unit 21, the network camera data processing unit 21A outputs multiple distribution data.

[0045] The network camera data processing unit 21A also converts the received video data using different parameters to generate and output multiple distribution data. The video distribution / recording setting file 2231A is an example of the setting parameters in this embodiment. Based on the settings in the video distribution / recording setting file 2231A, the network camera data processing unit 21A converts the video data received from the network camera 1 using compression ratios CR1 and CR2 to generate and output distribution data 1 and distribution data 2. At the same time, the network camera data processing unit 21 outputs the video data acquired from the network camera 1 as recording data with compression ratio 1 (no compression).

[0046] The distribution server 2A can output multiple distribution data sets as shown in the flowchart in Figure 3. In step S100 of Figure 3, when the parameters shown in the video distribution / recording setting file 2231A are set in the distribution setting change management screen processing unit 223 from the remote terminal 3, and the network camera live video distribution screen processing unit 221 receives a distribution data output command, the network camera data processing unit 21 executes steps S101 to S105 and S111 to S113 for each output data (distribution data 1, distribution data 2, and recording data).

[0047] Furthermore, when the distribution server 2A is outputting multiple distribution data, if only one distribution data is displayed on the monitor of the remote terminal 3, the network camera live video distribution screen processing unit 221 may, for example, display a button on the network camera live video distribution screen that has a function to switch distribution data, and the distribution data can be switched when the user selects the distribution data using the button on the web screen.

[0048] By following the above procedure, the remote terminal 3 can access the video output from the network camera as distributed data with different compression ratios depending on the state of network 4B, for example, enabling smooth real-time viewing of the video. (Third embodiment) The distribution server for the remote monitoring system of this embodiment includes a recording data processing function that converts recorded data into the same format as the distribution data and outputs the redistribution data via stream distribution or the like.

[0049] Figure 5 is a configuration diagram showing an example of the configuration of the distribution server functions according to the third embodiment.

[0050] The distribution server 2B is assumed to have the same configuration as the distribution server 2 in Figure 2, except for the web server 22B and the recording data processing unit 23B. Unlike the recording data processing unit 23, the recording data processing unit 23B re-converts the recording data with desired parameters and outputs the redistribution data. The web server 22B includes a recording data distribution screen processing unit 224 that displays the redistribution data output by the recording data processing unit 23B on the screen and provides it to the client.

[0051] The video distribution / recording setting file 2231B is an example of setting parameters for when recording data 1 stored in the storage unit 24 is converted with a compression ratio of CR1 and output as redistribution data. The video distribution / recording setting file 2231B may also be set from the distribution setting change management screen processing unit 223 to the recording data processing unit 23B. The recording data processing unit 23B acquires the specified video data 1 from the storage unit 24, converts it with a compression ratio of CR1 to generate redistribution data, and outputs it to the recording data distribution screen processing unit 224. The recording data distribution screen processing unit 224 outputs the video from the redistribution data, including it on the web screen.

[0052] Figure 6 is a flowchart showing the processing related to the distribution server in the third embodiment.

[0053] When the parameters of the video distribution / recording setting file 2231B are set in the distribution setting change management screen processing unit 223 from the remote terminal 3, and the recording data distribution screen processing unit 224 receives a redistribution data output command (step S301), the recording data processing unit 23B acquires the setting parameters of the video distribution / recording setting file 2231B (step S302). Based on the setting parameters, the recording data processing unit 23B compresses the recording data 1 acquired from the storage unit 24 with a compression ratio of CR1, converts it to the same format as the distribution data, and outputs it as redistribution data (step S303). The recording data distribution screen processing unit 224 outputs the recording data distribution screen, including the redistribution data, to the remote terminal 3.

[0054] By following the above procedure, the monitor can view past recorded data by viewing the recorded data distribution screen on the monitor of remote terminal 3 using a web browser. (Fourth embodiment) The distribution server 2C of this embodiment of the remote monitoring system has a function to generate and record tag data when a user marks a time or point they want to reconfirm later, or a time or point that requires attention, while viewing real-time video from a remote terminal 3. The distribution server 2C integrates the recorded tag data with the recorded video data and, for example, records only the time vicinity of the tag data, or records only the video area around the point. Tag data is, for example, information about the time or point marked while viewing real-time video.

[0055] Figure 7 is a configuration diagram showing an example of the configuration of the distribution server functions according to the fourth embodiment.

[0056] The distribution server 2C is assumed to have the same configuration as the distribution server 2 in Figure 2, except for the web server 22C and the recording data processing unit 23C.

[0057] Unlike the recording data processing unit 23, the recording data processing unit 23C associates the recorded tag data with the recording data output by the data generation unit 211. The recording data processing unit 23C may also record only the recording data near the time of the associated tag data in the storage unit 24, or record only the data around the point's location (for example, a predetermined rectangular area centered on the point's location) in the storage unit 24.

[0058] The Web server 22C has the same functions as the Web server 22, except for the tag data input / display screen processing unit 225. The tag data input / display screen processing unit 225 displays a tag data input / display screen on the monitor of the remote terminal 3 for the user to mark while viewing real-time video on the remote terminal 3.

[0059] Figure 8 shows an example of a tag data input and display screen generated by the distribution server according to the fourth embodiment.

[0060] Figure 8(a) shows an example where the network camera live video streaming screen 311 and the tag data input / display screen 312 are displayed on the monitor 31 of the remote terminal 3.

[0061] For example, when a user is viewing video on the network camera live video streaming screen 311 of monitor 31 and presses the input button 3122, the time information at the time of pressing is displayed on the tag data input / display screen 3121, and at the same time, it is input as marking information to the tag data input / display screen processing unit 225 and registered as tag data information to the distribution server 2C. The delete button on control button 313 is used to delete registered tag data information. For example, if the delete button is pressed while a time displayed on the tag data input / display screen 312 is selected, the tag data information related to the selected time is deleted, and the display on the tag data input / display screen disappears. Alternatively, if a user on remote terminal 3 directly inputs a time into the time input / display screen 3121 using a keyboard or the like and presses the input button, the entered time information may be registered as tag data information to the distribution server 2. Any method is possible.

[0062] Figure 9 is a flowchart showing the processing of the distribution server according to the fourth embodiment.

[0063] While viewing live video streaming from the network camera on the monitor 31 of the remote terminal 3, the user inputs time information, which is marking information, from the tag data input / display screen 312. When the tag data input / display screen processing unit 225 receives the time information (step S401), it generates tag data information including the time information (step S402). The tag data information may also include any other relevant information in addition to the time information. The recording data processing unit 23C integrates the simultaneously acquired recording data (video image data) and tag data information (step S403) and stores it in the storage unit 24 (step S404).

[0064] Figure 8(b) is an example of a recording data list / download screen used by the user to download recorded data from remote terminal 3.

[0065] The recording data list / download screen processing unit 222 refers to the tag data information integrated into the recording data and displays the recording data name (AAA.mp4) and the time information (xx:yy, xx:zz) that was added while viewing the recording data, as shown in the example of display 3141.

[0066] When a user selects a time, for example by clicking on the time portion of display 3141 (e.g., the xx:yy portion), the recording data list / download screen processing unit 224 extracts data surrounding the selected time (for example, a few minutes before and after) from the recording data (AAA.mp4) in the storage unit 24 and outputs it to the remote terminal 3. How the data surrounding the selected time is selected can be done in various ways; for example, when the user requests to download the recording data, they can set a time range, such as one minute before and after the selected time, from the recording data list / download screen 314, or it can be pre-configured on the distribution server 2C. Any method is possible.

[0067] By following the above procedure, the user of remote terminal 3 can select and download recorded data from a time they wish to review later, allowing them to view and download high-definition video of points of interest with less data usage. (Fifth embodiment) This embodiment The distribution server for the remote monitoring system of this embodiment includes a function to analyze abnormal or suspicious points from high-definition video images received from a network camera, and a function to output only the area around the analyzed points as distribution data.

[0068] Figure 10 is a configuration diagram showing an example of the configuration of the functions of the distribution server according to the fifth embodiment.

[0069] The distribution server 2D is assumed to have the same configuration as the distribution server 2 in Figure 2, except for the network camera data processing unit 21D. Unlike the network camera data processing unit 21, the network camera data processing unit 21D includes an AI image processing unit 212.

[0070] The AI ​​image processing unit 212 detects locations (points) with desired features in the video image data output by the network camera 1 by performing AI (artificial intelligence) processing on the video image data. The AI ​​of the AI ​​image processing unit 212 is assumed to have been pre-trained on the desired features.

[0071] Figure 11 shows an example of a distribution screen generated by the distribution server according to the fifth embodiment.

[0072] The video 310 corresponds to a display image obtained from video data captured by the network camera 1 or uncompressed distribution data output by the data generation unit 211D. When the video 310 data is input to the AI ​​image processing unit 212, the AI ​​image processing unit 212 detects points 3112 with desired features from the video 310 using AI image processing.

[0073] The AI ​​image processing unit 212 outputs the area 3111 surrounding point 3112 as distribution data to the network camera live video distribution screen processing unit 221. The size of the surrounding area 3111 may be predetermined, or it may be specified by the user when requesting distribution from the remote terminal 3.

[0074] The network camera live video streaming screen processing unit 221 displays the received video from the surrounding area 3111 as streaming data on the network camera live video streaming screen and outputs it to the remote terminal 3. The user can view the streaming data from the surrounding area 3111 via a web screen.

[0075] Figure 12 is a flowchart showing the processing of the distribution server according to the fifth embodiment. The processing flow of the distribution server 2D in this embodiment is the same as the flowchart in Figure 3, so we will explain the differences in detail.

[0076] Steps S500 to S504 are the same as steps S100 to S104 in Figure 3, so their explanation will be omitted. However, in the request for distribution data in step S500, information regarding "AI detection enabled / disabled" as shown in the video distribution / recording setting file 2231D in Figure 10 may be included in the request. Furthermore, if AI detection is enabled, the "distribution data size" may also be included in the request. Note that "AI detection enabled / disabled" and "distribution data size" may be fixed in the network camera data processing unit 21D beforehand.

[0077] The network camera data processing unit 21D checks whether "AI detection is enabled / disabled" in the video distribution / recording setting file 2231D (step S505). If AI detection is enabled, the AI ​​image processing unit 212 detects points with pre-learned features in the input video data (or distribution data with compression ratio 1) through AI processing (Yes in step S505, S506). The AI ​​image processing unit 212 extracts the distribution data size (400 x 600 pixels) from the video distribution / recording setting file 2231D, generates rectangular video data of the distribution data size centered on the points detected in step S506, and outputs it to the network camera live video distribution screen processing unit 221 (step S507).

[0078] On the other hand, if AI detection is "none" in step S505, without performing AI processing, the distribution data is generated based on the parameters of the video distribution / recording setting file 2231D, similar to S105 and S106 in Figure 3, and output to the network camera live video distribution screen processing unit 221 (steps S505, S508, and S509). After step S507 or S509, the network camera live video distribution screen processing unit 221 outputs the received distribution data to the network camera live video distribution screen, allowing the user of the remote terminal 3 to view the real-time video image of the network camera 3 displayed on the monitor at a size corresponding to the set distribution data size.

[0079] By following the above procedure, the AI ​​image processing unit 212 analyzes abnormal or suspicious points from the high-definition video images from the network camera and outputs only the video image data around the detected points to the remote terminal 3, thereby reducing the amount of video image data transmitted. As a result, the user of the remote terminal 3 can view important parts of the real-time video from the network camera 1 in high definition, even over the network 4B with its limited communication capacity. (Sixth Embodiment) In this embodiment, when multiple network cameras 1 are connected to a remote monitoring system, the distribution server 2 is shown to have a function that determines the priority for each network camera 1 and switches the setting parameters according to the priority. An example is shown in which two network cameras 1A and 1B are connected. Furthermore, the network 4A in this embodiment is assumed to be a local 5G network.

[0080] Figure 13 is an illustrative diagram showing a remote monitoring system according to the sixth embodiment.

[0081] Multiple network cameras 1A and 1B are connected to routers 15A and 15B, respectively. Network cameras 1A and 1B are assumed to be the same network camera 1, and unless otherwise specified, they are collectively referred to as network camera 1. Similarly, routers 15A and 15B are assumed to be the same router, and unless otherwise specified, they are collectively referred to as router 15. Router 15 in this embodiment is equipped with communication means (including antennas, etc.) for 5G communication. BS40 is a local 5G base station and communicates with router 15 via network 4A, which is a local 5G network. The local 5G network in this embodiment is equipped with a network slicing function that divides the communication bandwidth on network 4A and allocates the communication bandwidth to different communication connections.

[0082] Figure 14 is a configuration diagram showing an example of the configuration of the distribution server functions according to the sixth embodiment.

[0083] Distribution server 2E is an example of a system that has two systems with the same functions as distribution server 2 in Figure 2. Video data received from network camera 1A is processed in the network camera data processing unit 21, recording data processing unit 23, storage unit 24, and I / F unit 25. Video data received from network camera 1B is processed in the network camera data processing unit 21-2, recording data processing unit 23-2, storage unit 24-2, and I / F unit 25-2. The web server 22 and I / F unit 26 are used in common.

[0084] Depending on the performance of network camera 1, the amount of data for the captured video image may differ, and the communication bandwidth required on network 4A may also differ. In such cases, local 5G can use the network slicing function to determine the communication capacity to be allocated to each router 15 on network 4A for communication between each router 15 and BS40. Distribution server 2E obtains information such as the communication capacity allocated to each router 15, i.e., network camera 1, from BS40, etc. (referred to as network camera information).

[0085] Figure 15 is a flowchart showing the first processing of the distribution server according to the sixth embodiment.

[0086] The network camera data processing unit 21 receives a command to output distribution data from the remote terminal 3 (Yes in step S601).

[0087] Figure 16 shows an example of data acquired by the distribution server according to the sixth embodiment. Figures 16(a) and (c) show examples of setting parameters from the user, and Figure 16(b) shows an example of network camera information that has been acquired in advance from, for example, BS40 and stored as a database in the storage unit 24 or the like.

[0088] In step S601 of Figure 15, the network camera data processing unit 21 receives a request for high image quality priority, as shown in Figure 16(a), along with a command to output the distribution data from the network camera 1A (step S602). The network camera data processing unit 21 checks the information on the communication capacity required by the network camera 1A from the network camera information in Figure 16(b) (step S603). The network camera data processing unit 21 determines the compression ratio of the video data received from the network camera 1A based on the required communication capacity of the network camera 1A and the high image quality priority (step S604).

[0089] In step S604, since the image quality priority is "high," for example, the compression ratio may be set to the maximum amount of data that network 4B can tolerate. Alternatively, in step S604, a database linking the required communication capacity, image quality priority, and compression ratio, as well as a database linking the compression ratio and compression method, may be stored in the storage unit 24 or the like, and the network camera data processing unit 21 may retrieve the compression ratio or compression method linked in the database based on the required communication capacity and image quality priority.

[0090] The network camera data processing unit 21 converts the video data received from the network camera 1A using the determined compression ratio, generates distribution data, and outputs it (step S605).

[0091] Following the above procedure, the user of remote terminal 3 can specify the image quality priority and receive video data from network camera 1A. Next, assume that the same remote terminal 3 requests a connection to network camera 1B from distribution server 2E.

[0092] Figure 17 is a flowchart showing the second processing of the distribution server according to the sixth embodiment.

[0093] The network camera data processing unit 21-2 receives a command to output distribution data from the remote terminal 3 along with a request for the image size shown in Figure 16(c) (step S651). The network camera data processing unit 21-2 checks the data in Figures 16(b) and 16(c) (step S652). The network camera data processing unit 21 determines the compression ratio of the video data received from the network camera 1B based on the network slice capacity "N2 (low communication capacity)", which is the required communication capacity of network 4A for the network camera 1B, and the image size "100 × 100 [pixels]" (step S653).

[0094] The network camera data processing unit 21-2 converts the video data received from the network camera 1B using the determined compression ratio to generate and output distribution data (step S654). In step S653, a database linking network slice capacity, image size, and compression ratio, as well as a database linking compression ratio and compression method, may be stored in the storage unit 24 or the like, and the network camera data processing unit 21 may retrieve the network slice capacity, image size, and associated compression ratio and compression method from the database.

[0095] By following the above procedure, it is possible to view video data output from multiple network cameras 1A and 1B from a single remote terminal 3, specifying the image quality and image size. In the above example, the image quality and image size are specified separately, but they can also be specified in combination.

[0096] If the amount of video data output by network cameras 1A and 1B is determined by their respective performance, the communication priority (allocated communication capacity) in network 4A is determined by the local 5G slicing function. For example, in network 4A, network camera 1 with a higher communication priority (e.g., 1A) will have a larger communication capacity than network camera 1 with a lower communication priority (e.g., 1B).

[0097] The communication priority determined by the local 5G slicing function can be called the slicing priority, and the priority determined systematically by the remote monitoring system, such as image quality priority and image size set from the remote terminal 3, can be called the system priority. The system priority may include various parameters related to the video data monitored by the remote terminal 3. It is also possible to specify the slicing priority systematically from the remote terminal 3, in which case the slicing priority may be included in the system priority.

[0098] For example, if you want to view higher-resolution video images of the area you want to monitor (the area covered by each network camera 1), you can increase the system priority or slicing priority for the corresponding network camera 1.

[0099] Furthermore, the network camera data processing unit 21 (including 21-2) may, for example, lower the image quality priority of the system priority as a parameter for higher compression for network camera 1, which has a high slicing priority. This allows the remote terminal 3 to view real-time video output by network camera 1, regardless of the communication capacity of network 4A allocated to network camera 1 or the communication capacity of network 4B to which the remote terminal 3 is connected. (modified version) This modified example is a modification of the fourth embodiment and is another embodiment relating to the distribution server 2C in Figure 7.

[0100] Figure 18 shows an example of a tag data display screen generated by a distribution server related to a modified example.

[0101] Assume that the user of remote terminal 3 is viewing real-time video on monitor 31. When the user finds a point in the real-time video that they want to reconfirm later or that requires attention, they mark it by clicking with a mouse or other device on the network camera live video streaming screen 311. Point 3112A is an example of a marking clicked by the user. By clicking point 3112A, the tag data input / display screen processing unit 225 obtains the time information of point 3112A, along with its location information on the network camera live video streaming screen 311.

[0102] When the tag data input / display screen processing unit 225 detects that a point has been clicked, it generates tag data information including the point's time information and location information, and displays the time information and location information on the tag data display screen 315. (xx:yy) and (xx:zz) show examples of the displayed time information, and (x1, y1) and (x2, y2) show examples of the location information of the point clicked at each time.

[0103] The tag data information generated by the tag data input / display screen processing unit 225 is integrated with the recorded data and stored in the storage unit 24 by a process similar to the flowchart in Figure 9.

[0104] By following the above procedure, similar to the fourth embodiment, the user of the remote terminal 3 can select and download recorded data that contains the time or point they wish to reconfirm later.

[0105] According to at least one embodiment and modification described above, it is possible to provide a distribution server, a remote monitoring system, a remote monitoring method, and a program that utilize multiple networks.

[0106] It should be noted that the present invention is not limited to the embodiments described above, and the components can be modified and implemented in practice without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining the multiple components disclosed in the embodiments. For example, some components may be deleted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined. In addition, the processes shown in flowcharts, sequence charts, etc., may be realized by hardware such as a CPU, IC chip, digital signal processor (DSP), or software (such as a program) that runs on a computer including a microcomputer, or by a combination of hardware and software. Furthermore, even when the claims are expressed as control logic, as a program including instructions for a computer to execute, or as a computer-readable recording medium on which such instructions are written, the apparatus of the present invention is applied. Furthermore, the names and terms used are not limited, and other expressions are included in the present invention if they are substantially the same in content and spirit. [Explanation of symbols]

[0107] 1...Network camera, 2...Distribution server, 3...Remote terminal, 4A...Network, 4B...Network, 15...Router, 21...Network camera data processing unit, 22...Web server, 23...Recording data processing unit, 24...Storage unit, 25...I / F unit, 26...I / F unit, 31...Monitor, 40...BS, 211...Data generation unit, 212...AI screen processing unit, 221...Network camera live video distribution screen processing unit, 222...Recording data list / download screen processing unit, 223...Distribution settings change management screen processing unit, 224...Recording data distribution screen processing unit, 225...Tag data input / display screen processing unit.

Claims

1. A network camera processing unit receives video data output by a network camera in real time, and converts the video data using at least one parameter such as video bitrate, video resolution, video frame rate, video compression method, and image size to generate distribution data. A recording data processing unit that records the aforementioned video data and generates recording data, A web server generates a web screen for changing the parameter settings to change the aforementioned parameters and outputs it to an external device, and outputs the distribution data to the external device in real time. Equipped with, The network camera processing unit generates multiple distribution data converted with different combinations of the parameters, and switches the output distribution data according to the combination of parameters selected from the Web screen for changing the parameter settings on the external device. The Web server outputs the recorded data in response to a download request from the external device. Distribution server.

2. The recording data processing unit re-converts the recording data with one or more of the parameters and outputs the redistribution data. The aforementioned web server generates a web screen that displays the redistributed data and outputs it to the aforementioned external device. The distribution server according to claim 1.

3. The aforementioned Web server receives marking information for assigning tag data to the distributed data while the user is viewing the distributed data in real time, and generates tag data based on the marking information. The recording data processing unit records the tag data and the distribution data to which the tag data has been attached as recording data. The distribution server according to claim 2.

4. The Web server includes a tag data input screen processing unit that outputs a tag data input screen in which the user inputs the marking information. The tag data input screen processing unit receives the time information at the time the button is pressed as the marking information when a button placed on the tag data input screen is pressed. The distribution server according to claim 3.

5. When the user clicks on the display screen of the distribution data that they are viewing in real time, The tag data input screen processing unit receives the time information at the time of the click and the position on the display screen where the click occurred as the marking information. The distribution server according to claim 4.

6. The network camera processing unit extracts pre-learned features from the received video data, cuts out the area surrounding the points possessing the extracted features from the video data, and outputs it as distribution data. A distribution server according to any one of claims 1 to 5.

7. The aforementioned web screen for changing parameter settings includes an input means that allows setting a priority for each network camera. The network camera processing unit generates and outputs the distribution data by switching the above parameters according to the priority. The distribution server according to claim 1.

8. The distribution server described in claim 1, One or more network cameras, One or more remote terminals, The first network to which the aforementioned network camera is connected, The remote terminal is connected to a second network with a smaller communication capacity than the first network. A remote monitoring system equipped with the following features.

9. In the remote monitoring system described in claim 8, The remote terminal outputs a request for distribution of video data output by the network camera from the web browser, along with parameters for converting the video data. The distribution server receives the video data in real time based on the distribution request. The aforementioned video data is converted using the aforementioned parameters to generate distribution data. The aforementioned video data is recorded to generate recorded data, The aforementioned distribution data is output to the remote terminal in real time. In response to a download request from the remote terminal, the recorded data is output. The remote terminal displays the distribution data and the recorded data in the web browser. Remote monitoring method.

10. A procedure for receiving a request from a remote terminal for the distribution of video data output by a network camera, and at least one parameter for converting the video data, including the video bitrate, video resolution, video frame rate, video compression method, and image size, A procedure for generating a web screen for changing the parameter settings and outputting it to the remote terminal, A procedure for receiving the video data in real time based on the aforementioned distribution request, A procedure for generating multiple distribution data by converting the aforementioned video data with different combinations of parameters, and switching the output distribution data according to the combination of parameters selected from the Web screen for changing the parameter settings on the remote terminal, A procedure for recording the aforementioned video data and generating recorded data, A procedure for outputting the aforementioned distribution data to the remote terminal in real time, A procedure for outputting the recorded data in response to a download request from the remote terminal. A program that causes a computer to execute something.

Images