Information processing method, information processing device, and information processing program

Abstract

This technology improves the compression efficiency of surveillance camera footage without losing frames, and also simplifies the review process. [Solution] The information processing method includes the steps of: acquiring primary video data from a surveillance camera that has been converted into an intra frame which can be decoded independently and is created at regular intervals, and a plurality of predicted frames which are created between the intra frames by referring to past frames; combining the primary video data into one primary video data file in units of a regular interval; creating a secondary video data file by re-converting the primary video data file to shorten the playback time while maintaining the total number of frames; and transmitting the secondary video data file to the terminal device in a streaming or download manner in response to a request from the terminal device for playback.

Description

Technical Field

[0002] , , ,

[0004] , , , , , ,

[0006] , , , , , , , , , , ,

[0005] ,

[0003] ,

[0001] The present invention relates to an information processing method, an information processing apparatus, and an information processing program.

Background Art

[0002] Conventionally, for the purposes of crime prevention, investigation, or management, surveillance cameras have been installed at points to be monitored, such as entrances and exits of buildings such as apartment houses, storefronts, street corners, factories, and distribution centers, and the images of the surveillance cameras have been sent to a terminal device via a network such as the Internet (see, for example, Patent Document 1).

[0003] Since the images of surveillance cameras are not dramas or movies, it is considered that a frame rate of about 10 fps can sufficiently meet the requirements. However, since the frame interval of the 10-fps video becomes long, the efficiency drops even when compressed.

[0004] Although a technique of detecting a person or an object and processing the part other than the detected part (frame) in a highly compressed state has been proposed, in this technique, there is a possibility of deleting an important part (frame) when a person or an object is not detected, and the essence of the surveillance camera, which is to record everything, is not understood. [[ID=​​​​​​​​​​​​​​​​​​​​​ [Means for solving the problem]

[0007] The information processing method according to the first aspect of the present invention is: The steps include: acquiring primary video data from a surveillance camera that has been converted into an intra-frame that can be decoded independently and is created at regular intervals, and a prediction frame that is created multiple times between the intra-frames by referencing past frames; The steps include combining the aforementioned primary video data into a single primary video data file, each timed to a certain duration, The steps include: creating a secondary video data file by re-converting the primary video data file to shorten the playback time while maintaining the total number of frames, The steps include: sending the secondary video data file to the terminal device via streaming or download in response to a request from the terminal device and playing it back; Includes.

[0008] In this configuration, the primary use of surveillance camera footage is to confirm when an object performs some action. When playing back the created secondary video data file, playback is basically fast-forwarded, making the confirmation process easier. Furthermore, since no frames are lost from the primary video data captured by the surveillance camera, even at 1x speed playback (for example, if the primary video data file is converted to 1 / 4 of its playback time (i.e., 4x speed) to create a secondary video data file, playing back the secondary video data file at 1 / 4 speed will result in the same 1x speed normal playback as when the primary video data file is played at 1x speed), 1 / 4 speed playback (for example, if the primary video data file is converted to 1 / 4 of its playback time (i.e., 4x speed) to create a secondary video data file, playing back the secondary video data file at 1 / 16 speed will result in the same 1 / 4 speed slow playback as when the primary video data file is played at 1 / 4 speed), ...the action of the object can be played back without missing anything, and the monitor can confirm it. Furthermore, since secondary video data files have a smaller data size compared to primary video data files, the compression efficiency of surveillance camera footage can be improved.

[0009] An information processing method according to a second aspect of the present invention is an information processing method according to a first aspect, In the step of creating the secondary video data file, the primary video data file is re-encoded by reducing the number of intra frames and predicted frames by creating bidirectional predicted frames that are created by referencing both past and future frames, thereby creating the secondary video data file.

[0010] This configuration allows for further improvement of the compression efficiency of surveillance camera footage.

[0011] An information processing method according to a third aspect of the present invention is an information processing method according to a first or second aspect, The playback time of the secondary video data file is less than or equal to half the playback time of the primary video data file.

[0012] An information processing method according to a fourth aspect of the present invention is an information processing method according to a third aspect, The playback time of the secondary video data file is 1 / 4 or less of the playback time of the primary video data file.

[0013] An information processing method according to a fifth aspect of the present invention is an information processing method according to any of the first to fourth aspects, The frame rate of the aforementioned primary video data is 15fps or less, or 10fps or less, or 5fps or less.

[0014] An information processing device according to a sixth aspect of the present invention is: A means for acquiring primary video data from a surveillance camera that has been converted into an intra-frame that can be decoded independently and is created at regular intervals, and a plurality of predictive frames created between the intra-frames by referencing past frames, A means for combining the aforementioned primary video data into a single primary video data file in units of a certain time period, A means for creating a secondary video data file by re-converting the aforementioned primary video data file to shorten the playback time while maintaining the total number of frames, A means for transmitting the secondary video data file to the terminal device via streaming or download in response to a request from the terminal device and playing it back, It is equipped with.

[0015] The information processing program according to the seventh aspect of the present invention is: The steps include: acquiring primary video data from a surveillance camera that has been converted into an intra-frame that can be decoded independently and is created at regular intervals, and a prediction frame that is created multiple times between the intra-frames by referencing past frames; The steps include combining the aforementioned primary video data into a single primary video data file, each timed to a certain duration, A step of creating a secondary video data file by re-converting the primary video data file while maintaining the total number of frames and shortening the playback time; In response to a request from a terminal device, transmitting the secondary video data file to the terminal device in a streaming manner or a download manner for playback; To be executed by a computer.

Advantages of the Invention

[0016] According to the present invention, the compression efficiency of the video of a surveillance camera can be increased without losing frames, and the confirmation work can be facilitated.

Brief Description of the Drawings

[0017] [Figure 1] FIG. 1 is a diagram showing the overall configuration of a system including an information processing apparatus according to an embodiment. [Figure 2] FIG. 2 is a diagram showing a schematic configuration of an information processing apparatus according to an embodiment. [Figure 3] FIG. 3 is a flowchart showing an example of the operation of an information processing apparatus according to an embodiment. [Figure 4] FIG. 4 is a diagram for explaining an example of primary video data acquired from a surveillance camera. [Figure 5] FIG. 5 is a diagram for explaining an example of a primary video data file stored in a primary video folder. [Figure 6] FIG. 6 is a diagram for explaining an example of a secondary video data file stored in a secondary video folder. [Figure 7] FIG. 7 is a diagram for explaining a process of re-converting a primary video data file to create a secondary video data file.

Embodiments for Carrying Out the Invention

[0018] Embodiments of the present invention will be described in detail below with reference to the attached drawings. In the following description and the drawings used therein, the same reference numerals will be used for parts that can be identically configured, and redundant explanations will be omitted.

[0019] (Device configuration) Figure 1 shows the overall configuration of a system including an information processing device (recording server) 10 according to one embodiment. As shown in Figure 1, the information processing device 10 is connected to one or more terminal devices 20A to 20E and one or more surveillance cameras (network cameras) 30A to 30F via a network such as the Internet. The network may be either a wired or wireless line, and the type and form of the line are not limited. At least a part of the information processing device 10 is implemented by one or more computers.

[0020] Of these, terminal devices 20A to 20E are used by users (monitors or administrators) to view the video from surveillance cameras 30A to 30E, and are electronic devices such as mobile devices like smartphones and tablet devices, notebook computers, or desktop computers. In the example shown in Figure 1, five terminal devices 20A to 20E are shown as representative examples, but the number of terminal devices 20A to 20E is not limited to five; it may be one to four or six or more. In the following, terminal devices 20A to 20E may be denoted by the symbol 20 and described as representative.

[0021] Surveillance cameras 30A to 30F are, in themselves, known network cameras (IP cameras). Surveillance cameras 30A to 30F create primary video data by encoding the captured video into intra-frames (I-frames) that can be decoded independently and are created at regular intervals, and prediction frames (P-frames) that are created multiple times between intra-frames by referencing past frames, and output the created primary video data in a streaming manner. The frame rate of the primary video data is not particularly limited, but may be, for example, 15fps or less, 10fps or less, or 5fps or less. The compression standard for the primary video data is not particularly limited, but may be, for example, H.264 or H.265. In the following, surveillance cameras 30A to 30F may be referred to as 30 and described as representative.

[0022] Next, the configuration of the information processing device (recording server) 10 will be described. Figure 2 is a diagram showing the schematic configuration of the information processing device 10. As shown in Figure 2, the information processing device 10 has a communication unit 11, a control unit 12, and a storage unit 13. Each unit 11 to 13 is connected to each other so as to be able to communicate via a bus or network.

[0023] Of these, the communication unit 11 is a communication interface between the information processing device 10 and an external network. The communication unit 11 transmits and receives information between the information processing device 10 and terminal devices 20A to 20E and surveillance cameras 30A to 30F via an external network such as the Internet.

[0024] The storage unit 13 is a non-volatile data storage device such as flash memory or a hard disk. Various types of data handled by the control unit 12 are stored in the storage unit 13. For example, the storage unit 13 includes a primary video data storage area 13a, a primary video folder 13b, and a secondary video folder 13c.

[0025] Primary video data acquired by the primary video data acquisition unit 12a, described later, is sequentially stored in the primary video data storage area 13a. The primary video data stored in the primary video data storage area 13a may be combined into a primary video data file by the primary video data file creation unit 13b, described later, and after the primary video data file is stored in the primary video folder 13b, it may be deleted from the primary video data storage area 13a.

[0026] The primary video data folder 13b stores the primary video data files created by the primary video data file creation unit 12b, which will be described later. The primary video data files stored in the primary video data folder 13b may be converted back into secondary video data files by the secondary video data file creation unit 13c, which will be described later, and after the secondary video data files are stored in the secondary video folder 13c, they may be deleted from the primary video data folder 13b.

[0027] The secondary video data folder 13c stores the secondary video data files created by the secondary video data file creation unit 12c, which will be described later. The secondary video data files stored in the secondary video data folder 13c are read by the secondary video data file playback unit 12d, which will be described later, and sent to the terminal device 20, where they are played back.

[0028] Furthermore, the storage unit 13 does not necessarily have to be physically located within the information processing device 10; part or all of the storage unit 13 may be located in another device (for example, cloud storage) that is connected to the information processing device 10 via an external network in a communicable manner.

[0029] As shown in Figure 2, the control unit 12 includes a primary video data acquisition unit 12a, a primary video data file creation unit 12b, a secondary video data file creation unit 12c, and a secondary video data file playback unit 12d. Each of these units 12a to 12d may be realized by a processor in the information processing device 10 executing a predetermined program, or it may be implemented in hardware.

[0030] The primary video data acquisition unit 12a acquires primary video data from the surveillance camera 30 in a streaming manner, which has been converted into intraframes (I-frames) that can be decoded independently and are created at regular intervals, and prediction frames (P-frames) that are created in multiple quantities between the intraframes by referring to past frames, and sequentially stores it in the primary video data storage area 13a. Figure 4 is a diagram illustrating an example of primary video data acquired from the surveillance camera 30. In the example shown in Figure 4, the primary video data has a frame rate of 10fps (10 frames per second), and an I-frame is created every second.

[0031] The primary video data file creation unit 12b combines the primary video data acquired by the primary video data acquisition unit 12a into one primary video data file in fixed time intervals, and stores the created primary video data file in the primary video folder 13b. Figure 5 is a diagram illustrating an example of a primary video data file stored in the primary video folder 13b. In the example shown in Figure 5, the primary video data file has a playback time (video duration) of 10 minutes, a total number of frames (10fps × 600 seconds =) 6000 frames, the in-video time of the first frame is 9:00:00, and the in-video time of the last frame is 9:09:59 (that is, the first frame corresponds to video shot at 9:00:00, and the last frame corresponds to video shot at 9:09:59). In other words, this primary video data file has a frame rate of 10fps and a playback time (video duration) between frames of 0.1 seconds.

[0032] The secondary video data file creation unit 12c creates a secondary video data file by re-converting the primary video data file created by the primary video data file creation unit 12b in a way that shortens the playback time while maintaining the total number of frames, and stores the created secondary video data file in the secondary video folder 13c. The compression standard for the secondary video data is not particularly limited, but may be H.264 or H.265, for example. Figure 6 is a diagram illustrating an example of a secondary video data file stored in the secondary video folder 13c, and Figure 7 is a diagram illustrating the process of re-converting the primary video data file to create a secondary video data file. In the examples shown in Figures 6 and 7, the secondary video data file, like the primary video file shown in Figure 5, has a total of 6000 frames, with the in-video time of the first frame being 9:00:00 and the in-video time of the last frame being 9:09:59 (i.e., the first frame corresponds to video shot at 9:00:00 and the last frame corresponds to video shot at 9:09:59). However, the playback time (video duration) is compressed to 2 minutes and 30 seconds, which is shorter than that of the primary video file (one-quarter of the playback time of the primary video file). In other words, this secondary video data file has a playback time (video duration) of 0.025 seconds between frames and a frame rate of 40fps. Therefore, when this secondary video file is played, the video will play at four times the speed compared to when the original primary video file is played. In addition, the data size (file size) of the secondary video data file is smaller than that of the primary video data file. One reason why the amount of data (file size) is smaller is that while individual decodeable intraframes (I-frames) are created at regular intervals (for example, every second), secondary video data files with short playback times have fewer intraframes (I-frames) compared to primary video data files.

[0033] As shown in Figure 6, the secondary video data file creation unit 12c may create a secondary video data file by re-encoding the primary video data file, reducing the number of intra frames (I frames) and prediction frames (P frames) by creating bidirectional prediction frames (B frames) that are created by referencing both past and future frames. This further improves the compression efficiency of the secondary video data file.

[0034] The playback time (video duration) of the secondary video data file may be 1 / 2 or less, or 1 / 4 or less, of the playback time (video duration) of the primary video data file.

[0035] The secondary video data file playback unit 12d, in response to a playback request from the terminal device 20, reads the secondary video data file created by the secondary video data file creation unit 12c from the secondary video folder 13c, and transmits it to the terminal device 20 via streaming or download for playback on the terminal device 20. The primary use of the surveillance camera 30's video is to confirm scenes where objects are performing some action. When the secondary video data file is played back on the terminal device 20, it is played back in fast forward mode (i.e., even if played back at the same playback speed as when playing back the primary video data file (for example, the default setting of 1x speed), the video will play back faster than when playing back the primary video file). Specifically, for example, in the examples shown in Figures 6 and 7, the video taken at 9:00:04 is displayed 1 second after playback of the secondary video data file begins, and the video taken at 9:09:59 (i.e., the video corresponding to the last frame) is displayed 2 minutes and 30 seconds later. This makes it easier for the user to review the video. Furthermore, the secondary video data file maintains the total number of frames from the primary video data file; that is, no frames are lost from the primary video data captured by the surveillance camera 30. Therefore, even at 1x speed playback (for example, if the primary video data file is converted to 1 / 4 of its playback time (i.e., 4x speed) to create a secondary video data file, playing the secondary video data file at 1 / 4 speed will result in the same 1x speed normal playback as when the primary video data file is played at 1x speed), 1 / 4 speed playback (for example, if the primary video data file is converted to 1 / 4 of its playback time (i.e., 4x speed) to create a secondary video data file, playing the secondary video data file at 1 / 16 speed will result in the same 1 / 4 speed slow playback as when the primary video data file is played at 1 / 4 speed), ..., the actions of the objects can be played back without being missed, and the user can confirm this.

[0036] (An example of operation) Next, an example of the operation of the information processing device (recording server) 10 will be described with reference to Figure 3. Figure 3 is a flowchart showing an example of the operation of the information processing device (recording server) 10.

[0037] As shown in Figure 3, first, the primary video data acquisition unit 12a acquires primary video data from the surveillance camera 30 in a streaming manner (see Figure 4) which has been converted into intraframes (I-frames) that can be decoded independently and are created at regular intervals, and prediction frames (P-frames) that are created in multiple quantities between the intraframes by referring to past frames, and sequentially stores it in the primary video data storage area 13a (step S50).

[0038] Next, referring to Figure 5, the primary video data file creation unit 12b combines the primary video data acquired by the primary video data acquisition unit 12a into one primary video data file in increments of a certain time period (for example, 10 minutes), and stores the created primary video data file in the primary video folder 13b (step S51).

[0039] Next, referring to Figures 6 and 7, the secondary video data file creation unit 12c creates a secondary video data file by re-converting the primary video data file created by the primary video data file creation unit 12b to shorten the playback time while maintaining the total number of frames, and stores the created secondary video data file in the secondary video folder 13c (step S52).

[0040] In step S52, the secondary video data file creation unit 12c may create a secondary video data file by re-encoding the primary video data file, thereby reducing the number of intra frames (I frames) and prediction frames (P frames) by creating bidirectional prediction frames (B frames) that are created by referencing both past and future frames.

[0041] Subsequently, when the terminal device 20 sends a request (playback request) to the information processing device (recording server) 10 to play back the video from the surveillance camera 30, the secondary video data file playback unit 12d reads the secondary video data file created by the secondary video data file creation unit 12c from the secondary video folder 13c in response to the playback request from the terminal device 20, and sends it to the terminal device 20 in a streaming or download manner for playback on the terminal device 20 (step S53).

[0042] Incidentally, the primary use of the surveillance camera 30's video is to confirm scenes where objects perform some action. With the above embodiment, when the created secondary video data file is played back on the terminal device 20, it is basically played back in fast forward mode, making it easier for the user to review the video. Furthermore, since no frames are lost from the primary video data captured by the surveillance camera 30, even at 1x speed playback (for example, if the primary video data file is converted to 1 / 4 of its playback time (i.e., 4x speed) to create a secondary video data file, playing the secondary video data file at 1 / 4 speed results in the same 1x speed normal playback as when the primary video data file is played back at 1x speed), 1 / 4 speed playback (for example, if the primary video data file is converted to 1 / 4 of its playback time (i.e., 4x speed) to create a secondary video data file, playing the secondary video data file at 1 / 16 speed results in the same 1 / 4 speed slow playback as when the primary video data file is played back at 1 / 4 speed), ...the actions of objects can be played back without missing any, and the user can confirm them. Furthermore, since the secondary video data file has a smaller data size compared to the primary video data file, the compression efficiency of the video from the surveillance camera 30 can be improved. This allows the video to be transmitted to and played back on the terminal device 20 even when the network is congested and the communication speed is reduced.

[0043] Furthermore, according to this embodiment, when creating a secondary video data file, the primary video data file is re-encoded by reducing the number of intra frames (I frames) and prediction frames (P frames) by creating bidirectional prediction frames (B frames) that are created by referencing both past and future frames, thereby creating the secondary video data file. This further improves the compression efficiency of the video from the surveillance camera 30.

[0044] The above-described embodiments and the disclosure of drawings are merely examples for explaining the invention described in the claims, and the above-described embodiments or the disclosure of drawings do not limit the invention described in the claims. The components of the above-described embodiments can be combined arbitrarily without departing from the spirit of the invention.

[0045] Furthermore, although the information processing device (recording server) 10 according to the above-described embodiment may be composed of one or more computers, the program for realizing the information processing system 10 on one or more computers and the computer-readable recording medium on which the program is recorded non-transitory are also subject to protection in this case. [Explanation of Symbols]

[0046] 10 Information Processing Devices 11 Communications Department 12 Control Unit 12a Primary video data acquisition unit 12b Primary video data file creation section 12c Secondary video data file creation section 12d Secondary Video Data File Playback Unit 13 Storage section 13a Primary video data storage area 13b Primary video folder 13c Secondary video folder 20A~20E Terminal equipment 30A~30F Surveillance Cameras

Claims

1. The steps include: acquiring primary video data from a surveillance camera that has been converted into an intra-frame that can be decoded independently and is created at regular intervals, and a prediction frame that is created multiple times between the intra-frames by referencing past frames; The steps include combining the aforementioned primary video data into a single primary video data file, each portion representing a certain amount of time, The steps include: creating a secondary video data file by re-converting the primary video data file to shorten the playback time while maintaining the total number of frames, The steps include: sending the secondary video data file to the terminal device via streaming or download in response to a request from the terminal device and playing it back; Information processing methods including

2. In the step of creating the secondary video data file, the primary video data file is re-encoded by reducing the number of intra frames and predicted frames by creating bidirectional predicted frames that are created by referencing both past and future frames, thereby creating the secondary video data file. The information processing method according to claim 1.

3. The playback time of the secondary video data file is less than or equal to half the playback time of the primary video data file. The information processing method according to claim 1 or 2.

4. The playback time of the secondary video data file is 1 / 4 or less of the playback time of the primary video data file. The information processing method according to claim 3.

5. A means for acquiring primary video data from a surveillance camera that has been converted into an intra-frame that can be decoded independently and is created at regular intervals, and a plurality of predictive frames created between the intra-frames by referencing past frames, A means for combining the aforementioned primary video data into a single primary video data file in units of a certain time period, A means for creating a secondary video data file by re-converting the aforementioned primary video data file to shorten the playback time while maintaining the total number of frames, A means for transmitting the secondary video data file to the terminal device via streaming or download in response to a request from the terminal device and playing it back, Equipped with an information processing device.

6. The steps include: acquiring primary video data from a surveillance camera that has been converted into an intra-frame that can be decoded independently and is created at regular intervals, and a prediction frame that is created multiple times between the intra-frames by referencing past frames; The steps include combining the aforementioned primary video data into a single primary video data file, each portion representing a certain amount of time, The steps include: creating a secondary video data file by re-converting the primary video data file to shorten the playback time while maintaining the total number of frames, The steps include: sending the secondary video data file to the terminal device via streaming or download in response to a request from the terminal device and playing it back; An information processing program that causes a computer to execute something.

Images