Real-time historical video instant opening method and system based on railway video image specification and readable medium

Abstract

The invention relates to a real-time historical video instant opening method and system based on railway video image specifications and a readable medium, and the method comprises the steps: accessing a video stream in a camera, and coding the video stream; the method comprises the following steps: converting a video stream from a forwarding channel to a new channel through video stream forwarding, caching an I frame in a current GOP before forwarding, extracting the I frame after the new channel is generated, inserting the I frame into the new channel as a first frame, and setting a timestamp as 0; judging whether the video stream in the new channel is a newly forwarded video stream or a historically forwarded video stream; if the video stream is a newly forwarded video stream, inputting the video stream into the client, and decoding and displaying the video stream at the client; and inputting the history forwarding video stream into the storage server, and after time period information is added into the storage server, decoding and displaying the history forwarding video stream at the client. The problem of slow railway video image output is solved, and the video image output speed is improved by improving the system streaming service, the storage service and the playing process on the premise that the GOP value is not reduced under the existing network technical architecture.

Description

Technical Field

[0001] The invention relates to a method, system and readable medium for instantly opening real-time historical video based on railway video image specifications, and belongs to the technical field of video image storage coding and decoding. Background Art

[0002] The railway integrated video surveillance system has important application value in the field of railway transportation. The system has achieved large-scale network coverage of work areas, stations, railway bureaus / companies, and railway companies along the railway. With the development of railway transportation, the role of video surveillance systems in transportation command, public security protection, production operations, and safety monitoring has become increasingly prominent. Due to the system's large spatial coverage and all-weather time characteristics, the railway's requirements for the various functions of the video surveillance system have become increasingly stringent, especially in terms of storage services, which have performance requirements, disaster recovery requirements, and storage quality requirements.

[0003] In 2023, the "Q / CR 575-2022 Railway Integrated Video Surveillance System Technical Specification" (hereinafter referred to as the specification) was officially released, which has more and higher requirements for the railway integrated video surveillance system (hereinafter referred to as the video surveillance system). The specification requires that the video surveillance system must support video cloud computing and video cloud storage technology, and support the unified use and management of computing resources, storage resources, and network resources. The specification has requirements for the logical structure and interface of the video surveillance system, such as Figure 1 shown.

[0004] The railway integrated video system adopts a three-level architecture of access nodes, regional nodes, and core nodes to achieve unified monitoring and dispatching of railway videos across the country, providing indispensable security guarantees for the normal operation of railways. One of the daily tasks of the relevant departments of each railway bureau is to inspect relevant points of the railway through the video system to ensure normal status. In order to improve the inspection efficiency and user experience, the railway video image specification requires that the image be displayed within 3 seconds after clicking the video at the access node. The speed of image display is related to the network speed and the video picture group (GOP). When the network delay is stable, the speed of camera opening and the size of the video GOP value have a direct impact on the decoding output. In order to achieve 3 seconds of output, the GOP value is generally set to 50, and the camera is always in the streaming state.

[0005] The existing technology can use the following methods to improve the speed of video image output: 1) The GOP value can be reduced, but this will increase the video bit rate, and a larger GOP value is conducive to bandwidth saving; 2) The video frame can be cached through the streaming server, and the video data can be sent from the previous I frame when the user requests it. This method can ensure that the first frame can be correctly decoded and quickly output, but it introduces video delay. The larger the GOP value, the greater the possible delay, which does not meet the railway video surveillance system with high real-time requirements. Summary of the invention

[0006] In view of the above problems, the purpose of the present invention is to provide a real-time historical video instant opening method, system and readable medium based on railway video image specifications, which solves the problem of slow video output in the railway video monitoring system. Under the existing network technology architecture, without reducing the GOP value, the video output speed is improved by improving the system streaming service, storage service and playback process.

[0007] To achieve the above-mentioned purpose, the present invention proposes the following technical solutions: a real-time historical video instant opening method based on railway video image specifications, comprising the following steps: accessing the video stream in the camera, encoding the video stream, the encoding including I frames, P frames and B frames, and a video image group GOP between two I frames; converting the video stream from a forwarding channel to a new channel through video stream forwarding, caching the I frames in the current GOP before forwarding, extracting the I frames after the new channel is generated, inserting them into the new channel as the first frame, and setting the timestamp to 0; judging whether the video stream in the new channel is a new forwarded video stream or a historical forwarded video stream; if it is a new forwarded video stream, inputting it into the client, decoding and displaying it on the client; inputting the historical forwarded video stream into the storage service client, decoding and displaying it on the client after the storage service client adds the time period information.

[0008] Furthermore, the method of converting the video stream from the forwarding channel to the new channel through video stream forwarding is: if the currently received video frame is an I frame, the I frame is cached, the timestamp of the I frame is set to 0, and it is inserted into the new channel for forwarding; if the video frame is not an I frame, the cached I frame is directly inserted into the forwarding channel, and the timestamp is set to 0, and chn_state is set to 1 after the I frame insertion is completed; when chn_state=1 is detected, it is determined whether the video frame is a P frame. If so, the P frame is forwarded, and the forwarding channel identifier chn_state is set to 2; when chn_state=2 is detected, the video frame and all subsequent video frames are directly inserted into the new channel; if a forwarding channel full closure signal is received, the stream is closed.

[0009] Further, when chn_state=1 is detected, if the video frame is not a P frame, the video frame is determined to be a B frame, and the B frame is not inserted into the new channel.

[0010] Furthermore, the B frame is a B frame that cannot be decoded, which is a B frame without a forward reference frame, and the forward reference frame is a P frame or an I frame.

[0011] Furthermore, the method for decoding and displaying the newly forwarded video stream on the client is: after clicking play, open the video channel, decode the video in the video channel, determine whether the timestamp of the video is 0, if not, render the image, and determine whether to close after the image rendering. If so, directly determine whether to close. If not, continue video decoding, otherwise end the playback.

[0012] Furthermore, the storage service client stores the historical forwarded video stream in a file in sequence according to the GOP frame structure, and stores the file by indexing the position, type size and timestamp of the corresponding frame of the file. When playing the historical video stream, the start time and end time are input at the same time.

[0013] Furthermore, the method for decoding and displaying the historical forwarded video stream on the client is: receiving a request to play a historical video, retrieving the start time video frame, setting chn_state to 0, reading the video data frame, determining whether chn_state is 1, if so, sending the video data frame, determining whether the end time has been reached, if so, ending, otherwise continuing to read the video data.

[0014] Further, if chn_state is not 1, determine whether the video frame is an I frame. If so, set chn_state to 1, send the data, and then re-read the video data frame; otherwise, search forward for the I frame, set its timestamp to 0, and forward the data; determine whether the video frame of the forwarded data is a P frame. If so, set the chn_state to 1, send the video data frame, and otherwise discard the video data frame and continue to read the video data frame.

[0015] The present invention also discloses a real-time historical video instant opening system of railway video image standard, comprising: a video stream access module, used for accessing the video stream in the camera, encoding the video stream, wherein the encoding includes I frame, P frame and B frame, and a video image group GOP is included between two I frames; a stream forwarding module, used for converting the video stream from the forwarding channel to the new channel through video stream forwarding, before forwarding, caching the I frame in the current GOP, and after the new channel is generated, extracting the I frame, inserting it into the new channel as the first frame, and setting the timestamp to 0; a video stream judgment module, used for judging whether the video stream in the new channel is a new forwarded video stream or a historical forwarded video stream; a new forwarded video processing module, used for inputting the new forwarded video stream into the client, decoding and displaying it at the client; a historical forwarded video processing module, used for inputting the historical forwarded video stream into the storage service client, decoding and displaying it at the client after the storage service client adds the time period information.

[0016] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. The computer program is executed by a processor to implement any of the above-mentioned methods for instant opening of real-time historical videos based on railway video image specifications.

[0017] The technical solution of the present invention has at least the following technical effects or advantages: through the improved design of the client, streaming service and storage service, the present invention solves the problem of improving the image output speed of real-time video and historical video playback without caching the video sequence (only one I frame needs to be cached), increasing the video delay, reducing the GOP value and increasing the bit rate. The image output speed can be shortened from more than 3 seconds to less than 1 second, greatly improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of the logical structure and interface of the railway integrated video monitoring system in the prior art; Figure 2 It is a three-level architecture flow chart of railway integrated video monitoring system in the prior art; Figure 3 It is a schematic diagram of a real-time historical video instant opening method based on railway video image specifications in one embodiment of the present invention; Figure 4 is a schematic diagram of a video stream access forwarding mechanism method in one embodiment of the present invention; Figure 5 is a client decoding and display flow chart of an embodiment of the present invention; Figure 6 This is a schematic diagram of a historical video stream forwarded by a storage service in one embodiment of the present invention; Figure 7 This is a flow chart of a storage service replacing video frames according to time periods in one embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail through specific embodiments. However, it should be understood that the provision of specific embodiments is only for a better understanding of the present invention, and they should not be understood as limitations of the present invention. In the description of the present invention, it should be understood that the terms used are only for the purpose of description and cannot be understood as indicating or implying relative importance.

[0020] In order to solve the problem of slow video output in the railway video monitoring system in the prior art. The present invention proposes a real-time historical video instant opening method, system and readable medium based on the railway video image specification. When the stream is forwarded, the I frame in the current GOP is cached. After the new channel is generated, the cached I frame is taken out and the timestamp is set to 0, and then inserted into the new channel as the first frame; in the above-mentioned stream service distribution process, in order to avoid missing the I frame and causing the entire GOP image to be unable to be decoded, the stream service first inserts an I frame with a timestamp of 0 when forwarding the stream, and discards the undecodable B frame, and divides the video stream into a new forwarded video stream and a historical forwarded video stream, and processes them separately, thereby improving the output speed of real-time video and historical video playback without caching the video sequence, increasing the video delay, reducing the GOP value and increasing the bit rate. The following is a detailed description of the scheme of the present invention through embodiments in conjunction with the accompanying drawings.

[0021] When encoding a video, the video image is divided into three types of frames: I, P, and B. I is an internal coding frame, P is a forward prediction frame, and B is a bidirectional interpolation frame. Simply put, an I frame is a key frame, which can be understood as a complete picture, while P and B frames record the changes relative to the I frame. The P frame represents the difference from the previous frame, and the B frame represents the difference between the previous and next frames. Without an I frame, P and B frames cannot be decoded. The video data between an I frame and the next I frame is a GOP (Group of Pictures). That is, in a video stream, if the previous I frame is missed, the entire GOP will not be decoded until the next I frame arrives. That is, under the conditions of a frame rate of 25 and a GOP value of 50, if an I frame is missed, the image can only be decoded after 2 seconds. The speed at which the camera is turned on is another important factor that affects the speed of image output. The speed at which the camera is turned on varies from manufacturer to manufacturer, generally several seconds. If the camera is turned on every time a stream is acquired, it is impossible to guarantee that the image will be output in 3 seconds. The current solution is to obtain the stream through the storage service and keep the camera always turned on.

[0022] like Figure 2 As shown in the figure, at the access node, the storage service stores the real-time video stream and keeps the camera open. The video playback instructions of the core node will be forwarded to the access node layer by layer, and the video stream will be directly forwarded from the access node stream server to the client of the core node layer by layer. Since the video stream is forwarded through multiple layers, the overall image output speed is generally greater than 3 seconds, and the display delay can be clearly felt.

[0023] Embodiment 1 This embodiment discloses a method for instant opening of real-time historical video based on railway video image specifications, such as Figure 3 As shown, Figure 3 Where I, P, and B represent I frame, P frame, and B frame respectively, and include the following steps: S1 accesses the video stream in the camera and encodes the video stream. The encoding includes I frames, P frames and B frames. A video picture group GOP is included between two I frames.

[0024] S2 forwards the video stream from the forwarding channel to the new channel through video stream forwarding. Before forwarding, it caches the I frame in the current GOP. After the new channel is generated, it extracts the I frame and inserts it into the new channel as the first frame, and sets the timestamp to 0.

[0025] like Figure 4 As shown, the method of converting the video stream from the forwarding channel to the new channel through video stream forwarding is: if the currently received video frame is an I frame, the I frame is cached, the timestamp of the I frame is set to 0, and it is inserted into the new channel for forwarding; if the video frame is not an I frame, the cached I frame is directly inserted into the forwarding channel, and the timestamp is set to 0, and chn_state is set to 1 after the I frame insertion is completed; when chn_state=1 is detected, it is determined whether the video frame is a P frame, if so, the P frame is forwarded, and the forwarding channel identifier chn_state is set to 2; if the video frame is not a P frame, the video frame is determined to be a B frame, and the B frame is not inserted into the new channel; when chn_state=2 is detected, the video frame and all subsequent video frames are directly inserted into the new channel; if the forwarding channel full closure signal is received, the stream is closed.

[0026] In this embodiment, the B frame is an undecodable B frame, which is a B frame without a forward reference frame, and the forward reference frame is a P frame or an I frame. That is, not all B frames cannot be inserted into the new channel, but if the B frame has a forward reference frame, that is, the previous video frame is a P frame or an I frame, then the B frame and the forward P frame or I frame are forwarded to the new channel together. If the B frame has no forward P frame or I frame, it is judged as an undecodable B frame.

[0027] S3 determines whether the video stream in the new channel is a newly forwarded video stream or a historically forwarded video stream; if it is a newly forwarded video stream, it is input into the client, decoded and displayed on the client; the historically forwarded video stream is input into the storage service end, and after the storage service end adds the time period information, it is decoded and displayed on the client.

[0028] In the above stream forwarding process, in order to avoid missing the I frame and causing the entire GOP image to fail to decode, the streaming service first inserts an I frame with a timestamp of 0 when forwarding the stream, and discards the B frame that cannot be decoded. In order to ensure fast image output and continuous playback, such as Figure 5As shown, the method for decoding and displaying the newly forwarded video stream on the client is: after clicking play, open the video channel, decode the video in the video channel, determine whether the timestamp of the video is 0, if not, render the image, and determine whether to close it after the image is rendered. If so, directly determine whether to close it. If not, continue to decode the video, otherwise the playback ends.

[0029] like Figure 6 As shown, the storage service client stores the historical forwarded video stream in the file in sequence according to the GOP frame structure, and stores the file by indexing the position, type size and timestamp of the corresponding frame of the file. When playing the historical video stream, the start time and end time are input at the same time.

[0030] like Figure 7 As shown, the method for decoding and displaying the historical forwarded video stream on the client is as follows: receiving a request to play a historical video, retrieving the start time video frame, setting chn_state to 0, reading the video data frame, judging whether chn_state is 1, if so, sending the video data frame, if chn_state is not 1, judging whether the video frame is an I frame, if so, setting chn_state to 1, sending data, and then re-reading the video data frame; if not, searching for the I frame forward, setting its timestamp to 0, and forwarding the data, the direction of the video stream is backward, and the direction opposite to the video stream is forward, so the forward search is to find out whether there is an I frame in the video frame that has been read; judging whether the video frame of the forwarded data is a P frame, if so, setting chn_state to 1, sending the video data frame, if not, discarding the video data frame and continuing to read the video data frame. Judging whether the end time has been reached, if so, it ends, if not, continue to read the video data.

[0031] Embodiment 2 Based on the same inventive concept, this embodiment discloses a railway video image standard real-time historical video instant opening system, including: The video stream access module is used to access the video stream in the camera and encode the video stream. The encoding includes I frame, P frame and B frame. The video image group GOP is included between two I frames. The stream forwarding module is used to forward the video stream from the forwarding channel to the new channel through video stream forwarding. Before forwarding, the I frame in the current GOP is cached. After the new channel is generated, the I frame is extracted and inserted into the new channel as the first frame, and the timestamp is set to 0. A video stream determination module, used to determine whether the video stream in the new channel is a newly forwarded video stream or a historically forwarded video stream; A newly forwarded video processing module is used to input the newly forwarded video stream into the client, and decode and display it on the client; The historical forwarding video processing module is used to input the historical forwarding video stream into the storage service end, and after the storage service end adds the time period information, it is decoded and displayed on the client end.

[0032] Embodiment 3 Based on the same inventive concept, this embodiment discloses a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement any of the above-mentioned real-time historical video instant opening methods based on railway video image specifications.

[0033] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0034] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0035] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0036] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0037] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not deviate from the spirit and scope of the present invention should be included in the protection scope of the claims of the present invention. The above content is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A real-time historical video instant opening method based on railway video image specifications, characterized in that: The following steps are involved: Accessing a video stream in a camera, encoding the video stream, wherein the encoding includes an I frame, a P frame and a B frame, and a video picture group GOP is included between adjacent I frames; The encoded video stream is forwarded to the new channel through the forwarding channel. Before forwarding, the I frame in the current GOP is cached. After the new channel is generated, the I frame is extracted and inserted into the new channel as the first frame, and the timestamp is set to 0; Determine whether the video stream in the new channel is a newly forwarded video stream or a historically forwarded video stream; If it is the newly forwarded video stream, it is input into the client, decoded and displayed on the client; if it is the historically forwarded video stream, it is input into the storage service end, and after the storage service end adds the time period information, it is transmitted to the client, and decoded and displayed on the client.

2. The method for instant opening of real-time historical video based on railway video image specifications as claimed in claim 1, characterized in that: The method of forwarding the encoded video stream to the new channel through the forwarding channel is: If the currently received video frame is an I frame, the I frame is cached, the timestamp of the I frame is set to 0, and the frame is inserted into the new channel for forwarding; If the video frame is not an I frame, directly take the cached I frame and insert it into the forwarding channel, and set the timestamp to 0. After the I frame is inserted, set chn_state to 1; When chn_state=1 is detected, determine whether the video frame is a P frame, if so, forward the P frame, and set the forwarding channel identifier chn_state to 2; When chn_state=2 is detected, the video frame and all subsequent video frames are directly inserted into the new channel; If a signal indicating that all forwarding channels are closed is received, the flow will be closed.

3. The method for instant opening of real-time historical video based on railway video image specifications as claimed in claim 2, characterized in that: When chn_state=1 is detected, if the video frame is not a P frame, the video frame is determined to be a B frame, and the B frame is not inserted into a new channel.

4. The method for instant opening of real-time historical video based on railway video image specifications as claimed in claim 3, characterized in that: The B frame is a B frame that cannot be decoded, which is a B frame without a forward reference frame, and the forward reference frame is a P frame or an I frame.

5. The method for instant opening of real-time historical video based on railway video image specifications as claimed in claim 3, characterized in that: The method for decoding and displaying the newly forwarded video stream on the client is: after clicking play, open the video channel, decode the video in the video channel, determine whether the timestamp of the video is 0, if not, render the image, and determine whether to close it after the image is rendered. If so, directly determine whether to close it. If not, continue to decode the video, otherwise end the playback.

6. The method for instant opening of real-time historical video based on railway video image specifications as claimed in claim 3, characterized in that: The storage service client stores the historical forwarded video stream in a file in sequence according to the GOP frame structure, and stores the file by indexing the position, type size and timestamp of the corresponding frame of the file. When playing the historical video stream, the start time and end time are input at the same time.

7. The method for instant opening of real-time historical video based on railway video image standards as claimed in claim 6, characterized in that: The method for decoding and displaying the historical forwarded video stream on the client is as follows: receiving a request to play a historical video, retrieving the start time video frame, setting chn_state to 0, reading the video data frame, determining whether chn_state is 1, if so, sending the video data frame, determining whether the end time has been reached, if so, ending, otherwise continuing to read the video data.

8. The method for instant opening of real-time historical video based on railway video image standards as claimed in claim 7, characterized in that: If chn_state is not 1, determine whether the video frame is an I frame. If so, set chn_state to 1, send the data, and then re-read the video data frame; if not, search forward for the I frame, set its timestamp to 0, and then forward the data; It is determined whether the video frame of the forwarded data is a P frame. If so, the chn_state is set to 1 and the video data frame is sent. If not, the video data frame is discarded and the video data frame is continued to be read.

9. A railway video image standard real-time historical video instant opening system, characterized in that: include: A video stream access module is used to access the video stream in the camera and encode the video stream, wherein the encoding includes I frames, P frames and B frames, and a video picture group GOP is included between adjacent I frames; A stream forwarding module is used to forward the encoded video stream to the new channel through the forwarding channel. Before forwarding, the I frame in the current GOP is cached. After the new channel is generated, the I frame is extracted and inserted into the new channel as the first frame, and the timestamp is set to 0. A video stream determination module, used to determine whether the video stream in the new channel is a newly forwarded video stream or a historically forwarded video stream; A newly forwarded video processing module, used for inputting the newly forwarded video stream into the client, decoding and displaying it on the client; The historical forwarding video processing module is used to input the historical forwarding video stream into the storage service end, and after the storage service end adds the time period information, decode and display it on the client end.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the real-time historical video instant opening method based on railway video image specifications as described in any one of claims 1-8.

Images